Rendering difference comparison method and system based on multi-channel video coding

Abstract

The invention provides a rendering difference comparison method and system based on multi-channel video coding. Wherein at least two paths of video streams are obtained, and each path of video stream comprises a plurality of frame pictures; coding the frame picture in each path of video stream to generate corresponding coded data; extracting a feature vector of each frame based on the coded data; based on the feature vector of each frame picture, calculating a difference value between the frame pictures through a multi-dimensional comparison algorithm; and determining the difference of the rendering effects of the at least two paths of video streams according to the difference value. The technical scheme provided by the invention can help to quickly detect the nuisance between different video streams, thereby ensuring the quality consistency of the finally output video.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video rendering technology, and in particular to a rendering difference comparison method and system based on multi-channel video encoding. Background Art

[0002] With the rapid development of Internet technology and multimedia content, video content has become an indispensable part of daily life. Especially in the fields of distance education, online conferencing, live entertainment, etc., high-quality video transmission has become essential. In order to support large-scale concurrent viewing, distributed video decoding technology is usually used to reduce the pressure on single-point servers. In a distributed video decoding system, videos may be sent to end users from multiple sources. In order to ensure the consistency of user experience and the stability of video quality, an effective mechanism is needed to monitor and compare the rendering effects between different video streams, especially the quality changes that may occur after the video is transmitted through different paths.

[0003] Currently, common video quality detection methods mainly rely on manual review or simple automated detection tools. These methods can often only detect obvious quality problems, but it is difficult to detect some subtle differences. Summary of the invention

[0004] The embodiments of the present application provide a rendering difference comparison method and system based on multi-channel video encoding, so as to solve the problem of poor video detection efficiency in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a rendering difference comparison method based on multi-channel video encoding, comprising:

[0006] Obtain at least two video streams, each of which contains multiple frames;

[0007] Encode the frame images in each video stream to generate corresponding encoded data;

[0008] Extracting a feature vector of each frame based on the encoded data;

[0009] Based on the feature vector of each frame, a difference value between each frame is calculated by a multi-dimensional comparison algorithm;

[0010] The difference in rendering effect of the at least two video streams is determined according to the difference value.

[0011] Optionally, encoding the frame images in each video stream to generate corresponding encoded data includes:

[0012] Each frame is encoded using different encoding standards to obtain encoded data in different formats;

[0013] Based on the encoded data in different formats, a feature vector of each frame is generated.

[0014] Optionally, extracting a feature vector of each frame based on the encoded data includes:

[0015] Using a deep learning model to extract features from the encoded data, the deep learning model is pre-trained to adapt to multiple video encoding formats;

[0016] Based on the extracted feature information, a feature vector for each frame is constructed.

[0017] Optionally, the calculating the difference value between each frame picture by a multi-dimensional comparison algorithm based on the feature vector of each frame picture includes:

[0018] Define the given two frames f i ,f j The difference between the i , f j ), the difference value D(f i , f j ) is expressed as:

[0019]

[0020] Among them, F k represents the feature vector of the kth feature dimension, F k (f i ) represents the frame image f i The eigenvalue in the kth feature dimension, F k (f j ) represents the frame image f j The eigenvalue on the kth feature dimension, w k is the weight coefficient of the kth feature dimension, N is the number of feature dimensions, Δ motion (f i , f j ) is the motion vector difference between the two frames, β is the weighting factor of the motion vector difference, Δ color (f i , f j ) is the difference in color space between two frames, η is the weighting factor of the color space difference, and ∈ is a small positive number to ensure numerical stability.

[0021] Optionally, the weight coefficient w of the kth feature dimension is k Determined by the following formula:

[0022]

[0023] in, is the average value of the feature vectors of all frames in the kth feature dimension, and α is a regularization parameter.

[0024] Optionally, determining the difference in rendering effect of the at least two video streams according to the difference value includes:

[0025] The difference M(i, j) between video stream i and video stream j is defined as:

[0026]

[0027] in, and are the frames of video stream i and video stream j at time t, Represents the frame images of video stream i and video stream j at the same time t and The difference between them, T is the total number of frames in the video stream, Δ global (i, j) is the difference between video stream i and video stream j in global brightness, contrast and chromaticity, and γ is the weighting factor corresponding to global brightness, contrast and chromaticity.

[0028] Optionally, the difference Δglobal(i, j) in global brightness, contrast and chromaticity between the video stream i and the video stream j is determined by the following formula:

[0029]

[0030] Among them, i and j represent two different video stream indexes, T represents the total number of frames in the video stream. Indicates frame image The brightness value, Indicates frame image The contrast value of Indicates frame image The chromaticity value, Indicates frame image The brightness value, Indicates frame image The contrast value of Indicates frame image The chromaticity value of

[0031] Among them, the difference is used to evaluate the consistency of the rendering effects between the at least two video streams. When M(i, j) is less than a predetermined threshold, it is determined that the rendering effects of video stream i and video stream j are consistent. When M(i, j) is greater than a predetermined threshold, it is determined that there are significant differences in the rendering effects of video stream i and video stream j.

[0032] In a second aspect, an embodiment of the present application provides a rendering difference comparison system based on multi-channel video encoding, including:

[0033] An acquisition module is used to acquire at least two video streams, each of which contains multiple frames;

[0034] A generation module is used to encode the frame images in each video stream to generate corresponding encoded data;

[0035] An extraction module, used for extracting a feature vector of each frame based on the encoded data;

[0036] A comparison module, used for calculating the difference value between each frame through a multi-dimensional comparison algorithm based on the feature vector of each frame;

[0037] A determination module is used to determine the difference in rendering effect of the at least two video streams according to the difference value.

[0038] In a third aspect, a computing device is provided in an embodiment of the present application, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a rendering difference comparison method based on multi-channel video encoding as described in any one of the first aspects.

[0039] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a rendering difference comparison method based on multi-channel video encoding as described in any one of the first aspects.

[0040] In the embodiment of the present application, at least two video streams are obtained, each video stream contains multiple frames; the frames in each video stream are encoded to generate corresponding encoded data; the feature vector of each frame is extracted based on the encoded data; based on the feature vector of each frame, the difference value between each frame is calculated by a multi-dimensional comparison algorithm; and the difference in rendering effect of the at least two video streams is determined according to the difference value. The technical solution provided by the present application can help quickly detect the subtle differences between different video streams, thereby ensuring the quality consistency of the final output video.

[0041] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A flowchart of a rendering difference comparison method based on multi-channel video encoding provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of the structure of a rendering difference comparison system based on multi-channel video encoding provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0047] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0049] Figure 1 A flowchart of a rendering difference comparison method based on multi-channel video encoding is provided for an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0050] 101. Obtain at least two video streams, each of which includes multiple frames;

[0051] In this step:

[0052] Video stream: refers to a continuous data stream that contains video information, which can be understood as a collection of continuous image frames arranged in time order. Video streams can be real-time or read from storage media.

[0053] Frame: The basic unit of a video stream. A frame is a complete image. A video is a dynamic image formed by playing multiple such frames continuously at a certain rate.

[0054] In the embodiment of the present application, it is assumed that we are developing a video conferencing software and need to ensure that the quality of the video streams received by different clients is consistent. To this end, we need to obtain video streams from two different video sources (e.g., two different cameras) and perform subsequent processing.

[0055] Get at least two video streams: In a video conferencing scenario, we can get real-time video streams from two different locations (such as Conference Room A and Conference Room B) through a network interface. Each video stream consists of a series of frames. For example, Camera 1 captures the video of Conference Room A, and Camera 2 captures the video of Conference Room B. The two video sources can be different angles of the same activity, or independent activities in different locations.

[0056] For example, Camera 1 captures 30 frames per second (FPS), and Camera 2 also captures video at 30 FPS. This means that every second, the system will receive a total of 60 frames from both video sources.

[0057] Each video stream contains multiple frames: Continuing with the above video conference example, each video stream is composed of multiple consecutive frames. For example, camera 1 will generate 1800 frames in one minute, and camera 2 will also generate the same number of frames.

[0058] For example, for camera 1, the first frame at the first second may be a panoramic view of conference room A, the second frame may show participant A walking towards the podium, and the third frame may show participant A standing in front of the podium preparing to give a speech, etc.

[0059] In this way, we have completed the first step - obtaining at least two video streams, and understanding that each video stream is composed of multiple frames. The next steps will involve encoding these frames, generating corresponding encoded data, and further extracting the feature vector of each frame.

[0060] 102. Encode the frames in each video stream to generate corresponding encoded data;

[0061] Optionally, the step of encoding the frame images in each video stream to generate corresponding encoded data in step 102 includes:

[0062] Each frame is encoded using different encoding standards to obtain encoded data in different formats;

[0063] Based on the encoded data in different formats, a feature vector of each frame is generated.

[0064] In this step:

[0065] Encoding Process: Video encoding refers to the process of converting raw video data into a more efficient digital format, with the goal of reducing the amount of data for easier transmission and storage. Common video encoding standards include H.264 / MPEG-4AVC (Advanced Video Coding, AVC), H.265 / HEVC (High Efficiency Video Coding, HEVC), etc.

[0066] Encoded Data: Encoded video data, which usually takes up less storage space than the original data and is more suitable for transmission over the network.

[0067] Feature Vector: In machine learning, a feature vector is a set of numerical values ​​that represents the characteristics or attributes of a data sample. In video processing, feature vectors can be used to describe the key features of a frame, such as color distribution, edge strength, etc.

[0068] In the embodiment of the present application, it is assumed that we need to encode the frames in each video stream in a video surveillance system, generate corresponding encoded data, and then extract the feature vector of each frame.

[0069] Different encoding standards are used to encode each frame to obtain encoded data in different formats: Taking the video surveillance system as an example, we can choose two mainstream video encoding standards H.264 / MPEG-4AVC (AVC) and H.265 / HEVC (HEVC) to encode the same frame. Suppose we have a video stream with a frame rate of 30 frames per second, that is, there are 30 frames per second.

[0070] Example: After encoding the first frame using H.264, a 100KB coded data file is obtained; then the same frame is encoded using H.265, and the file size is 80KB. This shows that at the same image quality, the H.265 encoding standard is more efficient than H.264.

[0071] Based on the encoded data in different formats, a feature vector of each frame is generated: After encoding, in order to further analyze the video content, we also need to extract the feature vector of each frame. This step usually involves the application of a deep learning model to extract features through a pre-trained neural network.

[0072] Example: Use the pre-trained ResNet model to extract features from the frames generated by the two different coding standards mentioned above, and obtain two feature vectors. For example, for the frame encoded by H.264, we obtain a feature vector with a length of 1024; and for the frame encoded by H.265, we obtain another feature vector with a length of 1024, although they represent different encoded versions of the same frame.

[0073] Through the above steps, we can not only obtain video data under different coding standards, but also generate feature vectors for subsequent analysis based on these data. Such a process helps to improve the flexibility and accuracy of the video analysis system.

[0074] 103. Extracting a feature vector of each frame based on the encoded data;

[0075] In this step:

[0076] Encoded Data: This refers to the video data processed by the video encoder, which usually includes a series of compressed frames, including key frames (I frames) and non-key frames (P frames / B frames), which are optimized to reduce redundant information, thereby saving storage space and bandwidth.

[0077] Feature Vector: In machine learning or computer vision, a feature vector is a numerical vector extracted from raw data by an algorithm that can represent the main characteristics of the data. In video processing, feature vectors may include color histograms, texture features, shape descriptors, etc.

[0078] Feature Extraction: This is the process of automatically discovering useful features from data. For video data, feature extraction usually involves using a specific algorithm or model (such as a convolutional neural network (CNN)) to analyze the content of each frame and convert it into a feature vector.

[0079] In the embodiment of the present application, it is assumed that we have a video surveillance system that needs to extract features from each frame in the video stream in order to perform subsequent tasks such as anomaly detection or object recognition.

[0080] Encoded data preparation: First, we have encoded each frame in the video stream through the H.264 / MPEG-4AVC (Advanced Video Coding, AVC) encoding standard to obtain a series of encoded data files. For example, we have a video with a frame rate of 30fps and a duration of 1 minute, so there are 1800 frames in total, and each frame is encoded into a separate data packet.

[0081] Feature vector extraction: Next, we will use pre-trained deep learning models, such as VGG16 or ResNet50, to extract feature vectors for each frame. These models are pre-trained on the ImageNet dataset and can recognize many different objects and scene features.

[0082] Specific operation: Load a pre-trained ResNet50 model, which accepts RGB images as input and outputs a fixed-length vector (for example, 2048 dimensions). For each encoded frame, we first decode it back to an RGB image, then input it into the ResNet50 model, and finally get a 2048-dimensional feature vector. In this way, for the 1800 frames of video in the above example, we will get 1800 2048-dimensional feature vector sets.

[0083] Example formula: If we consider a simple feature extraction function F, which maps an image I to a feature vector V, then for the nth frame image I n , we have:

[0084] V n =F(I n )

[0085] Where V n is a 2048-dimensional vector, I n It is the image decoded from the encoded data.

[0086] Through the above steps, we can effectively extract the feature vectors of each frame from the video stream. These feature vectors can be used for further advanced processing, such as video retrieval, behavior recognition, etc.

[0087] 104. Calculate the difference value between each frame by using a multi-dimensional comparison algorithm based on the feature vector of each frame;

[0088] Optionally, the step 104 of calculating the difference value between each frame picture by a multi-dimensional comparison algorithm based on the feature vector of each frame picture includes:

[0089] Define the given two frames f i ,f j The difference between the values ​​D(fi,f j ), the difference value D(fi,f j ) is expressed as:

[0090]

[0091] Among them, F k represents the feature vector of the kth feature dimension, F k (f i ) represents the frame image f i The eigenvalue in the kth feature dimension, F k (f j ) represents the frame image f j The eigenvalue on the kth feature dimension, w k is the weight coefficient of the kth feature dimension, N is the number of feature dimensions, Δ motion (f i , f j ) is the motion vector difference between the two frames, β is the weighting factor of the motion vector difference, Δ color (f i , f j ) is the difference in color space between two frames, η is the weighting factor of the color space difference, and ∈ is a small positive number to ensure numerical stability.

[0092] Optionally, the weight coefficient w of the kth feature dimension in step 104 is k Determined by the following formula:

[0093]

[0094] in, is the average value of the feature vectors of all frames in the kth feature dimension, and α is a regularization parameter.

[0095] In this step:

[0096] Feature Vector: refers to a series of values ​​extracted from a video frame, which is used to represent the main features of the frame, which can be information such as color, texture, shape, etc.

[0097] Multidimensional Comparison Algorithm: This is an algorithm that evaluates the degree of difference between two or more objects in multiple dimensions. In this scenario, it is used to calculate the similarity or difference between different video frames.

[0098] Difference Value: Indicates the difference between two frames in each feature dimension, which can be understood as the distance or similarity score between the two frames.

[0099] Feature Dimension: In a feature vector, each value represents a feature dimension. For example, if the feature vector is 2048-dimensional, it means there are 2048 feature dimensions.

[0100] Weight Coefficient: It is used to measure the importance of different feature dimensions, that is, the proportion of different dimensions when calculating the difference value.

[0101] Motion Vector Difference: Indicates the difference in the movement of an object between two frames, usually obtained by calculating the optical flow field between the two frames.

[0102] Color Space Difference: refers to the difference in color distribution between two frames, which can be quantified by comparing the color histograms or other color models of the two frames.

[0103] Regularization Parameter: It is used to adjust the weight coefficient to prevent overfitting and ensure the stability of numerical calculation.

[0104] In the embodiment of the present application, it is assumed that we have two video frames f1 and f2, and we want to calculate the difference value D(f1, f2) between them.

[0105] Feature vector extraction: For each frame, we have obtained their feature vectors F(f1) and F(f2) through the previous steps, assuming that each vector has 2048 dimensions.

[0106] Difference value calculation: For simplicity, we assume that only the first five feature dimensions are considered and the motion vector difference and color space difference are ignored, that is,

[0107] Δmotion(f1, f2) = 0 and Δ color (f1, f2) = 0

[0108] Suppose F1(f1)=0.5, F1(f2)=0.6, w1=0.3, ∈=0.00001

[0109] Assume F2(f1)=0.4, F2(f2)=0.3, w2=0.2

[0110] And so on until F5.

[0111] Difference value formula application: According to the formula We can calculate:

[0112]

[0113] To calculate the specific value of D(f1, f2), it is necessary to further substitute data from other feature dimensions.

[0114] Weight coefficient determination: Suppose we want to calculate the weight coefficient w1 of the first feature dimension, according to the formula:

[0115]

[0116] Assumptions α = 1, and is the average value of other dimensions, then:

[0117]

[0118] Similarly, all w can be calculated k The value of .

[0119] Through this calculation method, we can get the difference value between two frames, which is very useful in video analysis, anomaly detection and other fields.

[0120] 105. Determine the difference in rendering effects of the at least two video streams according to the difference value.

[0121] Optionally, the determining of the difference in rendering effect of the at least two video streams according to the difference value in step 105 includes:

[0122] The difference M(i, j) between video stream i and video stream j is defined as:

[0123]

[0124] in, and are the frames of video stream i and video stream j at time t, Represents the frame images of video stream i and video stream j at the same time t and The difference between them, T is the total number of frames in the video stream, Δ global (i, j) is the difference between video stream i and video stream j in global brightness, contrast and chromaticity, and γ is the weighting factor corresponding to global brightness, contrast and chromaticity.

[0125] Optionally, the difference Δglobal(i, j) in global brightness, contrast and chromaticity between the video stream i and the video stream j in step 105 is determined by the following formula:

[0126]

[0127] Among them, i and j represent two different video stream indexes, T represents the total number of frames in the video stream. Indicates frame image The brightness value, Indicates frame image The contrast value of Indicates frame image The chromaticity value, Indicates frame image The brightness value, Indicates frame image The contrast value of Indicates frame image The chromaticity value of

[0128] Among them, the difference is used to evaluate the consistency of the rendering effects between the at least two video streams. When M(i, j) is less than a predetermined threshold, it is determined that the rendering effects of video stream i and video stream j are consistent. When M(i, j) is greater than a predetermined threshold, it is determined that there are significant differences in the rendering effects of video stream i and video stream j.

[0129] In this step:

[0130] Video stream: refers to a continuous sequence of video frames. Each video stream represents a video source or a video data stream at a specific perspective.

[0131] Dissimilarity: This is a measure of how different two video streams are in terms of rendering performance, including visual differences between frames and differences in global properties (such as brightness, contrast, and chroma).

[0132] Global Brightness, Contrast, and Chroma: This refers to the average brightness, contrast, and chroma of each frame in the entire video stream. These global properties can help us understand the overall visual style of the video.

[0133] Weight Factor: It is used in the formula to adjust the influence of different factors on the final difference score. For example, γ is used to adjust the influence of global attribute differences on the final difference.

[0134] Predefined Threshold: A standard used to determine whether the rendering effects of two video streams are consistent, usually determined by the application scenario.

[0135] In the embodiment of the present application, assume that we have two video streams i and j, and we want to determine the difference M(i, j) in rendering effects between the two video streams.

[0136] Difference calculation: First, we need to use the formula:

[0137]

[0138] , to calculate the difference between video stream i and video stream j.

[0139] Assume T = 300, that is, each video stream has 300 frames, and γ = 0.5, which means that the global attribute difference has a certain influence on the final difference score. For any time t, Represents the frame images of video streams i and j at the same time t and The difference between .

[0140] Global attribute difference calculation: Then, according to the formula:

[0141]

[0142] , we calculate the difference in global brightness, contrast, and chromaticity between video streams i and j.

[0143] Assumptions Video stream Brightness, contrast, and chrominance values ​​of frame t.

[0144] Likewise, is the corresponding value of the tth frame of video stream j.

[0145] Specific numerical example: To make it concrete, assume The average value of is 0.5, and The average value of is 0.6, then That is, |0.5-0.6|=0.1. The same method can be used to calculate the difference in contrast and chromaticity.

[0146] Comprehensive difference score: Substituting the above values ​​into the difference calculation formula, we can obtain the difference score M(i, j) between video streams i and j.

[0147]

[0148] Where 0.050 and 0.030 are example values ​​for contrast and chromaticity difference values, respectively.

[0149] In this way, we can evaluate the consistency of the rendering effects of the two video streams, and determine whether the video streams are consistent in rendering effects based on whether M(i, j) is less than a predetermined threshold.

[0150] Figure 2 A schematic diagram of the structure of a rendering difference comparison system based on multi-channel video encoding is provided for an embodiment of the present application. Figure 2 As shown, the device comprises:

[0151] An acquisition module 21 is used to acquire at least two video streams, each of which contains multiple frames;

[0152] A generating module 22, used for encoding frames in each video stream to generate corresponding encoded data;

[0153] An extraction module 23, configured to extract a feature vector of each frame based on the encoded data;

[0154] A comparison module 24, configured to calculate the difference value between each frame through a multi-dimensional comparison algorithm based on the feature vector of each frame;

[0155] The determination module 25 is used to determine the difference in rendering effect between the at least two video streams according to the difference value.

[0156] Figure 2 The rendering difference comparison system based on multi-channel video encoding can be performed Figure 1 The implementation principle and technical effect of the rendering difference comparison method based on multi-channel video coding described in the illustrated embodiment will not be described in detail. The specific manner in which each module and unit performs operations in the rendering difference comparison system based on multi-channel video coding in the above embodiment has been described in detail in the embodiment of the method, and will not be described in detail here.

[0157] In one possible design, Figure 2The rendering difference comparison system based on multi-channel video encoding of the embodiment shown can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0158] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0159] The processing component 32 is used to: obtain at least two video streams, each video stream contains multiple frames; encode the frames in each video stream to generate corresponding encoded data; extract the feature vector of each frame based on the encoded data; calculate the difference value between each frame through a multi-dimensional comparison algorithm based on the feature vector of each frame; and determine the difference in rendering effect of the at least two video streams according to the difference value.

[0160] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0161] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0162] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0163] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0164] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0165] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0166] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a rendering difference comparison method based on multi-channel video encoding.

[0167] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0169] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A rendering difference comparison method based on multi-channel video encoding, characterized in that: include: Obtain at least two video streams, each of which contains multiple frames; Encode the frame images in each video stream to generate corresponding encoded data; Extracting a feature vector of each frame based on the encoded data; Based on the feature vector of each frame, a difference value between each frame is calculated by a multi-dimensional comparison algorithm; The difference in rendering effect of the at least two video streams is determined according to the difference value.

2. The method according to claim 1, characterized in that The encoding process of the frame images in each video stream to generate corresponding encoded data includes: Each frame is encoded using different encoding standards to obtain encoded data in different formats; Based on the encoded data in different formats, a feature vector of each frame is generated.

3. The method according to claim 2, characterized in that The extracting of the feature vector of each frame based on the coded data comprises: Using a deep learning model to extract features from the encoded data, the deep learning model is pre-trained to adapt to multiple video encoding formats; Based on the extracted feature information, a feature vector for each frame is constructed.

4. The method according to claim 3, characterized in that The calculating the difference value between each frame picture by a multi-dimensional comparison algorithm based on the feature vector of each frame picture includes: Define the given two frames f i , f j The difference between the values ​​D(f i , f j ), the difference value D(f i , f j ) is expressed as: Among them, F k represents the feature vector of the kth feature dimension, F k (f i ) represents the frame image f i The eigenvalue in the kth feature dimension, F k (f j ) represents the frame image f j The eigenvalue in the kth feature dimension, w k is the weight coefficient of the kth feature dimension, N is the number of feature dimensions, Δ motion (f i , f j ) is the motion vector difference between the two frames, β is the weighting factor of the motion vector difference, Δ color (f i , f j ) is the difference in color space between two frames, η is the weighting factor of the color space difference, and ∈ is a small positive number to ensure numerical stability.

5. The method according to claim 4, characterized in that The weight coefficient w of the kth feature dimension k Determined by the following formula: in, is the average value of the feature vectors of all frames in the kth feature dimension, and α is a regularization parameter.

6. The method according to claim 5, characterized in that The determining, according to the difference value, the difference in rendering effect of the at least two video streams includes: The difference M(i, j) between video stream i and video stream j is defined as: Among them, f t i and f t j are the frames of video stream i and video stream j at time t, D(f t i , f t j ) represents the frame image f of video stream i and video stream j at the same time t t i and f t j The difference between them, T is the total number of frames in the video stream, Δ global (i, j) is the difference between video stream i and video stream j in global brightness, contrast and chromaticity, and γ is the weighting factor corresponding to global brightness, contrast and chromaticity.

7. The method according to claim 6, characterized in that The difference Δglobal(i, j) in global brightness, contrast and chromaticity between the video stream i and the video stream j is determined by the following formula: Among them, i and j represent two different video stream indexes, and t represents the total number of frames in the video stream. Indicates frame image The brightness value, Indicates frame image The contrast value of Indicates frame image The chromaticity value, Indicates frame f t j The brightness value, C(f t j ) represents the frame image f t j The contrast value of Indicates frame image The chromaticity value of Among them, the difference is used to evaluate the consistency of the rendering effects between the at least two video streams. When M(i, j) is less than a predetermined threshold, it is determined that the rendering effects of video stream i and video stream j are consistent. When M(i, j) is greater than a predetermined threshold, it is determined that there are significant differences in the rendering effects of video stream i and video stream j.

8. A rendering difference comparison system based on multi-channel video encoding, characterized in that: include: An acquisition module is used to acquire at least two video streams, each of which contains multiple frames; A generation module is used to encode the frame images in each video stream to generate corresponding encoded data; An extraction module, used for extracting a feature vector of each frame based on the encoded data; A comparison module, used for calculating the difference value between each frame through a multi-dimensional comparison algorithm based on the feature vector of each frame; A determination module is used to determine the difference in rendering effect of the at least two video streams according to the difference value.

9. A computing device, characterized in that It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a rendering difference comparison method based on multi-channel video encoding as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a rendering difference comparison method based on multi-channel video encoding as claimed in any one of claims 1 to 7 is implemented.

Images