Video encoding method, device, electronic device and readable medium

Abstract

The embodiments of the present disclosure disclose a video encoding method, device, electronic device and readable medium. A specific implementation of the method includes: obtaining a sequence of video images to be encoded; for each video image to be encoded, executing the following steps of generating video image encoding information: performing block division processing on the video image to be encoded to obtain a sequence of encoding units; for each encoding unit, executing the following steps: determining initial encoding method information and initial reconstructed image; performing visual perception quality detection on the initial reconstructed image to obtain an image quality score; adjusting the step size and image quality score based on a preset quantization parameter, updating the initial encoding method information to obtain target encoding method information; performing predictive encoding processing on the encoding unit to obtain an image block encoding result; performing entropy encoding processing on each image block encoding result to obtain video image encoding information; generating video encoding information. This implementation can improve the compression efficiency of the video.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a video encoding method, device, electronic device, and readable medium. Background Art

[0002] Video coding methods are technologies used to encode and compress the captured video so that it can be efficiently transmitted and played later. At present, the method usually used for video coding is: for each coding unit divided by the video image, the coding method with the smallest rate-distortion value is selected through the rate-distortion optimization method to encode the coding unit, so as to achieve the goal of a smaller transmission bit rate when the distortion rate is smaller.

[0003] However, it is found in practice that when the above method is used for video encoding, the following technical problems often occur:

[0004] First, since the image distortion involved in the rate-distortion optimization method only depends on the decoded reconstructed image after encoding, and does not take into account the perception requirements of the user's visual system for different image features, when there are isolated points with large differences in the reconstructed image, the image distortion of the coding unit is also high. Furthermore, in order to reduce the distortion of the reconstructed image, it is necessary to increase the transmission bit rate, which leads to low video compression efficiency.

[0005] Second, when encoding each coding unit in a video image, the image features of the coding unit itself are not taken into consideration. As a result, in order to ensure that the quality of each area of ​​the reconstructed image is high, more bit rate resources are occupied when transmitting the video image, resulting in low video compression efficiency.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose video encoding methods, devices, electronic devices, and readable media to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a video encoding method, the method comprising: obtaining a sequence of video images to be encoded; for each video image to be encoded in the above-mentioned sequence of video images to be encoded, performing the following steps of generating video image encoding information: performing block division processing on the above-mentioned video image to be encoded to obtain a sequence of encoding units, wherein each encoding unit corresponds to an image block in the above-mentioned video image to be encoded; for each encoding unit in the above-mentioned sequence of encoding units, performing the following steps: based on a preset candidate encoding method information set, determining initial encoding method information and initial reconstructed image corresponding to the above-mentioned encoding unit, wherein each candidate encoding method information includes an initial quantization parameter; performing visual perceptual quality detection on the above-mentioned initial reconstructed image to obtain an image quality score; based on a preset quantization parameter adjustment step size and the above-mentioned image quality score, updating the above-mentioned initial encoding method information to obtain target encoding method information corresponding to the above-mentioned encoding unit; based on the above-mentioned target encoding method information, performing predictive encoding processing on the above-mentioned encoding unit to obtain an image block encoding result; performing entropy encoding processing on each obtained image block encoding result to obtain video image encoding information corresponding to the above-mentioned video image to be encoded; based on each obtained video image encoding information, generating video encoding information.

[0010] In a second aspect, some embodiments of the present disclosure provide a video encoding device, the device comprising: an acquisition unit, configured to acquire a sequence of video images to be encoded; an execution unit, configured to perform the following steps of generating video image encoding information for each video image to be encoded in the sequence of video images to be encoded: performing block division processing on the video image to be encoded to obtain a sequence of encoding units, wherein each encoding unit corresponds to an image block in the video image to be encoded; for each encoding unit in the sequence of encoding units, performing the following steps: determining initial encoding method information and initial reconstructed image corresponding to the encoding unit based on a preset candidate encoding method information set, wherein: Each candidate encoding method information includes an initial quantization parameter; visual perception quality detection is performed on the above-mentioned initial reconstructed image to obtain an image quality score; based on the preset quantization parameter adjustment step and the above-mentioned image quality score, the above-mentioned initial encoding method information is updated to obtain the target encoding method information corresponding to the above-mentioned encoding unit; based on the above-mentioned target encoding method information, the above-mentioned encoding unit is predictively encoded to obtain an image block encoding result; entropy encoding is performed on each obtained image block encoding result to obtain video image encoding information corresponding to the above-mentioned video image to be encoded; a generation unit is configured to generate video encoding information based on each obtained video image encoding information.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0013] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the video encoding method of some embodiments of the present disclosure, the transmission bit rate can be reduced and the video compression efficiency can be improved. Specifically, the reason for the low video compression efficiency is that the image distortion involved in the rate-distortion optimization method only depends on the decoded reconstructed image after encoding, and does not take into account the perception requirements of the user's visual system for different image features. When there are isolated points with large differences in the reconstructed image, the image distortion of the encoding unit is also high. Further, in order to reduce the distortion of the reconstructed image, it is necessary to increase the transmission bit rate, thereby resulting in low video compression efficiency. Based on this, the video encoding method of some embodiments of the present disclosure first obtains a sequence of video images to be encoded. Thus, a video clip to be encoded and compressed can be obtained. Then, for each video image to be encoded in the above-mentioned video image sequence to be encoded, the following video image encoding information generation steps are performed: the above-mentioned video image to be encoded is subjected to block division processing to obtain a coding unit sequence, wherein each coding unit corresponds to an image block in the above-mentioned video image to be encoded; for each coding unit in the above-mentioned coding unit sequence, the following steps are performed: based on a preset candidate coding method information set, the initial coding method information and the initial reconstructed image corresponding to the above-mentioned coding unit are determined, wherein each candidate coding method information includes an initial quantization parameter; visual perceptual quality detection is performed on the above-mentioned initial reconstructed image to obtain an image quality score; based on the preset quantization parameter adjustment step size and the above-mentioned image quality score, the above-mentioned initial coding method information is updated to obtain the target coding method information corresponding to the above-mentioned coding unit; based on the above-mentioned target coding method information, the above-mentioned coding unit is subjected to predictive coding processing to obtain an image block coding result; and the obtained image block coding results are subjected to entropy coding processing to obtain the video image coding information corresponding to the above-mentioned video image to be encoded. Therefore, for each coding unit obtained by dividing the video image to be encoded, the target coding method can be determined according to the rate-distortion optimization method first, and then the initial quantization parameter can be adjusted according to the visual perception-based quality score of the decoded reconstructed image to ensure that the initial quantization parameter used for data compression is reduced without reducing the image quality, and the transmission bit rate after image encoding is further reduced. Finally, based on the obtained encoding information of each video image, video encoding information is generated. Therefore, the video encoding method of some embodiments of the present disclosure, by combining the rate-distortion optimization method based on image distortion with the image quality detection method based on visual perception, can further determine a smaller transmission bit rate according to the user's visual perception requirements when there are isolated points with large differences in the reconstructed image. Thereby, the compression efficiency of the video is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flowchart of some embodiments of the video encoding method according to the present disclosure;

[0016] Figure 2 is a schematic structural diagram of some embodiments of a video encoding device according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1A process 100 of some embodiments of a video encoding method according to the present disclosure is shown. The video encoding method comprises the following steps:

[0025] Step 101: Obtain a video image sequence to be encoded.

[0026] In some embodiments, the execution subject of the video encoding method (e.g., an encoder) can obtain a video image sequence to be encoded from a video shooting device through a wired connection or a wireless connection. The video image sequence to be encoded can be a sequence of video frames to be encoded and compressed arranged in chronological order. The video image sequence to be encoded can represent a video clip.

[0027] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0028] Step 102: for each video image to be encoded in the video image sequence to be encoded, perform the following steps of generating video image encoding information:

[0029] Step 1021, performing block division processing on the video image to be encoded to obtain a coding unit sequence.

[0030] In some embodiments, the execution subject may perform block division processing on the video image to be encoded to obtain a coding unit sequence. Each coding unit may correspond to an image block in the video image to be encoded. The coding unit sequence may be an ordered set of coding units arranged in a raster scan order. The coding units in the coding unit sequence may be CTU (Coding Tree Unit) coding tree units or CU (Coding Unit) coding units. The video image to be encoded may be subjected to block division processing by the tree-structured block division technology of HEVC (High Efficiency Video Coding) to obtain a coding unit sequence. It should be noted that the tree structure is not limited to a binary tree, a ternary tree or a quadtree.

[0031] Step 1022: for each coding unit in the coding unit sequence, perform the following steps:

[0032] Step 10221: Determine initial encoding method information and initial reconstructed image corresponding to the encoding unit based on a preset candidate encoding method information set.

[0033] In some embodiments, the execution subject may determine the initial coding method information and the initial reconstructed image corresponding to the coding unit based on a preset candidate coding method information set in various ways. Each candidate coding method information may include an initial quantization parameter. The initial quantization parameter may be a pre-set unoptimized quantization parameter. The quantization parameter may be a scaling factor for controlling the quantization processing operation during video encoding. The quantization parameter reflects the compression of spatial details. Each candidate coding method information in the candidate coding method information set may also include prediction mode information. The prediction mode information may include a prediction mode identifier and prediction mode association information. The prediction mode identifier may be a unique identifier of the prediction mode. The prediction mode may be intra-frame prediction or inter-frame prediction. For example, when the prediction mode is intra-frame prediction, the prediction mode association information may be information of an intra-frame prediction algorithm. When the prediction mode is inter-frame prediction, the prediction mode association information may be information of an inter-frame prediction algorithm and a reference frame. The initial coding method information may be information of a candidate coding method with the smallest rate-distortion optimization result. The initial reconstructed image may be an image reconstructed after video image encoding and decoding according to the initial coding method information.

[0034] In some optional implementations of some embodiments, the execution subject may determine the initial encoding method information and the initial reconstructed image corresponding to the encoding unit based on a preset candidate encoding method information set through the following steps:

[0035] In the first step, for each candidate encoding method information in the above candidate encoding method information set, the following rate-distortion value generation steps are performed:

[0036] The first sub-step is to encode the coding unit based on the candidate coding method information to obtain a coded data stream. The coded data stream may be a binary data stream obtained by encoding and compressing the original image block corresponding to the coding unit. The original image block may be an image block corresponding to the coding unit in the video image to be encoded. The coding unit may be encoded based on the candidate coding method information to obtain a coded data stream in various ways.

[0037] In some optional implementations of some embodiments, the execution subject may perform encoding processing on the encoding unit based on the candidate encoding method information through the following encoding processing steps to obtain an encoded data stream:

[0038] Step 1: Based on the candidate coding method information, the coding unit is subjected to image prediction processing to obtain a predicted image block. The predicted image block may be an image block corresponding to the coding unit predicted by a prediction mode. The coding unit may be subjected to image prediction processing according to the prediction mode information included in the candidate coding method information to obtain a predicted image block.

[0039] Step 2: Determine the residual matrix between the original image block and the predicted image block corresponding to the above coding unit. The elements in the above residual matrix are the differences between the pixels in the original image block and the pixels at the corresponding positions in the predicted image block. First, determine the residual image between the original image block and the predicted image block corresponding to the above coding unit by performing a pixel-by-pixel difference operation. Then, determine each pixel in the residual image as a matrix element to obtain a residual matrix.

[0040] Step 3: Perform frequency domain transformation on the residual matrix to obtain a residual frequency domain coefficient matrix. The residual frequency domain coefficient matrix can be a frequency domain representation of the residual matrix. The residual matrix can be transformed into a frequency domain by discrete cosine transform to obtain a residual frequency domain coefficient matrix.

[0041] Step 4, based on the initial quantization parameter included in the candidate coding method information, the residual frequency domain coefficient matrix is ​​quantized to obtain a quantized frequency domain coefficient matrix. The elements in the quantized frequency domain coefficient matrix can be the result of scaling the elements at corresponding positions in the residual frequency domain coefficient matrix. First, for each element in the residual frequency domain coefficient matrix, the result of rounding down the quotient of the element and the initial quantization parameter can be determined as the quantized frequency domain coefficient. Then, the matrix composed of the quantized frequency domain coefficients corresponding to the residual frequency domain coefficient matrix is ​​determined as the quantized frequency domain coefficient matrix.

[0042] Step 5: Generate coding unit associated data based on the division identifier corresponding to the coding unit, the candidate coding method information and the quantization frequency domain coefficient matrix. The division identifier can indicate whether the coding unit is divided again. The coding unit associated data can be the data required for decoding and reconstructing the image coding corresponding to the coding unit. For example, when the division identifier is 0, the coding unit is no longer divided further; when the division identifier is 1, the coding unit is further divided into at least 4 coding units. The division identifier corresponding to the coding unit, the quantization frequency domain coefficient matrix, the initial quantization parameter and the prediction mode identifier included in the candidate coding method information can be determined as coding unit associated data.

[0043] Step 6: Perform entropy coding on the data associated with the coding unit to obtain a coded data stream corresponding to the coding unit. The entropy coding may be, but is not limited to, one of the following: Huffman coding, variable length coding, and arithmetic coding.

[0044] The second sub-step is to decode and reconstruct the above-mentioned coded data stream to obtain a decoded image. The above-mentioned decoded image may be an image obtained by decoding and reconstructing the image coding corresponding to the above-mentioned coding unit. The above-mentioned coded data stream may be decoded and reconstructed by the decoder corresponding to the encoder in HEVC to obtain a decoded image. The decoding and reconstruction process may include inverse quantization, inverse frequency domain transformation, image prediction, and reconstructed image. The reconstructed image may be the result of superimposing the inverse frequency domain transformation result and the image prediction result on the same position pixel.

[0045] The third sub-step is to determine the number of each binary digit included in the above-mentioned encoded data stream as the transmission code rate.

[0046] The fourth sub-step is to determine the image distortion between the original image block corresponding to the coding unit and the decoded image. The image distortion can represent the degree of image distortion. The image distortion between the original image block and the decoded image can be determined according to the error of each pixel through the error square sum formula.

[0047] The fifth sub-step is to generate a rate-distortion value corresponding to the coding unit based on the transmission bit rate and the image distortion. The rate-distortion value can characterize the relationship between the transmission bit rate and the image distortion. First, the Lagrangian factor is determined based on the mapping relationship between the initial quantization parameter and the Lagrangian factor. Then, the rate-distortion value corresponding to the coding unit is determined based on the transmission bit rate, the image distortion and the Lagrangian factor through the rate-distortion optimization formula.

[0048] In the second step, a rate-distortion value satisfying a preset rate-distortion optimization condition is selected from the obtained rate-distortion values ​​as a target rate-distortion value, wherein the preset rate-distortion optimization condition may be that the rate-distortion value is the maximum value among the rate-distortion values.

[0049] In a third step, the candidate encoding method information corresponding to the target rate-distortion value is determined as the initial encoding method information corresponding to the encoding unit.

[0050] The fourth step is to determine the decoded image corresponding to the above initial encoding method information as the initial reconstructed image.

[0051] Step 10222: Perform visual perception quality detection on the initial reconstructed image to obtain an image quality score.

[0052] In some embodiments, the execution subject may perform visual perception quality detection on the initial reconstructed image in various ways to obtain an image quality score, wherein the image quality score may be a quality score of the initial reconstructed image in a visual perception dimension.

[0053] In the process of adopting technical solutions to solve the technical problems in the background technology, the following technical problems are often accompanied: when using a quality detection method based on visual perception to detect the quality of an image, how to extract the visual features of the image for quality detection. In response to the above technical problems, the conventional solution is generally to extract visual features by dividing the image into blocks. However, the above conventional solution still has the following problems: the way of extracting visual features by dividing the image into blocks easily ignores the connection between each image block, making the visual feature extraction inaccurate, resulting in difficulty in generating more accurate image quality detection results. Taking into account the problems of the above conventional solutions and combining the advantages of deep learning, the following solutions can be adopted:

[0054] In some optional implementations of some embodiments, the execution subject may perform visual perception quality detection on the initial reconstructed image to obtain an image quality score through the following steps:

[0055] In the first step, the initial reconstructed image is input into the image decomposition module included in the pre-trained image quality detection model to obtain the image low-frequency subband, the image horizontal high-frequency subband, the image vertical high-frequency subband and the image diagonal high-frequency subband. Among them, the image quality detection model can be a classification model based on subjective visual perception with a video image as input and an image quality score as output. The image quality detection model can also include a multi-scale feature fusion module and a classification module. The image decomposition module can be used to convert the input image from the spatial domain to the frequency domain using discrete wavelet transform to obtain frequency domain image content of different scales. The multi-scale feature fusion module can be used to fuse features of different scales of the image. The multi-scale feature fusion module can also include a first feature extraction submodule, a second feature extraction submodule, a third feature extraction submodule, a fourth feature extraction submodule and a feature fusion submodule. The first feature extraction submodule can be used to extract features from the image low-frequency subband. The second feature extraction submodule can be used to extract features from the image horizontal high-frequency subband. The third feature extraction submodule can be used to extract features from the image vertical high-frequency subband. The fourth feature extraction submodule can be used to extract features from the diagonal high-frequency subband of the image. The feature fusion submodule is used to fuse the extracted features. The classification module can be used to classify and score the input feature map. The spatial sizes of the image low-frequency subband, the image horizontal high-frequency subband, the image vertical high-frequency subband and the image diagonal high-frequency subband increase in sequence. For example, the spatial sizes of the image low-frequency subband, the image horizontal high-frequency subband, the image vertical high-frequency subband and the image diagonal high-frequency subband can be 32×32, 64×64, 128×128 and 256×256, respectively.

[0056] In the second step, the image low-frequency subband, the image horizontal high-frequency subband, the image vertical high-frequency subband and the image diagonal high-frequency subband are respectively input into the first feature extraction submodule, the second feature extraction submodule, the third feature extraction submodule and the fourth feature extraction submodule included in the multi-scale feature fusion module to obtain the first visual feature map, the second visual feature map, the third visual feature map and the fourth visual feature map. The spatial sizes of the first visual feature map, the second visual feature map, the third visual feature map and the fourth visual feature map are the same. For example, the spatial sizes of the first visual feature map, the second visual feature map, the third visual feature map and the fourth visual feature map can all be 4×4. The first feature extraction submodule can include 2 standard convolution modules and 1 residual block. The second feature extraction submodule can include 2 standard convolution modules and 2 residual blocks. The third feature extraction submodule can include 2 standard convolution modules and 3 residual blocks. The fourth feature extraction submodule can include 2 standard convolution modules and 4 residual blocks. The above-mentioned standard convolution module may include a normal convolution layer, a BN (Batch Normalization) batch normalization layer and a LeakyReLU (Leaky Rectified Linear Unit) activation function layer. Each residual block may be a residual block of a CSP1_X structure including one residual unit. CSP1_X may be a CSP (Cross Stage Partial) module with a residual connection. The residual unit may be a bottleneck structure. The above-mentioned first feature extraction submodule may sequentially perform a standard convolution operation, a residual connection and a standard convolution operation on the above-mentioned low-frequency subband of the image. The above-mentioned second feature extraction submodule may sequentially perform a standard convolution operation, two residual connections and a standard convolution operation on the above-mentioned horizontal high-frequency subband of the image. The above-mentioned third feature extraction submodule may sequentially perform a standard convolution operation, three residual connections and a standard convolution operation on the above-mentioned vertical high-frequency subband of the image. The fourth feature extraction submodule may sequentially perform a standard convolution operation, four residual connections and a standard convolution operation on the diagonal high-frequency subband of the image.

[0057] In the third step, the first visual feature map, the second visual feature map, the third visual feature map and the fourth visual feature map are input into the feature fusion submodule included in the multi-scale feature fusion module to obtain a first fused feature map. The first fused feature map may be the result of aggregating shallow features and deep features of the image. The feature fusion submodule may perform feature fusion on the first visual feature map, the second visual feature map, the third visual feature map and the fourth visual feature map by feature concatenation to obtain a first fused feature map.

[0058] In the fourth step, feature extraction is performed on the first fused feature map to obtain a second fused feature map. The second fused feature map may be a result of further extracting all features of the image. A convolution operation and a standard convolution operation may be performed on the first fused feature map in sequence to obtain the second fused feature map.

[0059] In the fifth step, the second fused feature map is input into the classification module included in the image quality detection model to obtain the image quality score. The classification module can sequentially perform a pooling operation, a full connection operation and a classification method based on a support vector machine on the second fused feature map to obtain the image quality score.

[0060] Optionally, the above image quality detection model can be pre-trained through the following steps:

[0061] The first step is to obtain a video sample set. Each video sample in the video sample set may include a sample reconstructed video image sequence and a sample video image quality score sequence. The degree of distortion corresponding to each sample reconstructed video image sequence included in the video sample set may be different. The sample video image sequence may represent a decoded and reconstructed video clip. The sample reconstructed video image in the sample reconstructed video image sequence may correspond to the sample video image quality score at the corresponding position in the sample video image quality score sequence. The sample video image quality score in the sample video image quality score sequence may be a quality score of the corresponding sample reconstructed video image in the user's subjective visual perception dimension.

[0062] The second step is to select video samples from the above video sample set, and perform the following training steps based on the selected video samples:

[0063] In the first sub-step, each sample reconstructed video image included in the selected video sample is input into an initial image quality detection model to obtain a video image quality score corresponding to each sample reconstructed video image in the above-mentioned sample reconstructed video images.

[0064] The second sub-step is to determine the difference between the quality scores of each sample video image and the quality scores of each video image corresponding to each sample reconstructed video image based on a preset loss function. The loss function may be the result of weighted summation of the first loss function and the second loss function. The first loss function may be a mean square error loss function. The second loss function may be a quality score versus average loss function. The difference value may characterize the difference between the true score and the model score of each sample reconstructed video image. The loss function may be defined by the following formula:

[0065]

[0066] Wherein, L represents the loss function. L1 represents the mean square error loss function. L2 represents the quality score comparison average loss function. n represents the number of sample reconstructed video images included in the selected sample. i and j represent the index numbers of the sample reconstructed video images in the sample reconstructed video image sequence. S represents the quality score of the sample video image. Represents the video image quality score output by the model. S i Represents the sample video image quality score corresponding to the i-th sample reconstructed video image. S represents the video image quality score corresponding to the video image reconstructed by the i-th sample. j Represents the sample video image quality score corresponding to the j-th sample reconstructed video image. Indicates the video image quality score corresponding to the jth sample reconstructed video image. L C Represents the quality score comparison loss between any two sample reconstructed video images. represents the quality score contrast loss between the video image reconstructed according to the lth sample and the video image reconstructed according to the jth sample. θ(·) represents the indicator function.

[0067] The third sub-step is, in response to determining that the difference value is less than a preset difference threshold, determining the trained initial image quality detection model as the image quality detection model, wherein the preset difference threshold may be a preset upper limit of the difference value.

[0068] Optionally, the above-mentioned execution entity may also adjust the network parameters of the initial image quality detection model in response to determining that the above-mentioned difference value is greater than or equal to the above-mentioned preset difference threshold, and reselect the video sample, and use the adjusted initial image quality detection model as the initial image quality detection model to perform the above-mentioned training step again.

[0069] As an example, the above-mentioned execution entity can adjust the network parameters of the initial image quality detection model through the stochastic gradient descent method, adaptively adjust the learning rate through the cosine annealing strategy, and train the initial image quality detection model for 200 to 250 rounds until the model converges.

[0070] The above technical solution and its related contents, as an inventive point of the embodiment of the present disclosure, solve the technical problem of "difficult to generate relatively accurate image quality detection results". The factors that make it difficult to generate relatively accurate image quality detection results are often as follows: the way of extracting visual features by dividing the image into blocks easily ignores the connection between each image block, making the visual feature extraction inaccurate, resulting in difficulty in generating relatively accurate image quality detection results. If the above factors are solved, the effect of improving the accuracy of the image quality detection results can be achieved. In order to achieve this effect, the above-mentioned execution subject performs a quality score on the reconstructed image through the above-mentioned image quality detection model. First, considering the different sensitivities of the human visual system to image components of different frequencies, the image can be decomposed into image contents of different scales through the image decomposition module included in the above-mentioned image quality detection model, thereby simulating the spatial frequency characteristics of the human visual system. Then, through the multi-scale feature fusion module included in the above-mentioned image quality detection model, the spatial frequency domain features of the image contents of different scales are extracted, and the image features of different scales are scale-unified and fused, thereby, the shallow features with a large amount of texture and detail information can be fused with the deep features with strong semantics and contour information. Finally, the more comprehensive image features after fusion are extracted again, and the image quality score corresponding to the reconstructed image is determined according to the extracted fusion features through the classification module included in the above-mentioned image quality detection model. Therefore, through the above-mentioned image quality detection model, there is no need to pay attention to the connection or difference between each image block, and the more accurate shallow and deep information of the image visual features can be obtained through the decomposition module and the multi-scale feature fusion module, and on this basis, a more accurate quality score of the reconstructed image can be further determined, which is convenient for the subsequent adjustment of the encoding compression method according to the visual perception quality results. In addition, when training the above-mentioned image quality detection model, by introducing the quality score comparison average loss function, it is possible to enrich the supervision information and guide the quality score regression calculation, thereby accelerating the convergence speed of the image quality detection model.

[0071] Step 10223, based on the preset quantization parameter adjustment step size and image quality score, the initial encoding method information is updated to obtain the target encoding method information corresponding to the encoding unit.

[0072] In some embodiments, the execution subject may update the initial encoding method information based on the preset quantization parameter adjustment step and the image quality score in various ways to obtain the target encoding method information corresponding to the encoding unit. The preset quantization parameter adjustment step may be a preset value that is increased each time the initial quantization parameter is adjusted. The target encoding method information may be the initial encoding method information including the optimization of the initial quantization parameter. The target encoding method information may be an encoding method that ensures a relatively balanced transmission bit rate and image distortion.

[0073] In some optional implementations of some embodiments, the execution subject may update the initial encoding method information based on a preset quantization parameter adjustment step size and the image quality score to obtain the target encoding method information corresponding to the encoding unit through the following steps:

[0074] The first step is to determine the reference quality score corresponding to the coding unit based on the coding unit region type identifier corresponding to the coding unit and the image quality score. The coding unit region type identifier can indicate whether the coding unit corresponds to the image background area. The reference quality score can indicate the quality of the video image reconstructed by the decoder. The following steps are performed:

[0075] In a first sub-step, in response to determining that the coding unit region type identifier corresponding to the coding unit is an unclassified identifier, the image quality score is determined as a reference quality score corresponding to the coding unit, wherein the unclassified identifier may indicate that the foreground region and the background region are not distinguished.

[0076] In a second sub-step, in response to determining that the coding unit region type identifier corresponding to the coding unit is a background region identifier, the difference between the image quality score and the preset adjustment score is determined as a reference quality score corresponding to the coding unit. The background region identifier indicates that the pixels in the corresponding image block are all pixels in the background region of the image. The preset adjustment score may be a value for adjusting the image quality score. For example, the preset adjustment score may be 5.

[0077] In a third sub-step, in response to determining that the coding unit region type identifier corresponding to the coding unit is a foreground region identifier, the image quality score is determined as a reference quality score corresponding to the coding unit, wherein the foreground region identifier indicates that the corresponding image block is a background region of the image.

[0078] In the second step, the sum of the initial quantization parameter included in the initial encoding method information and the preset quantization parameter adjustment step is determined as the optimized quantization parameter.

[0079] The third step is to use the above-mentioned initial coding method information as the optimized coding method information, and add the optimized coding method information to the preset optimized coding method information sequence. Among them, the above-mentioned optimized coding method information sequence can be a sequence of variable length. The above-mentioned optimized coding method information sequence can be empty when it is initially created. The optimized coding method information can be added to the end of the preset optimized coding method information sequence in an appended manner. It should be noted that when determining the target coding method information corresponding to the next coding unit, it is necessary to clear the various optimized coding method information corresponding to the previous coding unit in the optimized coding method information sequence.

[0080] In the fourth step, based on the optimized quantization parameters, the following target encoding method information generation steps are performed:

[0081] The first sub-step is to replace the initial quantization parameter included in the initial encoding method information based on the optimized quantization parameter to obtain the optimized encoding method information.

[0082] The second sub-step is to encode the coding unit based on the optimized coding method information to obtain an optimized coded data stream. The optimized coded data stream may be a binary data stream obtained by encoding and compressing the original image block corresponding to the coding unit through the optimized coding method information. It should be noted that the specific implementation of this step can refer to the coding processing step in the above step 10221, which will not be repeated here.

[0083] The third sub-step is to decode the above-mentioned optimized coded data stream to obtain an optimized decoded image. The above-mentioned optimized decoded image may be an image obtained by decoding and reconstructing the optimized coded data stream corresponding to the above-mentioned coding unit. It should be noted that the specific implementation of this step can refer to the decoding and reconstruction processing step in the above-mentioned step 10221, which will not be repeated here.

[0084] The fourth sub-step is to perform visual perception quality detection on the above-mentioned optimized decoded image to obtain an optimized image quality score. The above-mentioned optimized image quality score may be a quality score of the above-mentioned optimized decoded image in a visual perception dimension. It should be noted that the specific implementation of this step may refer to the above-mentioned step 10222, which will not be repeated here.

[0085] In a fifth sub-step, in response to determining that the optimized image quality score is less than the reference quality score, the optimized coding method information satisfying a preset position condition is selected from the optimized coding method information sequence as the target coding method information, wherein the preset position condition may be that the optimized coding method information is the last element in the optimized coding method information sequence.

[0086] Optionally, the above execution entity may further perform the following sub-steps:

[0087] Sub-step 1: In response to determining that the optimized image quality score is greater than or equal to the reference quality score, the optimized encoding method information corresponding to the optimized image quality score is added to the optimized encoding method information sequence. The optimized encoding method information corresponding to the optimized image quality score can be added to the end of the optimized encoding method information sequence in an appended manner.

[0088] Sub-step 2: Based on the preset quantization parameter adjustment step, the optimized quantization parameter is updated to obtain an updated quantization parameter, and the updated quantization parameter is used as the optimized quantization parameter to perform the target encoding method information generation step again. The updated quantization parameter may be the sum of the optimized quantization parameter and the preset quantization parameter adjustment step.

[0089] Optionally, the coding unit region type identifier corresponding to the above coding unit may be pre-generated by the following steps:

[0090] The first step is to determine the video type information corresponding to the video image sequence to be encoded. The video type information can indicate whether the video clip is a surveillance video. The video type information corresponding to the video image sequence to be encoded can be determined in various ways.

[0091] In some optional implementations of some embodiments, the execution subject may determine the video type information corresponding to the to-be-encoded video image sequence through the following steps:

[0092] Step 1, sampling the above-mentioned video image sequence to be encoded to obtain a sampled video image sequence. The above-mentioned sampled video image sequence can be a sequence obtained by uniformly sampling the video image sequence to be encoded. First, the length of the above-mentioned video image sequence to be encoded is determined as the number of video images. Secondly, the result of rounding down the quotient of the above-mentioned video image number and the preset sampling number is determined as the sampling interval length value. The above-mentioned preset sampling number can be a preset number to be sampled. Then, according to the above-mentioned sampling interval length value, the above-mentioned video image sequence to be encoded is non-overlappingly divided to obtain a sampling interval subsequence set. Afterwards, each sampling interval subsequence in the above-mentioned sampling interval subsequence set is randomly sampled to obtain a sampled video image set. Finally, the sampled video image set is sorted according to the time sequence corresponding to the sampled video images to obtain a sampled video image sequence.

[0093] Step 2: Perform background extraction processing on each sampled video image in the sampled video image sequence to obtain a background image sequence. The background image in the background image sequence may be an image of a background area corresponding to the sampled video image. A background extraction processing may be performed on each sampled video image in the sampled video image sequence using a preset background extraction processing method to obtain a background image sequence. For example, the background extraction processing method may be, but is not limited to, at least one of the following: an image segmentation algorithm based on edge detection, a foreground-background separation algorithm based on deep learning.

[0094] Step 3: Perform similarity analysis on each background image in the above background image sequence to obtain an image similarity value set. The image similarity values ​​in the above image similarity value set can represent the similarity between any two adjacent background images. First, perform denoising and enhancement on each background image using a filter-based denoising method to obtain an enhanced image sequence. Then, perform similarity measurement on any two adjacent background images in the above enhanced image sequence using a structural similarity index method to obtain an image similarity value set.

[0095] Step 4: In response to determining that the above-mentioned image similarity value set satisfies a preset similarity condition, a preset monitoring type identifier is determined as the video type information. The above-mentioned preset similarity condition may be: the ratio of the number of image similarity values ​​greater than the preset similarity value to the number of elements in the above-mentioned image similarity value set is greater than a preset ratio. The above-mentioned preset similarity value may be a preset similarity value. The above-mentioned preset ratio may be a preset percentage. The above-mentioned preset monitoring type identifier may indicate that the video image sequence to be encoded is a monitoring type video clip.

[0096] Step 5: In response to determining that the image similarity value set does not meet the preset similarity condition, a preset non-monitoring type identifier is determined as the video type information. The preset non-monitoring type identifier may indicate that the video image sequence to be encoded is not a surveillance video segment.

[0097] In the second step, in response to determining that the video type information satisfies the preset video type condition, the video image to be encoded corresponding to the encoding unit is subjected to foreground and background separation processing to obtain a binary video image. The preset video type condition may be that the video type information is the same as the preset monitoring type identifier. The binary video image may be a black and white image that distinguishes the foreground from the background. The video image to be encoded corresponding to the encoding unit may be subjected to foreground and background separation processing by a preset foreground and background separation processing method to obtain a binary video image. For example, the foreground and background separation processing method may be, but is not limited to, at least one of the following: a background subtraction model, a Gaussian mixture background segmentation algorithm.

[0098] In the third step, in response to determining that the original image block corresponding to the coding unit matches the background area corresponding to the binary video image, a preset background area identifier is determined as the coding unit area type identifier corresponding to the coding unit. Matching the background area corresponding to the binary video image may mean that the area corresponding to the coding unit is completely included in the background area corresponding to the binary video image.

[0099] In the fourth step, in response to determining that the original image block corresponding to the coding unit matches the foreground area corresponding to the binary video image, a preset foreground area identifier is determined as the coding unit area type identifier corresponding to the coding unit. The matching of the foreground area corresponding to the binary video image may be: overlapping with the foreground area corresponding to the binary video image, or the area corresponding to the coding unit is completely included in the foreground area corresponding to the binary video image.

[0100] Optionally, the execution subject may also determine a preset unclassified identifier as the coding unit region type identifier corresponding to the coding unit in response to determining that the video type information does not meet the preset video type condition.

[0101] The above-mentioned reference quality score generation step and its related contents, as an inventive point of an embodiment of the present disclosure, solve the technical problem 2 mentioned in the background technology, "low video compression efficiency". The reasons for the low video compression efficiency are often as follows: when encoding each coding unit in the video image, the image features of the coding unit itself are not considered, so that in order to ensure that the quality of each area of ​​the reconstructed image is high, more bit rate resources are occupied when transmitting the video image. If the above-mentioned problem is solved, the effect of reducing the bit rate and improving the video compression efficiency can be achieved. In order to achieve this effect, first, it is determined whether the video clip corresponding to the video image sequence to be encoded is a surveillance type video. Then, different quality scores are set according to the image area features of the video. When the video clip is a surveillance type video, according to the characteristics that the background of the surveillance video is usually relatively fixed, a lower quality score is determined for the background area image, and a higher quality score is determined for the foreground area image. Among them, the setting of the preset adjustment score needs to ensure that the image after adjustment is still relatively clear although the distortion is reduced compared with the existing reconstructed image. When the video clip is not a surveillance type video, for example, an entertainment video, a higher quality score can be set. Finally, when it is determined that the quality is unchanged, the bit rate can be reduced by increasing the quantization parameter. Therefore, the occupation of bit rate resources during video image transmission can be reduced while ensuring the quality of each area of ​​the reconstructed image, thereby improving the video compression efficiency.

[0102] Step 10224, based on the target encoding method information, perform predictive encoding processing on the encoding unit to obtain an image block encoding result.

[0103] In some embodiments, the execution subject may perform predictive coding on the coding unit based on the target coding method information to obtain an image block coding result. The image block coding result may be a result of residual coding of the original image block corresponding to the coding unit. The coding process may include image prediction, calculation of residual matrix, residual frequency domain transformation and quantization.

[0104] Step 1023, entropy coding is performed on the obtained encoding results of each image block to obtain video image encoding information corresponding to the video image to be encoded.

[0105] In some embodiments, the execution subject may perform entropy coding on the obtained encoding results of each image block to obtain video image encoding information corresponding to the video image to be encoded. The video image encoding information may be the result of encoding and compressing the corresponding video image to be encoded. The following steps are specifically performed:

[0106] In the first step, for each image block encoding result in each image block encoding result, entropy encoding is performed on the image block encoding result and other decoding-related information of the corresponding encoding unit by a preset entropy encoding method to obtain a compression result of the encoding unit. The other decoding-related information may be information required to decode the image encoding corresponding to the encoding unit. The other decoding-related information may include, but is not limited to, a quantization parameter and a prediction mode identifier.

[0107] In the second step, the compression results of each coding unit are sorted according to the arrangement order of the coding unit sequence corresponding to the video image to be encoded through a preset sorting algorithm, and the sorted results are determined as the video image encoding information corresponding to the above-mentioned video image to be encoded.

[0108] Step 103: Generate video encoding information based on the obtained encoding information of each video image.

[0109] In some embodiments, the execution subject may generate video coding information based on the obtained video image coding information. The video coding information may be the result of encoding and compressing the video clips corresponding to the video image sequence to be encoded. The video image coding information may be sorted according to the order of the video image sequence to be encoded corresponding to each video image coding information by a preset sorting algorithm, and the sorted result may be determined as the video coding information. For example, the sorting algorithm may be, but is not limited to, one of the following: bubble sort, quick sort.

[0110] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: through the video encoding method of some embodiments of the present disclosure, the transmission bit rate can be reduced and the video compression efficiency can be improved. Specifically, the reason for the low video compression efficiency is that the image distortion involved in the rate-distortion optimization method only depends on the decoded reconstructed image after encoding, and does not take into account the perception requirements of the user's visual system for different image features. When there are isolated points with large differences in the reconstructed image, the image distortion of the encoding unit is also high. Further, in order to reduce the distortion of the reconstructed image, it is necessary to increase the transmission bit rate, thereby resulting in low video compression efficiency. Based on this, the video encoding method of some embodiments of the present disclosure first obtains a sequence of video images to be encoded. Thus, a video clip to be encoded and compressed can be obtained. Then, for each video image to be encoded in the above-mentioned video image sequence to be encoded, the following video image encoding information generation steps are performed: the above-mentioned video image to be encoded is subjected to block division processing to obtain a coding unit sequence, wherein each coding unit corresponds to an image block in the above-mentioned video image to be encoded; for each coding unit in the above-mentioned coding unit sequence, the following steps are performed: based on a preset candidate coding method information set, the initial coding method information and the initial reconstructed image corresponding to the above-mentioned coding unit are determined, wherein each candidate coding method information includes an initial quantization parameter; visual perceptual quality detection is performed on the above-mentioned initial reconstructed image to obtain an image quality score; based on the preset quantization parameter adjustment step size and the above-mentioned image quality score, the above-mentioned initial coding method information is updated to obtain the target coding method information corresponding to the above-mentioned coding unit; based on the above-mentioned target coding method information, the above-mentioned coding unit is subjected to predictive coding processing to obtain an image block coding result; and the obtained image block coding results are subjected to entropy coding processing to obtain the video image coding information corresponding to the above-mentioned video image to be encoded. Therefore, for each coding unit obtained by dividing the video image to be encoded, the target coding method can be determined according to the rate-distortion optimization method first, and then the initial quantization parameter can be adjusted according to the visual perception-based quality score of the decoded reconstructed image to ensure that the initial quantization parameter used for data compression is reduced without reducing the image quality, and the transmission bit rate after image encoding is further reduced. Finally, based on the obtained encoding information of each video image, video encoding information is generated. Therefore, the video encoding method of some embodiments of the present disclosure, by combining the rate-distortion optimization method based on image distortion with the image quality detection method based on visual perception, can further determine a smaller transmission bit rate according to the user's visual perception requirements when there are isolated points with large differences in the reconstructed image. Thereby, the compression efficiency of the video is improved.

[0111] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a video encoding device, and these device embodiments are Figure 1Corresponding to the method embodiments shown, the video encoding device 200 can be specifically applied to various electronic devices.

[0112] like Figure 2 As shown, the video encoding device 200 of some embodiments includes: an acquisition unit 201, an execution unit 202 and a generation unit 203. The acquisition unit 201 is configured to acquire a sequence of video images to be encoded; the execution unit 202 is configured to perform the following steps of generating video image encoding information for each video image to be encoded in the above-mentioned sequence of video images to be encoded: performing block division processing on the above-mentioned video image to be encoded to obtain a sequence of encoding units, wherein each encoding unit corresponds to an image block in the above-mentioned video image to be encoded; for each encoding unit in the above-mentioned sequence of encoding units, performing the following steps: based on a preset candidate encoding method information set, determining the initial encoding method information and the initial reconstructed image corresponding to the above-mentioned encoding unit, wherein each candidate encoding method information includes Initial quantization parameter; performing visual perception quality detection on the above-mentioned initial reconstructed image to obtain an image quality score; based on the preset quantization parameter adjustment step size and the above-mentioned image quality score, updating the above-mentioned initial encoding method information to obtain the target encoding method information corresponding to the above-mentioned encoding unit; based on the above-mentioned target encoding method information, performing predictive encoding processing on the above-mentioned encoding unit to obtain an image block encoding result; performing entropy encoding processing on each obtained image block encoding result to obtain video image encoding information corresponding to the above-mentioned video image to be encoded; the generation unit 203 is configured to generate video encoding information based on each obtained video image encoding information.

[0113] It can be understood that the units described in the video encoding device 200 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the video encoding device 200 and the units included therein, and will not be described in detail here.

[0114] Further references Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0115] like Figure 3As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0116] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0117] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0118] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0119] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0120] The above-mentioned computer-readable medium may be included in the above-mentioned device; or it may exist independently and not be assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a sequence of video images to be encoded; for each video image to be encoded in the above-mentioned video image sequence to be encoded, performs the following video image encoding information generation steps: performs block division processing on the above-mentioned video image to be encoded to obtain a coding unit sequence, wherein each coding unit corresponds to an image block in the above-mentioned video image to be encoded; for each coding unit in the above-mentioned coding unit sequence, performs the following steps: based on a preset candidate coding method information set, determines the initial coding method information and the initial reconstruction image corresponding to the above-mentioned coding unit Image, wherein each candidate encoding method information includes an initial quantization parameter; performing visual perception quality detection on the above-mentioned initial reconstructed image to obtain an image quality score; based on a preset quantization parameter adjustment step and the above-mentioned image quality score, updating the above-mentioned initial encoding method information to obtain the target encoding method information corresponding to the above-mentioned encoding unit; based on the above-mentioned target encoding method information, performing predictive encoding processing on the above-mentioned encoding unit to obtain an image block encoding result; performing entropy encoding processing on each obtained image block encoding result to obtain video image encoding information corresponding to the above-mentioned video image to be encoded; based on each obtained video image encoding information, generating video encoding information.

[0121] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0123] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be arranged in a processor, for example, may be described as: a processor includes: an acquisition unit, an execution unit, and a generation unit. The names of these units do not constitute a limitation on the units themselves in some cases, for example, the acquisition unit may also be described as a "unit for acquiring a video image sequence to be encoded".

[0124] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0125] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A video encoding method, comprising: Obtaining a video image sequence to be encoded; For each video image to be encoded in the sequence of video images to be encoded, the following video image encoding information generation steps are performed: Performing block division processing on the video image to be encoded to obtain a coding unit sequence, wherein each coding unit corresponds to an image block in the video image to be encoded; For each coding unit in the coding unit sequence, the following steps are performed: Based on a preset candidate encoding method information set, determine initial encoding method information and initial reconstructed image corresponding to the encoding unit, wherein each candidate encoding method information includes an initial quantization parameter, the initial encoding method information is information of a candidate encoding method with the smallest rate-distortion optimization result, and the initial reconstructed image is an image reconstructed after encoding and decoding the video image according to the initial encoding method information; Performing visual perception quality detection on the initial reconstructed image to obtain an image quality score; Based on a preset quantization parameter adjustment step size and the image quality score, the initial encoding method information is updated to obtain target encoding method information corresponding to the encoding unit; Based on the target encoding method information, performing predictive encoding processing on the encoding unit to obtain an image block encoding result; Performing entropy coding processing on the obtained encoding results of each image block to obtain video image encoding information corresponding to the video image to be encoded; Generate video encoding information based on the obtained encoding information of each video image; The updating process of the initial encoding method information based on the preset quantization parameter adjustment step size and the image quality score to obtain the target encoding method information corresponding to the encoding unit includes: Based on the coding unit region type identifier corresponding to the coding unit and the image quality score, determining a reference quality score corresponding to the coding unit, wherein the coding unit region type identifier indicates whether the coding unit corresponds to the image background region, and the step of determining the reference quality score corresponding to the coding unit includes: In response to determining that the coding unit region type identifier corresponding to the coding unit is an unclassified identifier, determining the image quality score as a reference quality score corresponding to the coding unit; In response to determining that the coding unit region type identifier corresponding to the coding unit is a background region identifier, determining a difference between the image quality score and a preset adjustment score as a reference quality score corresponding to the coding unit; In response to determining that the coding unit region type identifier corresponding to the coding unit is a foreground region identifier, determining the image quality score as a reference quality score corresponding to the coding unit; Determine the sum of the initial quantization parameter included in the initial encoding method information and the preset quantization parameter adjustment step as the optimized quantization parameter; The initial coding method information is used as the optimized coding method information, and the optimized coding method information is added to a preset optimized coding method information sequence, and the added optimized coding method information sequence is obtained as an updated optimized coding method information sequence, wherein the optimized coding method information sequence is a sequence whose length changes with the addition of the optimized coding method information; Based on the optimized quantization parameters, the following target encoding method information generation steps are performed: Based on the optimized quantization parameter, the initial quantization parameter included in the initial encoding method information is replaced to obtain the optimized encoding method information after the replacement process as the optimized encoding method information after the replacement process; Based on the optimized encoding method information after the replacement process, the encoding unit is encoded to obtain an optimized encoded data stream; Decoding and reconstructing the optimized coded data stream to obtain an optimized decoded image; Performing visual perception quality detection on the optimized decoded image to obtain an optimized image quality score; In response to determining that the optimized image quality score is less than the reference quality score, the optimized encoding method information that meets the preset position condition is selected from the updated optimized encoding method information sequence as the target encoding method information.

2. The method according to claim 1, wherein: The method further comprises: In response to determining that the optimized image quality score is greater than or equal to the reference quality score, adding the replaced optimized encoding method information corresponding to the optimized image quality score to the updated optimized encoding method information sequence; Based on the preset quantization parameter adjustment step, the optimized quantization parameter is updated to obtain an updated quantization parameter, and the updated quantization parameter is used as the optimized quantization parameter to perform the target encoding method information generation step again.

3. The method according to any one of claims 1-2, wherein: The determining, based on a preset candidate encoding method information set, initial encoding method information and an initial reconstructed image corresponding to the encoding unit comprises: For each candidate encoding method information in the candidate encoding method information set, the following rate-distortion value generation step is performed: Based on the candidate encoding method information, encoding the encoding unit to obtain an encoded data stream; Decoding and reconstructing the encoded data stream to obtain a decoded image; Determining the number of each binary digit included in the coded data stream as a transmission bit rate; Determining an image distortion between an original image block corresponding to the coding unit and a decoded image, wherein the original image block is an image block corresponding to the coding unit in the video image to be coded; Based on the transmission bit rate and the image distortion, generating a rate-distortion value corresponding to the coding unit; Selecting a rate-distortion value that satisfies a preset rate-distortion optimization condition from the obtained rate-distortion values ​​as a target rate-distortion value; Determining the candidate encoding method information corresponding to the target rate-distortion value as the initial encoding method information corresponding to the encoding unit; The decoded image corresponding to the initial encoding method information is determined as the initial reconstructed image.

4. The method according to claim 3, wherein: The step of performing encoding processing on the encoding unit based on the candidate encoding method information to obtain an encoded data stream includes: Based on the candidate encoding method information, performing image prediction processing on the encoding unit to obtain a predicted image block; Determine a residual matrix between an original image block and a predicted image block corresponding to the coding unit; Performing frequency domain transformation on the residual matrix to obtain a residual frequency domain coefficient matrix; Based on the initial quantization parameter included in the candidate encoding method information, quantize the residual frequency domain coefficient matrix to obtain a quantized frequency domain coefficient matrix; Generate coding unit associated data based on the division identifier corresponding to the coding unit, the candidate coding method information and the quantized frequency domain coefficient matrix; Perform entropy coding on the data associated with the coding unit to obtain a coded data stream corresponding to the coding unit.

5. A video encoding device, comprising: An acquisition unit, configured to acquire a video image sequence to be encoded; The execution unit is configured to execute the following video image coding information generation step for each video image to be encoded in the video image sequence to be encoded: Performing block division processing on the video image to be encoded to obtain a coding unit sequence, wherein each coding unit corresponds to an image block in the video image to be encoded; For each coding unit in the coding unit sequence, the following steps are performed: Based on a preset candidate encoding method information set, determine initial encoding method information and initial reconstructed image corresponding to the encoding unit, wherein each candidate encoding method information includes an initial quantization parameter, the initial encoding method information is information of a candidate encoding method with the smallest rate-distortion optimization result, and the initial reconstructed image is an image reconstructed after encoding and decoding the video image according to the initial encoding method information; Performing visual perception quality detection on the initial reconstructed image to obtain an image quality score; Based on a preset quantization parameter adjustment step size and the image quality score, the initial encoding method information is updated to obtain target encoding method information corresponding to the encoding unit; Based on the target encoding method information, performing predictive encoding processing on the encoding unit to obtain an image block encoding result; Performing entropy coding processing on the obtained encoding results of each image block to obtain video image encoding information corresponding to the video image to be encoded; A generating unit, configured to generate video encoding information based on the obtained encoding information of each video image; The updating process of the initial encoding method information based on the preset quantization parameter adjustment step size and the image quality score to obtain the target encoding method information corresponding to the encoding unit includes: Based on the coding unit region type identifier corresponding to the coding unit and the image quality score, determining a reference quality score corresponding to the coding unit, wherein the coding unit region type identifier indicates whether the coding unit corresponds to the image background region, and the step of determining the reference quality score corresponding to the coding unit includes: In response to determining that the coding unit region type identifier corresponding to the coding unit is an unclassified identifier, determining the image quality score as a reference quality score corresponding to the coding unit; In response to determining that the coding unit region type identifier corresponding to the coding unit is a background region identifier, determining a difference between the image quality score and a preset adjustment score as a reference quality score corresponding to the coding unit; In response to determining that the coding unit region type identifier corresponding to the coding unit is a foreground region identifier, determining the image quality score as a reference quality score corresponding to the coding unit; Determine the sum of the initial quantization parameter included in the initial encoding method information and the preset quantization parameter adjustment step as the optimized quantization parameter; The initial coding method information is used as the optimized coding method information, and the optimized coding method information is added to a preset optimized coding method information sequence, and the added optimized coding method information sequence is obtained as an updated optimized coding method information sequence, wherein the optimized coding method information sequence is a sequence whose length changes with the addition of the optimized coding method information; Based on the optimized quantization parameters, the following target encoding method information generation steps are performed: Based on the optimized quantization parameter, the initial quantization parameter included in the initial encoding method information is replaced to obtain the optimized encoding method information after the replacement process as the optimized encoding method information after the replacement process; Based on the optimized encoding method information after the replacement process, the encoding unit is encoded to obtain an optimized encoded data stream; Decoding and reconstructing the optimized coded data stream to obtain an optimized decoded image; Performing visual perception quality detection on the optimized decoded image to obtain an optimized image quality score; In response to determining that the optimized image quality score is less than the reference quality score, the optimized encoding method information that meets the preset position condition is selected from the updated optimized encoding method information sequence as the target encoding method information.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images