Prediction method and device of coding mode, electronic equipment and storage medium

Abstract

The application provides a prediction method and device of an encoding mode, electronic equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring information of at least two frames of to-be-processed images, the two frames of to-be-processed images being consecutive front and rear images; inputting the information of the at least two frames of to-be-processed images into an encoding mode prediction network for prediction to determine a target encoding mode; wherein the encoding mode prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target encoding mode is used for encoding and / or decoding the to-be-processed images. The time complexity of the traditional algorithm for circularly traversing to find an optimal encoding mode can be reduced, the processing time of the to-be-processed images is reduced, and, through reasonable design of a training label and a network structure, a similar or higher compression ratio can be obtained, the image quality is guaranteed, and the encoding efficiency of the video image is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a prediction method and device of an encoding mode, an electronic device and a storage medium. BACKGROUND

[0002] At present, in the process of video encoding, a prediction encoding technology is usually used to eliminate the correlation between pixels, for example, the difference between a reference pixel and a current pixel is encoded to achieve the purpose of video compression.

[0003] However, when a traditional video encoding mode is selected, all prediction modes or part of the related prediction modes need to be traversed to select the optimal prediction mode as the final processing mode, so that the prediction process is complicated, the calculation complexity is greatly increased, and the processing time of the video file is prolonged. SUMMARY

[0004] The present application provides a prediction method and device of an encoding mode, an electronic device and a storage medium.

[0005] The present application provides a prediction method of an encoding mode, which comprises the following steps: obtaining information of at least two frames of to-be-processed images, the two frames of to-be-processed images being consecutive front and rear images; inputting the information of at least the two frames of to-be-processed images into an encoding mode prediction network for prediction to determine a target encoding mode; wherein the encoding mode prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target encoding mode is used for encoding and / or decoding the to-be-processed images.

[0006] The present application provides a prediction device of an encoding mode, which comprises: an obtaining module configured to obtain information of at least two frames of to-be-processed images, the two frames of to-be-processed images being consecutive front and rear images; and a prediction module configured to input the information of at least the two frames of to-be-processed images into an encoding mode prediction network for prediction to determine a target encoding mode; wherein the encoding mode prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target encoding mode is used for encoding and / or decoding the to-be-processed images.

[0007] The present application provides an electronic device, which comprises one or more processors, and a memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the prediction methods of an encoding mode in the present application.

[0008] The embodiment of the present application provides a readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the prediction method of any one of the encoding modes in the embodiment of the present application.

[0009] According to the prediction method, the device, the electronic device and the storage medium of the encoding mode provided in the embodiment of the present application, the information of the to-be-processed image can be determined by acquiring the information of at least two to-be-processed images, and subsequent processing of the continuous at least two to-be-processed images is facilitated; the information of the at least two to-be-processed images is input into the encoding mode prediction network for prediction to determine the target encoding mode, and the encoding mode prediction network is a network obtained by training a convolutional neural network based on a multi-size pixel block, so that the time complexity of the traditional algorithm for circularly traversing to find the optimal prediction encoding mode is reduced, thereby reducing the processing time length of the to-be-processed image when the to-be-processed image is encoded and / or decoded by using the target encoding mode, and a similar or higher compression ratio can be obtained through reasonable design of the training label and the network structure, so that the image quality is guaranteed, and the encoding efficiency of the video image is improved.

[0010] More details about the above embodiments and other aspects of the present application and the implementation thereof are provided in the description of drawings, the specific embodiments and the claims. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 An encoding flowchart based on a video compression protocol provided by the embodiment of the present application is shown.

[0012] Figure 2 A flowchart of the prediction method of the encoding mode provided by the embodiment of the present application is shown.

[0013] Figure 3 A network structure diagram of Res-CNN provided by the embodiment of the present application is shown.

[0014] Figure 4 A flowchart of the method for training Res-CNN by using sample images provided by the embodiment of the present application is shown.

[0015] Figure 5 A flowchart of the prediction method of the encoding mode based on Res-CNN provided by the embodiment of the present application is shown.

[0016] Figure 6 A flowchart of the prediction method of the encoding mode provided by the embodiment of the present application is shown.

[0017] Figure 7 A block diagram of the prediction device of the encoding mode provided by the embodiment of the present application is shown.

[0018] Figure 8 A structural diagram showing an exemplary hardware architecture of a computing device capable of implementing the prediction method and apparatus of the coding mode according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] For the purposes of the present application, the technical solutions and advantages will be more clearly apparent in the detailed description of the embodiments of the present application below with reference to the accompanying drawings. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other as long as they do not conflict.

[0020] According to the position of the reference pixel, video coding prediction mainly includes an intra prediction mode and an inter prediction mode. The intra prediction mode is a mode of predicting an uncoded pixel using a coded pixel in the current frame by using the correlation of the video space. Through the intra prediction mode, the prediction residual is quantized, and the spatial redundancy of the video can be effectively removed, and the clarity of the video image is improved.

[0021] When intra prediction is performed using different video coding protocols, different prediction modes are used. For example, the prediction modes supported by the H265 protocol or the High Efficiency Video Coding (HEVC) protocol include a plannar mode, a Dual Channel (DC) mode, and 33 angle modes. The prediction modes supported by the H266 protocol or the Versatile Video Coding (VVC) protocol include a plannar mode, a DC mode, and 65 angle modes.

[0022] Figure 1 A coding flowchart based on a video compression protocol is shown. The video compression protocol can include any one or several of the H265 protocol, the HEVC protocol, the H266 protocol, and the VVC protocol.

[0023] As shown in Figure 1 The coding method based on the video compression protocol includes but is not limited to the following steps.

[0024] Step S101: Determine whether the current coding unit needs to be divided into prediction units.

[0025] The code unit (CU) is a part of a coding tree unit (CTU), and the prediction unit (PU) defines all prediction modes of the code unit and all prediction-related information is defined in the PU. For example, the PU can include any one or more of the following information: an intra-prediction direction, an inter-prediction partitioning mode, a motion vector predictor, and an inter-prediction reference picture index number.

[0026] If it is determined that the PU partitioning is needed, step S103 is performed; otherwise, step S102 is performed.

[0027] In step S102, four sub-code units (SubCUs) are processed in a loop.

[0028] In step S103, the PU partitioning mode is determined.

[0029] The PU partitioning mode can include multiple modes, such as PU partition_1, PU partition_2, …, and PU partition_m, where m represents the number of PU partitioning modes, and m is an integer greater than or equal to 1.

[0030] It should be noted that in the process of determining the PU partitioning mode, each PU partitioning mode needs to be looped. After selecting the PU partitioning mode, step S104 needs to be performed.

[0031] In step S104, the prediction mode is determined.

[0032] The prediction mode can include multiple modes, such as mode_1, mode_2, …, and mode_k, where k represents the number of prediction modes, and k is an integer greater than or equal to 1.

[0033] It should be noted that in the process of determining the prediction mode, each prediction mode needs to be looped, so as to finally select the optimal prediction mode as the target prediction coding mode.

[0034] In step S105, the target prediction coding mode is obtained.

[0035] Through the processing of steps S103 to S104, all prediction modes need to be traversed to select the optimal prediction coding mode as the target prediction coding mode, which makes the prediction process cumbersome and greatly increases the computational complexity and prolongs the processing time of the video file.

[0036] The application provides a prediction method, device, electronic equipment and storage medium of an encoding mode. The method is used for optimizing the selection process of the prediction mode in step S104, reducing the time complexity of the traditional algorithm in the process of cyclically searching for the optimal prediction encoding mode, and reducing the processing time of the to-be-processed image.

[0037] Figure 2 A flowchart of the prediction method of the encoding mode provided by an embodiment of the application is shown. The prediction method of the encoding mode can be applied to a prediction device of the encoding mode. As shown in the figure, Figure 2 The prediction method of the encoding mode in the embodiment of the application includes but is not limited to the following steps.

[0038] In step S201, information of at least two frames of to-be-processed images is obtained.

[0039] The two frames of to-be-processed images are consecutive images.

[0040] In step S202, the information of the at least two frames of to-be-processed images is input into an encoding mode prediction network for prediction to determine a target encoding mode.

[0041] The encoding mode prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target encoding mode is used for encoding and / or decoding the to-be-processed image.

[0042] In the embodiment, by obtaining the information of the at least two frames of to-be-processed images, the information of the to-be-processed images can be determined, which facilitates the subsequent processing of the at least two consecutive frames of to-be-processed images. By inputting the information of the at least two frames of to-be-processed images into the encoding mode prediction network for prediction to determine the target encoding mode, the time complexity of the traditional algorithm in the process of cyclically searching for the optimal encoding mode can be reduced, so that when the target encoding mode is used to encode and / or decode the to-be-processed image, the processing time of the to-be-processed image can be reduced, so that the to-be-processed image can obtain a higher compression ratio, and the encoding efficiency of the video image can be improved while ensuring the image quality.

[0043] In some specific implementations, the information of the to-be-processed image includes at least one of pixel block information of the to-be-processed image, a prediction mode corresponding to the pixel block information, a number of the prediction modes, and division information of a coding unit (CU).

[0044] For example, the pixel block information of the to-be-processed image can include the size of the pixel block, whether the pixel block has been encoded, and the like. For example, different identifiers can be used to represent the encoded pixel block and the to-be-encoded pixel block, so as to distinguish different pixel blocks and speed up the processing of the image.

[0045] The number of prediction modes is determined based on the number of prediction modes that can be supported by different video coding protocols, such as H265 protocol, HEVC protocol, H266 protocol, VVC protocol, and the like.

[0046] It should be noted that different pixel block information corresponds to different prediction modes. For example, the larger the pixel block size, the more network layers of the encoding mode prediction network are required to ensure the accuracy of the prediction mode corresponding to the pixel block information.

[0047] For example, the two frames of to-be-processed images include a first frame of to-be-processed image and a second frame of to-be-processed image. The first frame of to-be-processed image and the second frame of to-be-processed image are two consecutive images. The larger the pixel block size corresponding to the first frame of to-be-processed image, the more network layers of the target encoding mode prediction network; the smaller the pixel block size corresponding to the second frame of to-be-processed image, the fewer network layers of the target encoding mode prediction network.

[0048] In some specific implementations, the information of the at least two frames of to-be-processed images is input into the encoding mode prediction network in step S202 for prediction to determine the target encoding mode, which can be implemented in the following manner:

[0049] According to the obtained CTU information of the first frame of to-be-processed image, the pixel block size corresponding to the first frame of to-be-processed image is determined; according to the pixel block size corresponding to the first frame of to-be-processed image, the multiple encoding mode prediction networks are screened to obtain the target encoding mode prediction network; the information of the first frame of to-be-processed image and the information of the second frame of to-be-processed image are input into the target encoding mode prediction network for prediction to determine the target encoding mode.

[0050] The CTU information is used to represent the encoding complexity of the first frame of to-be-processed image, and the target encoding mode prediction network matches the pixel block size corresponding to the first frame of to-be-processed image.

[0051] It should be noted that the more detailed the division of the first frame of to-be-processed image in the CTU information, the higher the encoding complexity of the first frame of to-be-processed image. The pixel block size corresponding to the first frame of to-be-processed image can determine which encoding mode prediction network is selected for prediction of the first frame of to-be-processed image, so that the obtained target encoding mode prediction network can meet the processing requirements of the first frame of to-be-processed image, speed up the image processing speed, and also make the first frame of to-be-processed image obtain more accurate prediction. In addition, the encoding mode of the first frame of to-be-processed image is predicted by the target encoding mode prediction network to determine whether the encoded image meets the requirements of the second frame of to-be-processed image, so that the determined target encoding mode is more accurate.

[0052] In some embodiments, the pixel block size corresponding to the first frame of to-be-processed image is determined according to the CTU information of the first frame of to-be-processed image, including: determining the pixel block size corresponding to the first frame of to-be-processed image according to at least one of the number of CUs, the number of PUs and the number of TUs of the first frame of to-be-processed image.

[0053] The CU is a basic unit for prediction, transformation, quantization and entropy coding, the PU is a basic unit for intra prediction and / or inter prediction, and the TU is a basic unit for transformation and quantization. The separation of the three units not only makes each processing link of the transformation prediction and coding corresponding to the to-be-processed image more flexible, but also makes the division of each link more in line with the texture characteristics of the video image, thereby ensuring the optimization of the coding performance.

[0054] At least one of the number of CUs, the number of PUs and the number of TUs of the first frame of to-be-processed image can reflect the texture complexity of the first frame of to-be-processed image, so that the pixel block size corresponding to the first frame of to-be-processed image is determined according to different texture complexity.

[0055] In some embodiments, the CTU information of the first frame of to-be-processed image includes: the number of CUs and the CUs.

[0056] The target coding mode prediction network is obtained by screening a plurality of coding mode prediction networks according to the pixel block size corresponding to the first frame of to-be-processed image, including: performing clustering analysis on the prediction coding mode corresponding to the to-be-coded pixel block in the CU according to the number of CUs, the division mode of each CU and the information of the coded pixel block in the CU, to obtain an analysis result; and determining the target coding mode prediction network according to the analysis result.

[0057] The prediction coding mode is a pixel-based prediction coding mode, and the analysis result includes: the prediction coding mode based on the current prediction pixel block. The clustering analysis can be statistical clustering of the pixel-based prediction coding mode output by the coding mode prediction network, to obtain the prediction coding mode of the final to-be-predicted pixel block.

[0058] In specific implementation, each CU can be processed in a loop based on the number of CUs, so that the prediction coding mode corresponding to the to-be-coded pixel block in the CU is classified according to the division mode of each CU and the information of the coded pixel block in the CU, so that the analysis result obtained can reflect the category of the prediction coding mode, and the target coding mode prediction network is determined based on the analysis result.

[0059] In some embodiments, the analysis result comprises the number of occurrences of the prediction encoding mode corresponding to the pixel block to be encoded in the CU. Determining the target encoding mode prediction network according to the analysis result comprises: sorting the number of occurrences of the prediction encoding mode corresponding to the pixel block to be encoded in the CU to obtain a sorting result; and determining the target encoding mode prediction network according to the sorting result.

[0060] For example, the prediction encoding mode corresponding to the pixel block to be encoded in the CU comprises a first prediction encoding mode, a second prediction encoding mode, and a third prediction encoding mode. Correspondingly, the first prediction encoding mode occurs 3 times, the second prediction encoding mode occurs 2 times, and the third prediction encoding mode occurs 5 times. By sorting the number of occurrences of the above prediction encoding modes, it can be known that the third prediction encoding mode occurs the most (i.e., 5 times). Therefore, the encoding mode prediction network corresponding to the third prediction encoding mode can be used as the target encoding mode prediction network, so that the target encoding mode prediction network can be quickly and accurately determined, and the acquisition speed of the prediction network is improved.

[0061] In some embodiments, before the information of the at least two frames of images to be processed is obtained in step S201, the method further comprises: training the convolutional neural network according to the plurality of sample images and a plurality of preset pixel block sizes to obtain a plurality of encoding mode prediction networks corresponding to the preset pixel block sizes.

[0062] The preset pixel block size can include any one or more of a size corresponding to an image resolution of 8x8, a size corresponding to an image resolution of 16x16, a size corresponding to an image resolution of 32x32, and a size corresponding to an image resolution of 64x64. The above is only an example of the preset pixel block size, which can be specifically set according to actual needs. Other unexplained preset pixel block sizes are also within the protection scope of the present application, and will not be described here.

[0063] Based on a plurality of different preset pixel block sizes, the convolutional neural network is trained using a plurality of sample images, respectively, to obtain a plurality of encoding mode prediction networks corresponding to the preset pixel block sizes. In this way, in the process of predicting the encoding mode of the image, the encoding mode prediction network corresponding to different pixel block sizes can be screened, the prediction speed of different pixel block sizes is improved, and the processing speed of the image is accelerated.

[0064] In some embodiments, the convolutional neural network is trained according to a plurality of sample images and a plurality of preset pixel block sizes to obtain a plurality of coding mode prediction networks corresponding to the preset pixel block sizes, including: screening the plurality of sample images according to the plurality of preset pixel block sizes to obtain a plurality of test sample image sets; and inputting test sample images in the plurality of test sample image sets into the convolutional neural network for training to obtain the plurality of coding mode prediction networks corresponding to the preset pixel block sizes.

[0065] In some embodiments, the convolutional neural network is trained according to a plurality of sample images and a plurality of preset pixel block sizes to obtain a plurality of coding mode prediction networks corresponding to the preset pixel block sizes, including: screening the plurality of sample images according to the plurality of preset pixel block sizes to obtain a plurality of test sample image sets; and inputting test sample images in the plurality of test sample image sets into the convolutional neural network for training to obtain the plurality of coding mode prediction networks corresponding to the preset pixel block sizes.

[0066] It should be noted that one test sample image set corresponds to one coding mode prediction network based on the number of preset pixel block sizes, and a plurality of coding mode prediction networks are determined and obtained, and the number of the coding mode prediction networks is the same as the number of the preset pixel block sizes.

[0067] For example, the number of preset pixel block sizes is 4, and the specific sizes include: 8x8, 16x16, 32x32 and 64x64. Correspondingly, according to the above four sizes, the plurality of sample images are screened to obtain four test sample image sets, and each test sample image set includes a plurality of test sample images.

[0068] For example, the first test sample image set corresponds to the preset pixel block size of 8x8; the second test sample image set corresponds to the preset pixel block size of 16x16; the third test sample image set corresponds to the preset pixel block size of 32x32; and the fourth test sample image set corresponds to the preset pixel block size of 64x64. The above four test sample image sets are input into the convolutional neural network (for example, any one of Res-CNN network, Transformer network and GAN network) for training to obtain four coding mode prediction networks, for example, a coding mode prediction network corresponding to the preset pixel block size of 8x8; a coding mode prediction network corresponding to the preset pixel block size of 16x16; a coding mode prediction network corresponding to the preset pixel block size of 32x32; and a coding mode prediction network corresponding to the preset pixel block size of 64x64. Thus, different coding mode prediction networks can be suitable for different pixel block sizes, and the adaptability to images of different pixel block sizes is improved.

[0069] In some embodiments, the convolutional neural network is trained according to a plurality of sample images and a plurality of preset pixel block sizes to obtain a plurality of coding mode prediction networks corresponding to the preset pixel block sizes, including:

[0070] The following processing is performed on each of the sets of test sample images: inputting the test sample images in the set of test sample images into the convolutional neural network for training to obtain a to-be-verified encoding mode prediction network; and in a case where it is determined that an output result of the to-be-verified encoding mode prediction network satisfies a preset condition, obtaining an encoding mode prediction network corresponding to the preset pixel block size.

[0071] It should be noted that the process of inputting the test sample images in the set of test sample images into the convolutional neural network for training is performed in a loop. If it is determined that the output result of the to-be-verified encoding mode prediction network does not satisfy the preset condition, the convolutional neural network needs to be continuously trained until the output result of the to-be-verified encoding mode prediction network satisfies the preset condition, so as to obtain the encoding mode prediction network corresponding to the preset pixel block size.

[0072] The preset condition is a preset test condition. For example, a verification image can be encoded based on the to-be-verified predicted encoding mode output by the to-be-verified encoding mode prediction network, so as to obtain a to-be-verified encoded image. If the image quality of the to-be-verified encoded image satisfies the use demand of a user, or the peak signal-to-noise ratio of the to-be-verified encoded image satisfies a preset signal-to-noise ratio threshold, it is determined that the output result of the to-be-verified encoding mode prediction network satisfies the preset condition.

[0073] In some specific implementations, the output result of the to-be-verified encoding mode prediction network includes a predicted mode probability value of a pixel point corresponding to an output image and a preset number of preset encoding modes supported by a preset encoding protocol.

[0074] In a case where it is determined that the output result of the to-be-verified encoding mode prediction network satisfies the preset condition, obtaining the encoding mode prediction network corresponding to the preset pixel block size includes: calculating a loss value according to the predicted mode probability value of the pixel point corresponding to the output image and the preset number of preset encoding modes supported by the preset encoding protocol; and in a case where it is determined that the loss value is within a preset loss threshold range, obtaining the encoding mode prediction network corresponding to the preset pixel block size.

[0075] The loss value is used to represent the loss of multi-classification cross entropy in the convolutional neural network. The loss value can also be used to measure the similarity between the distribution of real labels and the predicted label distribution of the trained encoding mode prediction network.

[0076] The predicted mode probability value (for example, the probability value of the predicted mode of each pixel point of a pixel block X in an output image) and loss, where loss represents a loss value used to represent the loss of multi-classification cross entropy in the Res-CNN network.

[0077] In a case where it is determined that the loss value is within the range of the preset loss threshold, the training of the convolutional neural network is stopped, and an encoding mode prediction network corresponding to the preset pixel block size is obtained to adapt to the prediction requirement of pixel blocks of different sizes.

[0078] For example, Figure 3 A network structure schematic diagram of the Res-CNN provided by the embodiment of the present application is shown. As shown in Figure 3 The structure of the Res-CNN network includes but is not limited to the following modules.

[0079] The first convolutional module 311, the second convolutional module 312, …, the nth convolutional module 31n; the pooling layer 320; the global pooling processing module 330, the full connection layer 340, and the activation function 350.

[0080] Wherein, n represents the number of convolutional modules, and n is an integer greater than or equal to 1. A / B / C respectively represent encoded pixel blocks; X represents a pixel block to be predicted.

[0081] The input parameters of the first convolutional module 311 include but are not limited to the following information: image information of the encoded pixel block A (and / or, the encoded pixel block B, and / or, the encoded pixel block C, etc.), CU division information, and prediction mode information corresponding to the encoded pixel block A (and / or, the encoded pixel block B, and / or, the encoded pixel block C, etc.).

[0082] Figure 4 A flowchart of a method for training the Res-CNN using sample images provided by the embodiment of the present application is shown. As shown in Figure 4 The method for training the Res-CNN using sample images includes but is not limited to the following steps.

[0083] Step S401, information of a plurality of sample images is obtained.

[0084] Wherein, the information of the sample images can include pixel block information (for example, the encoded pixel block A / the encoded pixel block B / the encoded pixel block C, and the pixel block X to be predicted, etc.) of the plurality of sample images. Through the plurality of pixel block information of the plurality of sample images, the prediction accuracy of the prediction encoding mode of the pixel block X to be predicted can be improved.

[0085] Step S402, the number of network layers of the adaptive Res-CNN network is determined according to the size of the pixel block in the sample image.

[0086] The number of layers in the adaptive Res-CNN network can be adaptively adjusted based on the size of at least one of the encoded pixel blocks A, B, and C in the input. For example, the size of the pixel block can be 8×8, 16×16, 32×32, or 64×64, etc. The above examples of pixel block sizes are merely illustrative and can be set according to actual needs. Other unspecified pixel block sizes are within the scope of protection of this application and will not be elaborated further here.

[0087] It should be noted that the larger the pixel size in the sample image, the more layers the adaptive Res-CNN network should have, which can capture the feature information of large pixel blocks; the smaller the pixel size in the sample image, the fewer layers the adaptive Res-CNN network should have, which can offset the computational cost caused by the increase in the number of pixel blocks while ensuring the training effect.

[0088] Step S403: Input the information of multiple sample images into the convolutional module of the adaptive Res-CNN network for training to obtain the training results.

[0089] like Figure 3 As shown, the information of encoded pixel block A, encoded pixel block B, encoded pixel block C, and the pixel block to be predicted X are sequentially input into the first convolution module 311, the second convolution module 312, ..., the nth convolution module 31n for convolution operations to obtain the convolution processing result. This convolution result is then input into the pooling layer 320 for pooling processing to obtain the pooling result and residual value. A judgment is made based on the residual value to obtain a judgment result. Based on this judgment result, it is determined whether to feed the pooling result back to the first convolution module 311 for iterative processing. When it is determined that it is not necessary to perform iterative processing, the pooling result output from the pooling layer 320 is input into the global pooling processing module 330, the fully connected layer 340, and the activation function 350 for sequential processing to obtain the training result.

[0090] Step S404: Perform a performance test on the training results, and determine whether to end the training of the adaptive Res-CNN network based on the performance test results.

[0091] If the performance test results meet the preset test indicators, the training of the adaptive Res-CNN network can be terminated; otherwise, the training of the adaptive Res-CNN network needs to continue.

[0092] For example, the training results include: the prediction encoding pattern of the pixel block X to be predicted, encoding the pixel block X to be predicted using the prediction encoding pattern to obtain the encoding result, and determining to end the training of the adaptive Res-CNN network when the encoding result meets the preset encoding requirements.

[0093] In a case where it is determined that the training of the adaptive Res-CNN network is ended, step S405 is performed; in a case where it is determined that the training of the adaptive Res-CNN network needs to be continued, step S403 is performed.

[0094] Step S405: obtaining the trained adaptive Res-CNN network for subsequent processing of different to-be-processed images.

[0095] In this embodiment, the Res-CNN network is trained by using multiple sample images respectively in multiple different preset pixel block sizes, and multiple adaptive Res-CNN networks corresponding to the preset pixel block sizes are obtained, so that in the process of predicting the coding mode of an image, the trained adaptive Res-CNN network can be used to process different to-be-processed images, the prediction speed of different to-be-processed images is improved, and the processing speed of the image is accelerated.

[0096] Figure 5 A flowchart of a Res-CNN-based coding mode prediction method provided by an embodiment of the present application is shown. As shown in Figure 5 The input frame block sequence is input into the adaptive Res-CNN coding mode prediction network 501 for prediction, and the prediction mode information of the X pixel block of the i-th frame block can be obtained. Further, the X pixel block of each frame block is processed by clustering, and the rate-distortion cost is calculated, so that the prediction mode corresponding to the minimum rate-distortion cost in the prediction mode set is selected as the target coding mode.

[0097] The input frame block sequence includes an i-2-th frame block, an i-1-th frame block, and an i-th frame block, where i is an integer greater than or equal to 2. Each frame block includes a coded frame block A, a coded frame block B, a coded frame block C, and a to-be-coded frame block X.

[0098] It should be noted that the adaptive Res-CNN coding mode prediction network 501 can automatically select the network layer number corresponding to the residual convolutional neural network according to the size of the input frame block. The larger the size of the frame block, the more network layers corresponding to the residual convolutional neural network.

[0099] By traversing all CU division manners corresponding to the to-be-encoded frame block X in a loop, for each CU division manner, a rate-distortion cost is calculated, thereby obtaining a prediction mode set including multiple prediction modes; further, based on the rate-distortion cost, the multiple prediction modes in the prediction mode set are sorted, a prediction mode corresponding to a minimum rate-distortion cost is obtained, and the prediction mode corresponding to the minimum rate-distortion cost is taken as a target coding mode, while a CU division manner corresponding to the target coding mode is obtained, so as to realize fast prediction of the prediction coding mode, reduce the processing time length of the image, and improve the coding efficiency.

[0100] Figure 6 A flowchart of a prediction method of a coding mode provided by another embodiment of the present application is shown. As shown in the flowchart, the prediction method of the coding mode includes but is not limited to the following steps. Figure 6

[0101] In step S601, training set data and verification set data are obtained.

[0102] The training set data includes multiple sample images, each sample image includes multiple pixel blocks, and the size of each pixel block is different (for example, the size of a pixel block is 64x64, 32x32, etc.). The verification set data includes to-be-verified images, and each to-be-verified image includes multiple to-be-verified pixel blocks.

[0103] For example, the multiple sample images are filtered according to multiple preset pixel block sizes, and multiple to-be-tested sample image sets are obtained, wherein the multiple to-be-tested sample images in one to-be-tested sample image set correspond to the same pixel block size, and the multiple to-be-tested sample images in two different to-be-tested sample image sets correspond to different pixel block sizes.

[0104] It should be noted that the size of a pixel block is determined by the texture complexity in the pixel block. For example, if the texture complexity of a pixel block is high, the size of the pixel block is large; conversely, if the texture complexity of a pixel block is low, the size of the pixel block is small. The texture complexity of a pixel block is determined by at least one of the number of CUs (coding units), the number of PUs (prediction units), and the number of TUs (transform units) corresponding to the pixel block. The larger the size of a pixel block is, the more layers of a neural network that need to be trained.

[0105] In step S602, the training set data is input into a Res-CNN network for training, and a training result is obtained.

[0106] ​The training set data can further include at least one of pixel block information (for example, pixel values corresponding to the encoded frame block A, pixel values corresponding to the encoded frame block B, and pixel values corresponding to the encoded frame block C, etc.) of the to-be-processed image, a prediction mode corresponding to the pixel block information, a number of prediction modes, and division information of a coding unit (CU).

[0107] The Res-CNN network can include a plurality of different convolution modules (for example, Figure 3 As shown, the first convolution module 311, the second convolution module 312, …, the nth convolution module 31n, etc.), each convolution module corresponding to different convolution kernels.

[0108] The network structure as shown in Figure 3 The network structure as shown in

[0109] Wherein, W represents the length of the output pixel block X, H represents the width of the output pixel block X, and C represents the number of prediction modes supported by the current protocol.

[0110] Further, it is also necessary to calculate the probability value p(x i ) of each pixel point in the output pixel block X, and select the prediction coding mode corresponding to the maximum probability based on each pixel point. For example, a multi-class cross-entropy can be used as the loss function of the Res-CNN network, which can be represented by the following formula:

[0111]

[0112] Wherein, C represents the number of prediction modes supported by the current protocol; p(x i ) represents the probability value of the prediction mode of each pixel point in the output pixel block X; and loss represents the loss value, which is used to represent the loss of the multi-class cross-entropy in the Res-CNN network.

[0113] In the case where the loss value is within the range of the preset loss threshold, the training of the Res-CNN network is stopped, and a training result is obtained. The training result can include a plurality of coding mode prediction networks, each coding mode prediction network corresponding to a pixel block of different size to adapt to the prediction requirements of pixel blocks of different sizes.

[0114] It should be noted that the pixel block size corresponding to different training set data is different, but the pixel block size inside each training set data is the same. When training the Res-CNN network, the Res-CNN network needs to be trained respectively according to the different pixel block sizes, so as to obtain multiple encoding mode prediction networks.

[0115] Step S603, using the validation set data to test the performance of the multiple encoding mode prediction networks obtained by training, to determine whether the encoding mode prediction network needs to be retrained.

[0116] The validation set data includes multiple images to be verified.

[0117] For example, the validation set data is divided using an artificial intelligence (AI) divider to obtain multiple images to be verified of different pixel block sizes, and the images to be verified of different pixel block sizes are input into the encoding mode prediction network corresponding to the pixel block size for verification. In a case where it is determined that the encoded image obtained by encoding the image using the output target encoding mode can meet the preset encoding quality requirement, it is determined that the encoding mode prediction network does not need to be trained, thereby obtaining a trained encoding mode prediction network; otherwise, return to step S602 to continue training the encoding mode prediction network.

[0118] The preset encoding quality requirement can include at least one of a peak signal-to-noise ratio, an image similarity, and an encoding speed.

[0119] Step S603, obtaining a first frame of image to be processed and a second frame of image to be processed, and determining the texture complexity of the first frame of image to be processed.

[0120] For example, the texture complexity of the first frame of image to be processed can be determined by the number of CUs, the number of PUs, and the number of TUs corresponding to the first frame of image to be processed.

[0121] Step S604, determining the pixel block size corresponding to the first frame of image to be processed according to the obtained CU information of the first frame of image to be processed, and selecting a target encoding mode prediction network matching the pixel block size corresponding to the first frame of image to be processed from the multiple encoding mode prediction networks based on the pixel block size corresponding to the first frame of image to be processed.

[0122] The pixel block size corresponding to the target encoding mode prediction network is the same as the pixel block size of the first frame of image to be processed.

[0123] In step S605, the information of the first frame to-be-processed image and the information of the second frame to-be-processed image are input into the target coding mode prediction network for prediction to determine the target coding mode.

[0124] In the step S601, the information of the first frame to-be-processed image and the information of the second frame to-be-processed image are input into the target coding mode prediction network for prediction to determine the target coding mode.

[0125] Further, the rate-distortion cost can be calculated, and finally the target coding mode predicted by the target coding mode prediction network corresponding to the minimum rate-distortion cost is selected.

[0126] The rate-distortion cost represents the mutual relationship between the image distortion degree and the coding code rate. The image distortion degree can be measured by the peak signal to noise ratio (PSNR) between the original image and the coding reconstructed image. The PSNR can be the luminance PSNR or the linear combination of the luminance and chrominance PSNRs.

[0127] The peak signal to noise ratio represents the ratio of the power between the peak signal and the noise signal. The peak signal is used to represent the maximum value of the pixels in the image (for example, the maximum value of the pixel luminance). The noise signal is used to represent the mean square error of the pixel values between the original image and the reconstructed image (for example, the square of the difference value).

[0128] In some specific implementations, the target coding mode can also be identified by using a label. For example, the prediction mode label represents the target coding mode, i.e., the target coding mode predicted by the target coding mode prediction network corresponding to the minimum rate-distortion cost.

[0129] In the embodiment, by training the Res-CNN network based on deep learning, a plurality of coding mode prediction networks of different sizes are obtained. When the coding mode prediction is performed for different pixel block sizes, the target coding mode prediction network corresponding to the pixel block size can be selected based on the pixel block size for prediction, so that the target coding mode obtained can correspond to a smaller rate-distortion cost, i.e., the compression ratio of the transmitted video is higher. Compared with the traditional cyclic iteration algorithm, the time complexity of finding the optimal prediction coding mode can be reduced, the image quality is guaranteed, and the coding efficiency of the video image is improved.

[0130] Figure 7 A constituent block diagram of the prediction device of the encoding mode provided by the embodiments of the present application is shown. As shown, the prediction device of the encoding mode 700 includes but is not limited to the following modules. Figure 7

[0131] The acquisition module 701 is configured to acquire information of at least two frames of to-be-processed images, the two frames of to-be-processed images being consecutive front and rear frames of images;

[0132] The prediction module 702 is configured to input the information of the at least two frames of to-be-processed images into an encoding mode prediction network for prediction to determine a target encoding mode; wherein the encoding mode prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target encoding mode is used for encoding and / or decoding the to-be-processed images.

[0133] In some specific implementations, the two frames of to-be-processed images include a first frame of to-be-processed image and a second frame of to-be-processed image; and the prediction module 702 is specifically configured to: determine a pixel block size corresponding to the first frame of to-be-processed image according to acquired tree coding unit (CTU) information of the first frame of to-be-processed image, wherein the CTU information is used to represent the encoding complexity corresponding to the first frame of to-be-processed image; filter a plurality of encoding mode prediction networks according to the pixel block size corresponding to the first frame of to-be-processed image to obtain a target encoding mode prediction network, the target encoding mode prediction network being matched with the pixel block size corresponding to the first frame of to-be-processed image; and input the information of the first frame of to-be-processed image and the information of the second frame of to-be-processed image into the target encoding mode prediction network for prediction to determine the target encoding mode.

[0134] In some specific implementations, the larger the pixel block size corresponding to the first frame of to-be-processed image is, the more the number of network layers corresponding to the target encoding mode prediction network is.

[0135] In some specific implementations, determining the pixel block size corresponding to the first frame of to-be-processed image according to the acquired tree coding unit (CTU) information of the first frame of to-be-processed image includes: determining the pixel block size corresponding to the first frame of to-be-processed image according to at least one of the number of coding units (CUs), the number of prediction units (PUs) and the number of transform units (TUs) corresponding to the first frame of to-be-processed image.

[0136] ​In some embodiments, the tree coding unit CTU information of the first frame to-be-processed image comprises: a coding unit CU and a number of CUs; and a target coding mode prediction network is obtained by screening a plurality of coding mode prediction networks according to a pixel block size corresponding to the first frame to-be-processed image, comprising: clustering analysis of a prediction coding mode corresponding to a to-be-coded pixel block in the CU is performed according to the number of CUs, a division mode of each CU and information of a coded pixel block in the CU, to obtain an analysis result; and the target coding mode prediction network is determined according to the analysis result.

[0137] In some embodiments, the analysis result comprises: a number of occurrences of the prediction coding mode corresponding to the to-be-coded pixel block in the CU; and the target coding mode prediction network is determined according to the analysis result, comprising: the number of occurrences of the prediction coding mode corresponding to the to-be-coded pixel block in the CU is sorted to obtain a sorting result; and the target coding mode prediction network is determined according to the sorting result.

[0138] In some embodiments, the prediction device 700 of the coding mode further comprises a training module (not shown in the figure) configured to train the convolutional neural network according to a plurality of sample images and a plurality of preset pixel block sizes to obtain a plurality of coding mode prediction networks corresponding to the preset pixel block sizes.

[0139] In some embodiments, the convolutional neural network is trained according to a plurality of sample images and a plurality of preset pixel block sizes to obtain a plurality of coding mode prediction networks corresponding to the preset pixel block sizes, comprising: a plurality of to-be-tested sample image sets are obtained by screening the plurality of sample images according to the plurality of preset pixel block sizes, wherein a plurality of to-be-tested sample images in a to-be-tested sample image set correspond to the same pixel block size, and two different to-be-tested sample image sets correspond to different pixel block sizes; and the to-be-tested sample images in the plurality of to-be-tested sample image sets are input into the convolutional neural network for training to obtain the plurality of coding mode prediction networks corresponding to the preset pixel block sizes.

[0140] In some embodiments, the to-be-tested sample images in the plurality of to-be-tested sample image sets are input into the convolutional neural network for training to obtain the plurality of coding mode prediction networks corresponding to the preset pixel block sizes, comprising: each to-be-tested sample image set is processed as follows: the to-be-tested sample images in the to-be-tested sample image set are input into the convolutional neural network for training to obtain a to-be-verified coding mode prediction network; and in a case where an output result of the to-be-verified coding mode prediction network satisfies a preset condition, a coding mode prediction network corresponding to the preset pixel block size is obtained.

[0141] In some embodiments, the output result of the to-be-verified coding mode prediction network comprises: a prediction mode probability value of a pixel point corresponding to an output image and a preset number of preset coding modes supported by a preset coding protocol; in a case where the output result of the to-be-verified coding mode prediction network meets a preset condition, obtaining a coding mode prediction network corresponding to a preset pixel block size comprises: calculating a loss value according to the prediction mode probability value of the pixel point corresponding to the output image and the preset number of preset coding modes supported by the preset coding protocol, the loss value being used to represent a loss of multi-classification cross entropy in a convolutional neural network; and in a case where the loss value is within a preset loss threshold, obtaining the coding mode prediction network corresponding to the preset pixel block size.

[0142] In some embodiments, the information of the to-be-processed image comprises at least one of: pixel block information of the to-be-processed image, a prediction mode corresponding to the pixel block information, a number of prediction modes, and division information of a coding unit (CU).

[0143] It should be noted that the prediction of the coding mode in this embodiment can implement any of the prediction methods of the coding mode in the embodiments of the present application.

[0144] According to the device of the embodiments of the present application, the information of the at least two to-be-processed images is obtained by the obtaining module, which can clearly determine the information of the to-be-processed images, facilitating subsequent processing of the at least two consecutive to-be-processed images; the prediction module inputs the information of the at least two to-be-processed images into the coding mode prediction network for prediction to determine the target coding mode, which can reduce the time complexity of the traditional algorithm for cyclically traversing to find the optimal coding mode, so that the processing time of the to-be-processed image can be reduced when the to-be-processed image is encoded and / or decoded using the target coding mode, so that the to-be-processed image can obtain a higher compression ratio, ensuring the image quality while improving the coding efficiency of the video image.

[0145] It should be noted that the present application is not limited to the specific configurations and processes described in the above embodiments and shown in the drawings. For the convenience and brevity of description, detailed descriptions of known methods are omitted, and the specific working processes of the above-described systems, modules and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0146] Figure 8 A structural diagram showing an exemplary hardware architecture of a computing device capable of implementing the prediction method and apparatus of the coding mode according to the embodiments of the present application.

[0147] As Figure 8As shown, the computing device 800 includes an input device 801, an input interface 802, a central processing unit 803, a memory 804, an output interface 805, and an output device 806. Among them, the input interface 802, the central processing unit 803, the memory 804, and the output interface 805 are connected to each other through a bus 807, and the input device 801 and the output device 806 are connected to the bus 807 through the input interface 802 and the output interface 805 respectively, and then connected to other components of the computing device 800.

[0148] Specifically, the input device 801 receives input information from the outside, and transmits the input information to the central processing unit 803 through the input interface 802; the central processing unit 803 processes the input information based on the computer executable instructions stored in the memory 804 to generate output information, temporarily or permanently stores the output information in the memory 804, and then transmits the output information to the output device 806 through the output interface 805; the output device 806 outputs the output information to the outside of the computing device 800 for the user to use.

[0149] In one embodiment, Figure 8 The computing device shown can be implemented as an electronic device, which can include a memory configured to store a program, and a processor configured to run the program stored in the memory to execute the prediction method of the coding mode described in the above embodiments.

[0150] In one embodiment, Figure 8 The computing device shown can be implemented as a prediction system of a coding mode, which can include a memory configured to store a program, and a processor configured to run the program stored in the memory to execute the prediction method of the coding mode described in the above embodiments.

[0151] The above is only an exemplary embodiment of the present application, and is not intended to limit the protection scope of the present application. Generally, various embodiments of the present application can be implemented in hardware or special circuit, software, logic or any combination thereof. For example, some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, microprocessor or other computing device, although the present application is not limited thereto.

[0152] Embodiments of the present application can be implemented by a data processor of a mobile device executing computer program instructions, for example, in a processor entity, or by hardware, or by a combination of software and hardware. The computer program instructions can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages.

[0153] The block diagrams of any logical flow of the present application in the drawings can represent program steps or can represent interconnected logic circuits, modules, and functions, or can represent a combination of program steps and logic circuits, modules, and functions. The computer program can be stored on a memory. The memory can be of any type suitable to the local technical environment and can be implemented using any suitable data storage technology, such as, but not limited to, random access memory (RAM), read only memory (ROM), optical storage devices, and systems such as digital versatile disc (DVD) or CD-ROM, and the like. The computer readable media can include non-transitory storage media. The data processor can be of any type suitable to the local technical environment, and can include one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), programmable logic devices (PLD), and processors based on multi-core processor architectures, as examples.

[0154] The foregoing detailed description of the exemplary embodiments of the present application has been presented for the purposes of illustration and description. Many modifications and variations of the above-described embodiments are possible in light of the above teachings. It is intended that the scope of the application be limited not by this detailed description, but rather by the claims appended hereto.

Claims

1. A method for predicting coding patterns, characterized in that, The method includes: Acquire information from at least two frames of images to be processed, wherein the two frames of images to be processed are consecutive frames; the two frames of images to be processed include: a first frame of images to be processed and a second frame of images to be processed. Based on the obtained tree coding unit (CTU) information of the first frame image to be processed, the pixel block size corresponding to the first frame image to be processed is determined, wherein the CTU information is used to characterize the coding complexity corresponding to the first frame image to be processed. Based on the pixel block size corresponding to the first frame image to be processed, multiple coding mode prediction networks are filtered to obtain a target coding mode prediction network, which is matched with the pixel block size corresponding to the first frame image to be processed. The information of the first frame of the image to be processed and the information of the second frame of the image to be processed are input into the target coding mode prediction network for prediction to determine the target coding mode; The encoding pattern prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target encoding pattern is used for subsequent encoding and / or decoding of the image to be processed.

2. The method according to claim 1, characterized in that, The larger the pixel block size of the first frame image to be processed, the more network layers the target coding mode prediction network has.

3. The method according to claim 1, characterized in that, The step of determining the pixel block size corresponding to the first frame of the image to be processed based on the obtained tree coding unit (CTU) information includes: The pixel block size of the first frame image to be processed is determined based on at least one of the number of coding units (CU), prediction units (PU), and conversion units (TU) corresponding to the first frame image to be processed.

4. The method according to claim 1, characterized in that, The tree coding unit (CTU) information of the first frame image to be processed includes: coding units (CUs) and the number of CUs; The step of filtering multiple coding mode prediction networks based on the pixel block size corresponding to the first frame of the image to be processed to obtain a target coding mode prediction network includes: Based on the number of CUs, the division method of each CU, and the information of the encoded pixel blocks in the CUs, cluster analysis is performed on the predicted coding patterns corresponding to the pixel blocks to be encoded in the CUs to obtain the analysis results; The target coding pattern prediction network is determined based on the analysis results.

5. The method according to claim 4, characterized in that, The analysis results include: the number of occurrences of the predicted coding pattern corresponding to the pixel block to be encoded in the CU; Determining the target coding pattern prediction network based on the analysis results includes: The occurrence counts of the predicted coding patterns corresponding to the pixel blocks to be encoded in the CU are sorted to obtain the sorting results; Based on the sorting results, the target coding pattern prediction network is determined.

6. The method according to any one of claims 1 to 5, characterized in that, Before acquiring information from at least two frames of images to be processed, the process also includes: The convolutional neural network is trained based on multiple sample images and multiple preset pixel block sizes to obtain multiple encoding pattern prediction networks corresponding to the preset pixel block sizes.

7. The method according to claim 6, characterized in that, The step of training the convolutional neural network based on multiple sample images and multiple preset pixel block sizes to obtain multiple encoding pattern prediction networks corresponding to the preset pixel block sizes includes: Multiple sample images are filtered according to multiple preset pixel block sizes to obtain multiple test sample image sets, wherein multiple test sample images in one test sample image set have the same pixel block size, and two different test sample image sets have different pixel block sizes. The test sample images from multiple sets of test sample images are input into the convolutional neural network for training, thereby obtaining multiple encoding pattern prediction networks corresponding to the preset pixel block size.

8. The method according to claim 7, characterized in that, The step of inputting test sample images from multiple test sample image sets into the convolutional neural network for training to obtain multiple encoding pattern prediction networks corresponding to the preset pixel block size includes: The following processing is performed on each of the test sample image sets: The test sample images in the set of test sample images are input into the convolutional neural network for training to obtain the coding pattern prediction network to be verified. If the output of the coding pattern prediction network to be verified meets the preset conditions, a coding pattern prediction network corresponding to the preset pixel block size is obtained.

9. The method according to claim 8, characterized in that, The output of the coding mode prediction network to be verified includes: the prediction mode probability value of the pixel corresponding to the output image and the number of preset coding modes supported by the preset coding protocol. The step of obtaining a coding pattern prediction network corresponding to the preset pixel block size when the output result of the coding pattern prediction network to be verified meets the preset conditions includes: Based on the predicted mode probability value of the pixel corresponding to the output image and the number of preset encoding modes supported by the preset encoding protocol, a loss value is calculated. The loss value is used to characterize the loss of multi-class cross-entropy in the convolutional neural network. If the loss value is determined to be within the range of a preset loss threshold, an encoding pattern prediction network corresponding to the preset pixel block size is obtained.

10. The method according to claim 1, characterized in that, The information of the image to be processed includes at least one of the following: pixel block information of the image to be processed, prediction mode corresponding to the pixel block information, number of prediction modes, and division information of coding unit (CU).

11. A device for predicting a coding pattern, characterized in that, It includes: The acquisition module is configured to acquire information of at least two frames of images to be processed, wherein the two frames of images to be processed are consecutive frames. The two frames of images to be processed include: a first frame of images to be processed and a second frame of images to be processed; The prediction module is configured to: determine the pixel block size corresponding to the first frame of the image to be processed based on the obtained tree coding unit (CTU) information of the first frame of the image to be processed, wherein the CTU information is used to characterize the coding complexity corresponding to the first frame of the image to be processed; filter multiple coding mode prediction networks based on the pixel block size corresponding to the first frame of the image to be processed to obtain a target coding mode prediction network, wherein the target coding mode prediction network matches the pixel block size corresponding to the first frame of the image to be processed; input the information of the first frame of the image to be processed and the information of the second frame of the image to be processed into the target coding mode prediction network for prediction to determine the target coding mode; wherein the coding mode prediction network is a network obtained by training a convolutional neural network based on pixel blocks of multiple sizes, and the target coding mode is used to encode and / or decode the image to be processed.

12. An electronic device, characterized in that, include: One or more processors; A memory having stored one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method for predicting the encoded pattern as described in any one of claims 1 to 10.

13. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements the prediction method for the encoded pattern as described in any one of claims 1 to 10.

Images