Intra-frame prediction method and device, electronic equipment and storage medium
By searching the target prediction area based on pixel difference and texture similarity in the intra prediction method, the problem of inaccurate prediction results in the prior art is solved, and the video compression effect is improved.
Patent Information
- Application Number
- CN202310662604.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-06
- Publication Date
- 2024-12-06
AI Technical Summary
In the prior art, the region matching prediction method is based only on pixel value errors, resulting in inaccurate prediction results and affecting the video compression effect.
By searching for the initial prediction area that meets the preset difference conditions and texture similarity in the frame to be encoded, the target prediction area is determined, and prediction is made based on the pixel value of the target prediction area, thereby improving the accuracy of prediction.
The accuracy of regional matching prediction and video compression effect are improved, and the transmission bandwidth and storage resources of video data are reduced.
Smart Images

Figure CN119094776A_ABST
Abstract
Description
Background Art
[0002] At present, with the rapid development of Internet technology, video image interaction can be applied to a variety of different scenarios, such as video calls, remote conferences, and short videos.
[0003] In order to reduce the network bandwidth and storage space during transmission, video images need to be encoded and decoded. Video encoding mainly includes acquisition, prediction, quantization and entropy coding. Prediction is divided into intra-frame prediction and inter-frame prediction, which are used to remove spatial redundancy and temporal redundancy respectively. Intra-frame prediction has multiple prediction modes, such as region matching mode.
[0004] In the prior art, the area matching mode is to determine a target area with the smallest pixel value error with the coding reference area based on the coding reference area formed by the upper left part of the block to be coded, and then obtain the corresponding coded block as the prediction block of the block to be coded according to the target area.
[0005] See also Figure 1 As shown, the coding reference area refers to the upper left part L0 of the block to be coded CU, L1 is the target area, and the coded block P1 corresponding to L1 is used as the prediction block of the block to be coded.
[0006] However, when performing region matching prediction using the method in the related art, determining the target region of the encoding reference region based solely on pixel value errors will lead to inaccurate prediction results, because two regions with small pixel value errors may not be similar. If the region with the smallest pixel value error is used as the target region, it will affect the prediction results of the block to be encoded, resulting in poor video compression effects.
[0007] Therefore, how to improve the accuracy of regional matching prediction is a technical problem that needs to be solved at present. Summary of the invention
[0008] The embodiments of the present application provide an intra-frame prediction method, device, electronic device and storage medium to improve the accuracy of region matching prediction.
[0009] The specific technical solutions provided by the embodiments of this application are as follows:
[0010] In a first aspect, an embodiment of the present application provides an intra-frame prediction method, the method comprising:
[0011] Obtaining a coding reference area of a block to be coded in a frame to be coded, the coding reference area comprising: coded pixels within a set range around the block to be coded;
[0012] Based on the coding reference area, searching for at least one initial prediction area in the coded pixel area in the frame to be coded, wherein the pixel difference between each initial prediction area and the coding reference area meets the preset difference condition and has the same shape and size as the coding reference area;
[0013] Selecting at least one target prediction region from at least one initial prediction region, wherein the texture similarity between each target prediction region and the coding reference region meets a preset texture similarity condition;
[0014] Based on each pixel value included in the encoded block corresponding to each of the at least one target prediction area, a prediction value of each pixel included in the block to be encoded is determined.
[0015] In a second aspect, an embodiment of the present application further provides an intra-frame prediction device, the device comprising:
[0016] An acquisition module is used to acquire a coding reference area of a block to be coded in a frame to be coded, wherein the coding reference area includes: coded pixels within a set range around the block to be coded;
[0017] A first processing module is used to search and obtain at least one initial prediction area in the encoded pixel area in the frame to be encoded based on the encoding reference area, wherein the pixel difference between each initial prediction area and the encoding reference area meets a preset difference condition and has the same shape and size as the encoding reference area;
[0018] A second processing module is used to select at least one target prediction region from the at least one initial prediction region, wherein the texture similarity between each target prediction region and the coding reference region meets a preset texture similarity condition;
[0019] The generating module is used to determine the predicted values of each pixel contained in the block to be encoded based on the pixel values contained in the encoded blocks corresponding to each of the at least one target prediction area.
[0020] In a possible implementation, based on the coding reference area, when searching for at least one initial prediction area in a coded pixel area in a frame to be coded, the first processing module is used to:
[0021] According to the shape and size of the coding reference area, searching for each candidate area in the coded pixel area in the frame to be coded;
[0022] For each candidate region, the following operations are respectively performed: based on each pixel value included in the candidate region and each pixel value included in the coding reference region, first pixel difference values between two pixel points at corresponding positions are respectively determined, and based on each obtained first pixel difference value, pixel difference between the candidate region and the coding reference region is determined;
[0023] From each candidate region, at least one initial prediction region whose pixel difference meets a preset difference condition is selected.
[0024] In a possible implementation, when at least one target prediction region is selected from at least one initial prediction region, the second processing module is used to:
[0025] Determine the total gradient value of at least one initial prediction region and the coding reference region based on the pixel values respectively included in the at least one initial prediction region and the coding reference region;
[0026] For at least one initial prediction region, the following operations are respectively performed: based on a total gradient value of the initial prediction region and a total gradient value of the coding reference region, a gradient difference between the initial prediction region and the coding reference region is determined;
[0027] At least one target prediction region whose gradient difference meets a preset difference condition is selected from the at least one initial prediction region.
[0028] In a possible implementation, when determining the total gradient value of at least one initial prediction region and the coding reference region based on the pixel values respectively included in the at least one initial prediction region and the coding reference region, the second processing module is further used to:
[0029] At least one initial prediction region and a coding reference region are used as each processing region, and the following operations are performed for each processing region:
[0030] Determine a second pixel difference between every two adjacent pixels based on the pixel values of each pixel included in a processing area;
[0031] Based on the obtained second pixel difference values, a total gradient value of a processing area is determined.
[0032] In a possible implementation, when determining the prediction value of each pixel included in the block to be encoded based on each pixel value included in the encoded block corresponding to each of the at least one target prediction regions, the generating module is used to:
[0033] For each coded block corresponding to at least one target prediction region, the following operations are respectively performed: a pixel value of each pixel point included in one coded block is respectively used as a pixel intermediate value of a pixel point corresponding to a position in the block to be coded;
[0034] Based on the respective intermediate values of the pixels in the block to be encoded, the pixel prediction values of the pixels in the block to be encoded are determined.
[0035] In a possible implementation, after determining the prediction value of each pixel included in the block to be encoded based on each pixel value included in the encoded block corresponding to each of the at least one target prediction regions, the device further includes a prediction mode determination module, the prediction mode determination module being configured to:
[0036] Determining the distortion degree of the region matching mode based on the original value of each pixel and the predicted value of each pixel contained in the block to be encoded;
[0037] For each other intra-frame prediction mode, the following operations are respectively performed: using an intra-frame prediction mode to perform prediction processing on the block to be encoded, obtaining a test block corresponding to the block to be encoded, and based on the test block, obtaining prediction values of each other pixel included in the block to be encoded, and determining a distortion degree of an intra-frame prediction mode based on each other pixel prediction value and each pixel original value;
[0038] From the obtained distortion degrees, a target distortion degree that meets the distortion condition is screened out, and a prediction mode corresponding to the target distortion degree is used as a target prediction mode.
[0039] In a possible implementation manner, after the prediction mode corresponding to the target distortion is used as the target prediction mode, the device further includes a coding module, which is used to:
[0040] Determine the prediction value of each target pixel included in the block to be encoded based on the target prediction mode;
[0041] The block to be coded is coded according to the prediction value of each target pixel and the target prediction mode to obtain a coded bitstream, wherein the coded bitstream at least includes a mode identifier, and the mode identifier is used to indicate the target prediction mode of the block to be coded.
[0042] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the above-mentioned intra-frame prediction methods are implemented.
[0043] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any one of the above-mentioned intra-frame prediction methods.
[0044] In a fifth aspect, an embodiment of the present application further provides a computer program product, including computer instructions, which are executed by a processor to perform the steps of any of the above-mentioned intra-frame prediction methods.
[0045] The beneficial effects of this application are as follows:
[0046] In an embodiment of the present application, in response to a received encoding instruction, the server obtains an encoding reference area of a block to be encoded in a frame to be encoded, searches in an encoded pixel area in the frame to be encoded to obtain at least one target prediction area whose pixel difference with the encoding reference area meets a preset difference condition, and whose shape and size are the same as the encoding reference area, and whose texture similarity with the encoding reference area meets a preset texture similarity condition, and determines the prediction value of each pixel contained in the block to be encoded based on the encoded blocks corresponding to each of the obtained at least one target prediction area. In this way, when searching for at least one matching target prediction area in the encoded pixel area in the frame to be encoded, the matching is not only performed from the dimension of the pixel difference between the two areas, but also from the dimension of the texture similarity between the two areas. Since the similarity between the target prediction area whose pixel difference meets the preset difference condition and the texture similarity meets the preset texture similarity condition and the encoded reference area is high, a more accurate at least one target prediction area can be determined by comprehensively considering the pixel difference and the texture similarity. In addition, since the pixel value of the encoded block corresponding to a single target prediction area is used as the pixel prediction value of the block to be encoded, the possibility of prediction mismatch is relatively large. Therefore, in order to reduce the mismatch probability, the pixel prediction value of each pixel point contained in the block to be encoded is determined based on the pixel value of the encoded block corresponding to at least one target prediction area, thereby reducing the uncertainty of the prediction of the single target prediction area, thereby obtaining a more accurate pixel prediction value and improving the accuracy of the area matching prediction.
[0047] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Figure 1 A schematic diagram of a regional matching mode;
[0049] Figure 2 This is an optional schematic diagram of an application scenario in an embodiment of the present application;
[0050] Figure 3 is a flowchart of an implementation method of an intra-frame prediction method in an embodiment of the present application;
[0051] Figure 4 This is a schematic diagram of the coding reference area in an embodiment of the present application;
[0052] Figure 5 A schematic diagram of a process for obtaining at least one initial prediction area in an embodiment of the present application;
[0053] Figure 6A schematic diagram of obtaining each candidate region in an embodiment of the present application;
[0054] Figure 7 A schematic diagram of obtaining at least one initial prediction area in an embodiment of the present application;
[0055] Figure 8 A schematic diagram of a process for obtaining at least one target prediction area in an embodiment of the present application;
[0056] Fig. 9 A schematic diagram of a process for determining a total gradient value of a processing area in an embodiment of the present application;
[0057] Fig.10 A schematic diagram of determining a total gradient value of a processing area in an embodiment of the present application;
[0058] Fig.11 A schematic diagram of obtaining at least one target prediction area in an embodiment of the present application;
[0059] Fig.12 A schematic diagram of a process for obtaining predicted values of each pixel contained in a block to be encoded in an embodiment of the present application;
[0060] Fig.13 A schematic diagram of an encoded block corresponding to a target prediction area in an embodiment of the present application;
[0061] Fig.14 A schematic diagram of a process for determining a target prediction model in an embodiment of the present application;
[0062] Fig.15 A schematic diagram of determining the distortion degree of the area matching mode in an embodiment of the present application;
[0063] Fig.16 It is a schematic diagram of determining the target prediction mode in an embodiment of the present application, and the distortion condition is the minimum distortion degree;
[0064] Fig.17 A schematic diagram of a process for encoding a block to be encoded in an embodiment of the present application;
[0065] Fig.18 This is an example diagram of the intra-frame prediction method in the embodiment of the present application;
[0066] Fig.19 A schematic diagram of the structure of an intra-frame prediction device provided in an embodiment of the present application;
[0067] Fig. 20 A schematic diagram of a hardware structure of a computer device in an embodiment of the present application;
[0068] Fig.21 A schematic diagram of the hardware composition structure of another computer device in an embodiment of the present application. DETAILED DESCRIPTION
[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0070] To facilitate understanding of the technical solutions provided in the embodiments of the present application, some key terms used in the embodiments of the present application are explained here:
[0071] Intra-frame prediction: Use the encoded pixels in the current frame image to generate pixel prediction values for the pixels to be encoded.
[0072] Planar mode: Use the pixel values on the adjacent boundaries in the four directions of up, down, left, and right corresponding to the pixels to be predicted of the block to be encoded as reference pixel values, and obtain the predicted value of the block to be encoded by averaging calculation.
[0073] Direct Current (DC) mode: All prediction values within the block to be encoded are the average values of the reference pixel values on the left and top of the block to be encoded.
[0074] Multi-angle mode: All prediction values in the block to be encoded are reference pixel values in the angle direction. If the reference pixel point pointed to by the angle direction is an integer pixel point, the pixel value of the integer pixel point is used to fill it. Otherwise, the pixel values of the integer pixels on the left and right sides of the reference pixel point are interpolated and then filled.
[0075] The following is a brief introduction to the design concept of the embodiment of the present application:
[0076] At present, with the rapid development of Internet technology, video image interaction can be applied to a variety of different scenarios, such as video calls, remote conferences, and short videos.
[0077] Since the amount of video image data is relatively large, it is necessary to encode and decode the video image during video image interaction to reduce the network bandwidth and storage space during the transmission process. Among them, video encoding mainly includes acquisition, prediction, quantization and entropy coding. The prediction is divided into intra-frame prediction and inter-frame prediction, which are used to remove spatial redundancy and temporal redundancy respectively. There are multiple prediction modes for intra-frame prediction, such as area matching mode, Planar mode, DC mode, multi-angle mode, etc.
[0078] In the prior art, the area matching mode is to determine a target area with the smallest pixel value error with the coding reference area based on the coding reference area formed by the upper left part of the block to be coded, and then obtain the corresponding coded block as the prediction block of the block to be coded according to the target area.
[0079] like Figure 1 As shown, the coding reference area refers to the upper left part L0 of the block to be coded CU, L1 is the target area, and the coded block P1 corresponding to L1 is used as the prediction block of the block to be coded.
[0080] However, when performing region matching prediction using the method in the related art, two regions with small pixel value errors may not be similar. Therefore, if the region with the smallest pixel value error is used as the target region, it will affect the prediction result of the block to be encoded. Determining the target region of the encoding reference area based only on the pixel value error will lead to inaccurate prediction results, thereby causing poor video compression effects.
[0081] In view of this, in an embodiment of the present application, an intra-frame prediction method, device, electronic device and storage medium are proposed to obtain a coding reference area of a block to be encoded in a frame to be encoded, the coding reference area including: encoded pixels within a set range around the block to be encoded; based on the coding reference area, at least one initial prediction area is searched in the encoded pixel area in the frame to be encoded, wherein the pixel difference between each initial prediction area and the coding reference area meets a preset difference condition and has the same shape and size as the coding reference area; at least one target prediction area is selected from the at least one initial prediction area; based on the pixel values contained in the encoded blocks corresponding to each of the at least one target prediction areas, the prediction values of each pixel contained in the block to be encoded are determined.
[0082] In this way, at least one initial prediction region is first obtained based on the pixel difference, then at least one target prediction region is screened out from the initial prediction region based on the texture similarity, and finally, the prediction value of each pixel of the block to be encoded is determined based on the at least one target prediction region. When obtaining the at least one matching target prediction region, the matching is performed not only from the dimension of the pixel difference between the two regions, but also from the dimension of the texture similarity between the two regions, so that at least one more accurate target prediction region can be determined, thereby obtaining a more accurate pixel prediction value, improving the accuracy of the region matching prediction, and improving the video compression effect.
[0083] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.
[0084] like Figure 2 As shown, it is a schematic diagram of an application scenario in an embodiment of the present application. The application scenario schematic diagram includes a terminal device 210 and a server 220, and the terminal device 210 and the server 220 can communicate with each other through a communication network.
[0085] In an optional implementation, the communication network may be a wired network or a wireless network. Therefore, the terminal device 210 and the server 220 may be directly or indirectly connected via a wired or wireless communication. For example, the terminal device 210 may be indirectly connected to the server 220 via a wireless access point, or the terminal device 210 may be directly connected to the server 220 via the Internet, and this application does not limit this.
[0086] In the embodiment of the present application, the terminal device 210 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, car terminals and other devices; various clients can be installed on the terminal device, and the client can be an application (such as a browser, game software, etc.), or a web page, a small program, etc.
[0087] Server 220 can be an independent physical server 220, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.
[0088] It should be noted that the intra-frame prediction method in the embodiment of the present application can be executed by the server 220 or the terminal device 210 alone, or can be executed by the server 220 and the terminal device 210 together. For example, the server executes alone, encodes the video image frame through the server, and sends the encoded code stream to the terminal device or other servers.
[0089] It should be noted that Figure 2 The figure is only an example. In fact, the number of terminal devices 210 and servers 220 is not limited and is not specifically limited in the embodiments of the present application.
[0090] The following describes the intra-frame prediction method provided by the exemplary embodiment of the present application in combination with the application scenario described above and with reference to the accompanying drawings. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this regard.
[0091] See also Figure 3As shown, it is an implementation flow chart of an intra-frame prediction method provided in an embodiment of the present application. Here, a server is used as an example for introduction. The specific implementation process of the method is as follows:
[0092] Step 30: Obtain a coding reference area of a block to be coded in a frame to be coded.
[0093] The coding reference area includes: coded pixels within a set range around the block to be coded.
[0094] In an embodiment of the present application, in response to a received encoding instruction, a frame to be encoded in a video to be encoded is obtained, a frame to be encoded is divided into non-overlapping image blocks, and an encoding reference area of the current block to be encoded is obtained.
[0095] For details, see Figure 4 As shown, it is a schematic diagram of the coding reference area in the embodiment of the present application. The coding reference area can be an "L"-shaped area on the upper and left sides of the block to be encoded, or it can be an area on the upper side of the block to be encoded, or it can be an area on the left side of the block to be encoded. The coding reference area can be adjacent to the block to be encoded, or it can be non-adjacent to the block to be encoded. The embodiment of the present application does not limit the shape and position of the coding reference area.
[0096] Step 31: Based on the coding reference area, search and obtain at least one initial prediction area in the coded pixel area in the frame to be coded.
[0097] The pixel difference between each initial prediction region and the coding reference region meets a preset difference condition and has the same shape and size as the coding reference region.
[0098] Specifically, when executing step 31, the server specifically performs the following operations. Figure 5 As shown, it is a schematic diagram of the process of obtaining at least one initial prediction area in the embodiment of the present application. Figure 5 , and explain in detail the specific operations performed:
[0099] Step 310: Search and obtain candidate regions in the encoded pixel region in the frame to be encoded according to the shape and size of the encoding reference region.
[0100] In an embodiment of the present application, a global search strategy is adopted to traverse the encoded pixel area in the frame to be encoded according to a preset starting point and step size to obtain candidate areas with the same shape and size as the encoding reference area.
[0101] Among them, the preset starting point is any pixel point in the frame to be encoded, and the preset step size can be the step size of one pixel point or the step size of multiple pixels points. The embodiment of the present application does not limit this.
[0102] For example, see Figure 6 As shown, it is a schematic diagram of obtaining each candidate area in an embodiment of the present application. Assuming that the coding reference area is an "L"-shaped area composed of the encoded pixels on the upper and left sides of the block to be encoded, starting from the upper left corner pixel in the frame to be encoded, the search is performed from left to right and then from top to bottom in the encoded pixel area with a step size of one pixel to obtain each candidate area, S1 is the first candidate area to be searched, and S2 is the second candidate area to be searched.
[0103] In this way, adopting a global search strategy and searching with a step size of one pixel can improve the accuracy of the search and obtain more accurate candidate regions.
[0104] In addition, it is worth mentioning that in order to reduce the search time and improve the search efficiency, the search can be performed with a step size of multiple pixels, or a sampling search strategy can be adopted.
[0105] Step 311: For each candidate area, perform the following operations respectively: based on each pixel value contained in a candidate area and each pixel value contained in the coding reference area, determine the first pixel difference between two pixel points at corresponding positions respectively, and based on each obtained first pixel difference, determine the pixel difference between a candidate area and the coding reference area.
[0106] In an embodiment of the present application, after obtaining each candidate area, the following operations are performed for each candidate area: based on the pixel values of two pixel points at corresponding positions in a candidate area and the coding reference area, a first pixel difference between two pixel points at corresponding positions is determined, and then based on each obtained first pixel difference, the pixel difference between a candidate area and the coding reference area is determined.
[0107] The pixel difference may be a sum of absolute differences, a mean square error value, a mean absolute error, etc., which is not limited in the embodiment of the present application.
[0108] Specifically, the pixel difference is the sum of absolute differences, and the absolute values of the obtained first pixel differences are taken respectively, and the absolute values are summed to obtain the pixel difference between a candidate area and the encoding reference area.
[0109] Among them, the pixel difference between a candidate area and the encoded reference area can be expressed as: LOSS(i) = ∑|L0(x, y)-Si(x, y)|, L0(x, y) is the pixel value of the pixel point with coordinates (x, y) in the encoded reference area, and Si(x, y) is the pixel value of the pixel point with coordinates (x, y) in the i-th candidate area.
[0110] For example, assuming that the first pixel difference values between the second candidate area and the encoding reference area are 1, 2, 3, 4, 5, 6, 7, 8, and 9 respectively, the pixel difference between the second candidate area and the encoding reference area is 45.
[0111] Step 312: Select at least one initial prediction region whose pixel difference meets a preset difference condition from each candidate region.
[0112] In the embodiment of the present application, after obtaining the pixel difference between each candidate area and the encoding reference area, at least one target pixel difference whose pixel difference meets the preset difference condition is selected from the pixel differences, and the candidate area corresponding to the target pixel difference is used as an initial prediction area.
[0113] The preset difference condition may be a minimum difference of N pixels, or a pixel difference less than a preset difference threshold, which is not limited in the present application.
[0114] For example, see Figure 7 As shown, it is a schematic diagram of obtaining at least one initial prediction area in an embodiment of the present application. Assuming that the preset difference condition is the minimum 8 pixel differences, the obtained pixel differences are 10, 20, 30, 15, 40, 26, 37, 58, 70, 90, 100, 32, 56, 22, 13, and 57 respectively. Then, the pixel differences are sorted, and the sorting results are 10, 13, 15, 20, 22, 26, 30, 32, 37, 40, 56, 57, 58, 70, 90, 100, and at least one target pixel difference is 10, 13, 15, 20, 22, 26, 30, and 32 respectively. The candidate area corresponding to the at least one target pixel difference is respectively used as an initial prediction area.
[0115] For another example, assuming that the preset difference condition is that the pixel difference is less than the preset difference threshold, the preset difference threshold is 30, and the obtained pixel differences are 10, 20, 30, 15, 40, 26, 37, 58, 70, 90, 100, 32, 56, 22, 13, and 57 respectively, then each pixel difference is compared with the preset difference threshold 25 respectively, and at least one target pixel difference is obtained, which is 10, 13, 15, 10, and 22 respectively, and the candidate area corresponding to the at least one target pixel difference is respectively used as an initial prediction area.
[0116] Step 32: Select at least one target prediction region from at least one initial prediction region.
[0117] The texture similarity between each target prediction region and the coding reference region meets a preset texture similarity condition.
[0118] Specifically, when executing step 32, the server specifically performs the following operations. Figure 8 As shown, it is a schematic diagram of the process of obtaining at least one target prediction area in the embodiment of the present application. Figure 8 , and explain in detail the specific operations performed:
[0119] Step 320: Determine the total gradient value of at least one initial prediction region and the coding reference region based on the pixel values respectively contained in the at least one initial prediction region and the coding reference region.
[0120] In an embodiment of the present application, at least one initial prediction area and a coding reference area are used as each processing area, and the following operations are performed for each processing area: based on the pixel value of a processing area, a total gradient value of a processing area is determined.
[0121] Specifically, when determining the total gradient value of a processing area based on the pixel value of a processing area, the server specifically performs the following operations. Fig. 9 As shown in the figure, it is a schematic diagram of the process of determining the total gradient value of a processing area in the embodiment of the present application. Fig. 9 , and explain in detail the specific operations performed:
[0122] Step 3200: Determine a second pixel difference between every two adjacent pixels based on the pixel values of each pixel contained in a processing area.
[0123] In an embodiment of the present application, for every two adjacent pixel points contained in a processing area, the following operations are performed: based on the respective pixel values of the two adjacent pixel points, the difference between the two pixel values is calculated, and the obtained difference is used as the second pixel difference between the two adjacent pixel points.
[0124] For example, if the pixel values of two adjacent pixel points are 99 and 100 respectively, then the second pixel difference between the two adjacent pixel points is 100-99=1.
[0125] Step 3201: Determine a total gradient value of a processing area based on the obtained second pixel difference values.
[0126] In the embodiment of the present application, after obtaining the second pixel difference values of a processing area, the second pixel difference values are summed to obtain the total gradient value of the processing area.
[0127] For example, see Fig.10As shown, it is a schematic diagram of determining the total gradient value of a processing area in an embodiment of the present application. Assuming that the coding reference area is an "L"-shaped area composed of encoded pixels on the upper and left sides of the block to be coded, the second pixel difference between every two adjacent upper pixels is determined based on the pixel values of each upper pixel point included in a processing area, and the second pixel difference between every two adjacent left pixel points is determined based on the pixel values of each left pixel point included in a processing area. The second pixel differences are summed to obtain the total gradient value of the processing area.
[0128] Among them, the total gradient value of a processing area can be expressed as: W is the width of the block to be encoded, H is the height of the block to be encoded, Lj(x, -1) is the pixel value of an upper pixel in the jth processing area, and Lj(-1, y) is the pixel value of a left pixel in the ith processing area.
[0129] In addition, it is worth noting that after obtaining the second pixel difference values of a processing area, the average value of the second pixel difference values can be calculated, and the obtained average value can be used as the total gradient value of the processing area. The embodiment of the present application does not limit this.
[0130] In this way, based on the second pixel difference between every two adjacent pixels in a processing area, the total gradient value of the processing area is determined, and the second pixel difference between two adjacent pixels is used to measure the texture features of the processing area, so that accurate texture features can be quickly obtained, thereby facilitating the quick acquisition of accurate texture similarity.
[0131] Step 321: for at least one initial prediction region, perform the following operations respectively: based on the total gradient value of the initial prediction region and the total gradient value of the coding reference region, determine the gradient difference between the initial prediction region and the coding reference region.
[0132] In an embodiment of the present application, after obtaining the total gradient values of at least one initial prediction region and the coding reference region, the following operations are performed for at least one initial prediction region: the difference between the total gradient value of an initial prediction region and the total gradient value of the coding reference region is calculated, and the obtained difference is used as the gradient difference between the initial prediction region and the coding reference region.
[0133] Among them, the gradient difference between an initial prediction area and the coding reference area can be expressed as D=|G(N)-G(0)|, G(N) is the total gradient value of the Nth initial prediction area, and G(0) is the total gradient value of the coding reference area.
[0134] For example, assuming that the total gradient value of the second initial prediction region is 50 and the total gradient value of the coding reference region is 40, the gradient difference between the second initial prediction region and the coding reference region is 50-40=10.
[0135] Step 322: Select at least one target prediction region whose gradient difference meets a preset difference condition from at least one initial prediction region.
[0136] In an embodiment of the present application, after obtaining the gradient difference between at least one initial prediction region and the encoded reference region, at least one target gradient difference whose gradient difference meets a preset difference condition is selected from each gradient difference, and an initial prediction region corresponding to a target gradient difference is used as a target prediction region.
[0137] Among them, the preset difference condition can be the minimum M gradient differences, or the gradient difference can be less than the preset difference threshold, and the present application does not limit this.
[0138] For example, see Fig.11 As shown, it is a schematic diagram of obtaining at least one target prediction area in an embodiment of the present application. Assuming that the preset difference condition is the minimum 4 gradient differences, the obtained gradient differences are 10, 30, 15, 25, 9, 37, 58, and 70, respectively. Then, the gradient differences are sorted, and the sorting results are 9, 10, 15, 25, 30, 37, 58, and 70, respectively. At least one target gradient difference is 9, 10, 15, and 25, respectively, and the initial prediction area corresponding to at least one target gradient difference is used as a target prediction area.
[0139] For another example, assuming that the preset difference condition is that the gradient difference is less than the preset difference threshold, the preset difference threshold is 25, and the obtained gradient differences are 10, 30, 15, 25, 9, 37, 58, and 70, respectively, then each gradient difference is compared with the preset difference threshold 25, and at least one target gradient difference is obtained, which is 9, 10, 15, and 25, respectively, and the initial prediction area corresponding to at least one target gradient difference is respectively used as a target prediction area.
[0140] In this way, based on the gradient difference between at least one initial prediction area and the coding reference area, at least one target prediction area whose gradient difference meets the preset difference condition is selected, and the gradient difference between an initial prediction area and the coding reference area is used to measure the texture similarity between the initial prediction area and the coding reference area, so that accurate texture similarity can be quickly obtained, and screening from the dimension of texture similarity between two areas can determine at least one more accurate target prediction area, so that more accurate pixel prediction values can be obtained, thereby improving the accuracy of regional matching prediction and improving video compression effect.
[0141] Step 33: Determine the prediction value of each pixel included in the block to be encoded based on the pixel values included in the encoded blocks corresponding to at least one target prediction area.
[0142] Specifically, when executing step 33, the server specifically performs the following operations. Fig.12 As shown in FIG. 1 , it is a schematic diagram of a flow chart of obtaining the predicted values of each pixel contained in the block to be encoded in an embodiment of the present application. Fig.12 , and explain in detail the specific operations performed:
[0143] Step 330: for each coded block corresponding to at least one target prediction region, perform the following operations: respectively use the pixel value of each pixel point contained in an coded block as the pixel intermediate value of the pixel point corresponding to the position in the block to be coded.
[0144] In the embodiment of the present application, the relative position relationship between an encoded block corresponding to a target prediction region and the target prediction region is the same as the relative position relationship between the encoding reference region and the block to be encoded.
[0145] For example, see Fig.13 As shown, it is a schematic diagram of the encoded block corresponding to the target prediction area in the embodiment of the present application. The relative position relationship between the target prediction area L1 and the encoded block P1, the relative position relationship between the target prediction area L2 and the encoded block P2, the relative position relationship between the target prediction area L3 and the encoded block P3, the relative position relationship between the target prediction area L4 and the encoded block P4 are the same as the relative position relationship between the coding reference area L0 and the block to be encoded CU. For at least one encoded block P1, P2, P3, and P4, the following operations are performed: the pixel values of each pixel point contained in an encoded block are respectively used as the pixel intermediate values of the pixel points corresponding to the position in the block to be encoded.
[0146] Step 331: Determine the pixel prediction value of each pixel point included in the block to be encoded based on the respective intermediate values of each pixel point included in the block to be encoded.
[0147] In an embodiment of the present application, after obtaining the intermediate values of each pixel of each pixel point included in the block to be encoded, the following operations are performed for each pixel point included in the block to be encoded: the average value of the intermediate values of each pixel point of a pixel point is calculated, and the obtained average value is used as the pixel prediction value of the pixel point.
[0148] For example, if the intermediate values of pixels of a pixel in the to-be-encoded block are 89, 94, 98, and 85, then the pixel prediction value of the pixel is (89+94+98+85) / 4=91.5.
[0149] In this way, due to the complexity of the image, the possibility of prediction mismatch is relatively high when the pixel value of the encoded block corresponding to a single target prediction area is used as the pixel prediction value of the block to be encoded. Therefore, in order to reduce the mismatch probability, the pixel prediction value of each pixel point contained in the block to be encoded is determined based on the pixel value of the encoded block corresponding to at least one target prediction area. This can expand the source of the final prediction and reduce the uncertainty of the prediction of a single target prediction area.
[0150] In addition, it is worth noting that when determining the pixel prediction value of each pixel point contained in the block to be encoded, the weighted average of the intermediate values of each pixel point can also be calculated respectively, and the obtained weighted average value can be used as the pixel prediction value of the corresponding pixel point. The embodiment of the present application does not limit this.
[0151] Specifically, for each pixel point included in the block to be encoded, the following operations are performed respectively: based on the weights of the intermediate values of each pixel point and the intermediate values of each pixel point, a weighted average value is determined, and the obtained weighted average value is used as the pixel prediction value of the pixel point.
[0152] Among them, when determining the weights of the intermediate values of each pixel of a pixel point, any of the following methods may be used, and the embodiments of the present application are not limited to this:
[0153] Method 1: Based on the texture similarity between at least one target prediction region and the coding reference region, the weights of the intermediate values of each pixel point are determined.
[0154] Method 2: Based on the pixel difference between at least one target prediction region and the coding reference region, the weights of the intermediate values of each pixel point are determined.
[0155] Method three: Based on the texture similarity and pixel difference between at least one target prediction region and the coding reference region, the weights of the intermediate values of each pixel point are determined.
[0156] Further, after determining the prediction values of each pixel contained in the block to be encoded based on the pixel values contained in the encoded blocks corresponding to at least one target prediction area, the target prediction model can also be determined based on the distortion degree, see Fig.14 As shown, it is a schematic diagram of the process of determining the target prediction model in the embodiment of the present application. Fig.14 , and explain in detail the specific operations performed:
[0157] Step 34: Determine the distortion of the region matching mode based on the original value of each pixel and the predicted value of each pixel contained in the block to be encoded.
[0158] In an embodiment of the present application, after obtaining the predicted value of each pixel contained in the block to be encoded, the following operations are performed for each pixel point contained in the block to be encoded: the pixel error between the original pixel value and the pixel predicted value of a pixel point is calculated, and based on the obtained pixel errors, the distortion degree of the area matching mode is determined.
[0159] Among them, the distortion of the area matching mode can be expressed as W is the width of the block to be encoded, H is the height of the block to be encoded, pred(x, y) is the pixel prediction value of the pixel point with coordinates (x, y) in the block to be encoded, and orig(x, y) is the pixel original value of the pixel point with coordinates (x, y) in the block to be encoded.
[0160] For example, see Fig.15 As shown in FIG. 1 , it is a schematic diagram of determining the distortion degree of the region matching mode in an embodiment of the present application. Assuming that the pixel errors between the original pixel value and the predicted pixel value of each pixel point are 2, 4, 5, 3, 1, 3, 4, 5, 6, 5, 4, 3, 2, 1, 3, 1 respectively, the distortion degree of the region matching mode is 2 2 +4 2 +5 2 +3 2 +1 2 +3 2 +4 2 +5 2 +6 2 +5 2 +4 2 +3 2 +2 2 +1 2 +3 2 +1 2 =206.
[0161] Step 35: For each other intra-frame prediction mode, the following operations are performed respectively: adopt an intra-frame prediction mode to perform prediction processing on the block to be encoded, obtain a test block corresponding to the block to be encoded, and based on the test block, obtain prediction values of other pixels contained in the block to be encoded, and determine the distortion of an intra-frame prediction mode based on the prediction values of other pixels and the original value of each pixel.
[0162] Among them, other intra-frame prediction modes at least include: Planar mode, DC mode, multi-angle mode, etc., and the embodiments of the present application do not limit this.
[0163] For example, the Planar mode is adopted to perform prediction processing on the block to be encoded, and a test block corresponding to the block to be encoded is obtained. Based on the test block, the prediction value of each Planar pixel contained in the block to be encoded is obtained, and based on each Planar pixel prediction value and each pixel original value, the distortion degree of the Planar mode is determined.
[0164] Among them, the distortion of each other intra-frame prediction mode can be expressed as W is the width of the block to be encoded, H is the height of the block to be encoded, Pred(x, y) is the predicted value of other pixels of the pixel point with coordinates (x, y) in the block to be encoded, and Orig(x, y) is the original pixel value of the pixel point with coordinates (x, y) in the block to be encoded.
[0165] Step 36: Filter out a target distortion degree that meets the distortion condition from the obtained distortion degrees, and use the prediction mode corresponding to the target distortion degree as the target prediction mode.
[0166] In the embodiment of the present application, after obtaining each distortion degree, a target distortion degree whose distortion degree meets the distortion condition is selected from the obtained distortion degrees, and a prediction mode corresponding to a target distortion degree is used as a target prediction mode.
[0167] Among them, the distortion condition can be a minimum distortion degree, which is not limited in the implementation of this application.
[0168] For example, see Fig.16 As shown, it is a schematic diagram of determining the target prediction mode in an embodiment of the present application. The distortion condition is the minimum distortion. Assuming that the distortion of the area matching mode is 206, the distortion of the Planar mode is 240, the distortion of the DC mode is 260, and the distortion of the multi-angle mode is 255, the distortions are compared, and the distortion 206 is used as the target distortion, and a prediction mode corresponding to a target distortion 206, namely the area matching mode, is used as the target prediction mode.
[0169] In this way, by comparing the distortion of the region matching mode with the distortion of other intra-frame prediction modes, the target prediction mode can be determined, which can further improve the accuracy of intra-frame prediction, thereby improving the compression rate of intra-frame coding and reducing the transmission bandwidth of video data and the storage resources of video data.
[0170] Furthermore, after the prediction mode corresponding to the target distortion is used as the target prediction mode, the coding block can be encoded based on the target prediction mode. Fig.17 As shown, it is a schematic diagram of the process of encoding the block to be encoded in the embodiment of the present application. Fig.17 , and explain in detail the specific operations performed:
[0171] Step 37: Based on the target prediction mode, determine the prediction value of each target pixel included in the block to be encoded.
[0172] In the embodiment of the present application, the aforementioned intra-frame prediction is performed on the block to be encoded to obtain a target prediction mode of the block to be encoded, and each pixel prediction value corresponding to the target prediction mode is used as each target pixel prediction value of the block to be encoded.
[0173] For example, assuming that the target prediction mode is the region matching mode, each pixel prediction value corresponding to the region matching mode is used as each target pixel prediction value of the block to be encoded.
[0174] For another example, assuming that the target prediction mode is the Planar mode, each Planar pixel prediction value corresponding to the Planar mode is used as each target pixel prediction value of the to-be-encoded block.
[0175] Step 38: Encode the block to be encoded according to the prediction value of each target pixel and the target prediction mode to obtain an encoded bitstream.
[0176] The coded bitstream at least includes a mode identifier, and the mode identifier is used to indicate a target prediction mode of a block to be coded.
[0177] Specifically, the mode identifier may be set according to whether the target prediction mode is the region matching mode, and a one-to-one corresponding mode identifier may be set for each intra-frame prediction mode, which is not limited in the present application.
[0178] For example, the mode flag is set according to whether the target prediction mode is the area matching mode. If the target prediction mode is the area matching mode, the mode flag is set to "1"; if the target prediction mode is other intra-frame prediction modes, the mode flag is set to "0".
[0179] For another example, a corresponding mode identifier is set for each intra-frame prediction mode. If the target prediction mode is the area matching mode, the mode identifier is set to "1"; if the target prediction mode is the Planar mode, the mode identifier is set to "2"; if the target prediction mode is the DC mode, the mode identifier is set to "3"; if the target prediction mode is the multi-angle mode, the mode identifier is set to "4".
[0180] In the embodiment of the present application, the residual of each pixel point is obtained according to each target pixel prediction value and each original pixel value, and based on the residual and the target prediction mode, the coding block is encoded to obtain a coded bit stream.
[0181] For example, assuming that the original pixel values are 100, 97, 80, and 85, and the predicted values of the target pixels are 99, 95, 79, and 84, the residuals of the pixels are 1, 2, 1, and 1, respectively.
[0182] Furthermore, after the encoding block is encoded and compressed, the encoding block needs to be decoded when the video is played.
[0183] In an embodiment of the present application, in response to a received decoding instruction, the encoded code stream is parsed to obtain a target prediction mode and a residual of each pixel point contained in the block to be encoded, and then based on the target prediction mode, a prediction value of each target pixel is obtained, and finally, based on the residual of each pixel point and the target pixel prediction value, the original pixel value of each pixel point is determined.
[0184] Based on the above embodiment, the intra-frame prediction method in the embodiment of the present application is described in detail using a specific application scenario. Fig.18 As shown, it is an example diagram of the intra-frame prediction method in the embodiment of the present application, which specifically includes:
[0185] First, in response to the received encoding instruction, a frame to be encoded in the short video to be encoded is obtained, and an encoding reference area of a current block to be encoded in the frame to be encoded is obtained.
[0186] Then, based on the pixel difference between each candidate area and the coding reference area, at least one initial prediction area whose pixel difference meets the preset difference condition is selected from the encoded pixel area in the frame to be encoded, and based on the total gradient value of each of the at least one initial prediction area and the coding reference area, a gradient difference between an initial prediction area and the coding reference area is determined, and at least one target prediction area (L1, L2, L3, L4) whose gradient difference meets the preset difference condition is selected from the at least one initial prediction area.
[0187] Secondly, based on the pixel values contained in the encoded blocks (P1, P2, P3, P4) corresponding to at least one target prediction area, the pixel prediction values 99, 95, 79, 84 contained in the block to be encoded are determined.
[0188] Furthermore, based on the original value of each pixel contained in the block to be encoded and the predicted value of each pixel, the distortion of the regional matching mode is determined, and based on the predicted value of each other pixel and the original value of each pixel, the distortion of each other intra-frame prediction mode is determined, and the target prediction mode that meets the distortion conditions is selected.
[0189] Finally, based on the target prediction mode, the prediction values of each target pixel contained in the block to be encoded are determined, and according to the prediction values of each target pixel and the target prediction mode, the block to be encoded is encoded to obtain an encoded bitstream.
[0190] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides an intra-frame prediction device, see Fig.19 FIG. 1 is a schematic diagram of the structure of an intra-frame prediction device provided in an embodiment of the present application. The device 1900 includes:
[0191] The acquisition module 1901 is used to acquire a coding reference area of a block to be coded in a frame to be coded, where the coding reference area includes: coded pixels within a set range around the block to be coded;
[0192] The first processing module 1902 is used to search and obtain at least one initial prediction region in the encoded pixel region in the frame to be encoded based on the encoding reference region, wherein the pixel difference between each initial prediction region and the encoding reference region meets a preset difference condition and has the same shape and size as the encoding reference region;
[0193] The second processing module 1903 is used to select at least one target prediction region from the at least one initial prediction region, wherein the texture similarity between each target prediction region and the coding reference region meets a preset texture similarity condition;
[0194] The generating module 1904 is used to determine the prediction value of each pixel included in the block to be encoded based on the pixel values included in the encoded blocks corresponding to each of the at least one target prediction areas.
[0195] In a possible implementation, based on the coding reference area, when searching for at least one initial prediction area in the coded pixel area in the frame to be coded, the first processing module 1902 is used to:
[0196] According to the shape and size of the coding reference area, searching for each candidate area in the coded pixel area in the frame to be coded;
[0197] For each candidate region, the following operations are respectively performed: based on each pixel value included in the candidate region and each pixel value included in the coding reference region, first pixel difference values between two pixel points at corresponding positions are respectively determined, and based on each obtained first pixel difference value, pixel difference between the candidate region and the coding reference region is determined;
[0198] From each candidate region, at least one initial prediction region whose pixel difference meets a preset difference condition is selected.
[0199] In a possible implementation, when at least one target prediction region is selected from at least one initial prediction region, the second processing module 1903 is used to:
[0200] Determine the total gradient value of at least one initial prediction region and the coding reference region based on the pixel values respectively included in the at least one initial prediction region and the coding reference region;
[0201] For at least one initial prediction region, the following operations are respectively performed: based on a total gradient value of the initial prediction region and a total gradient value of the coding reference region, a gradient difference between the initial prediction region and the coding reference region is determined;
[0202] At least one target prediction region whose gradient difference meets a preset difference condition is selected from the at least one initial prediction region.
[0203] In a possible implementation, when determining the total gradient value of at least one initial prediction region and the coding reference region based on the pixel values respectively included in the at least one initial prediction region and the coding reference region, the second processing module 1903 is further configured to:
[0204] At least one initial prediction region and a coding reference region are used as each processing region, and the following operations are performed for each processing region:
[0205] Determine a second pixel difference between every two adjacent pixels based on the pixel values of each pixel included in a processing area;
[0206] Based on the obtained second pixel difference values, a total gradient value of a processing area is determined.
[0207] In a possible implementation, when determining the prediction value of each pixel included in the block to be encoded based on each pixel value included in the encoded block corresponding to each of the at least one target prediction regions, the generation module 1904 is used to:
[0208] For each coded block corresponding to at least one target prediction region, the following operations are respectively performed: a pixel value of each pixel point included in one coded block is respectively used as a pixel intermediate value of a pixel point corresponding to a position in the block to be coded;
[0209] Based on the respective intermediate values of the pixels in the block to be encoded, the pixel prediction values of the pixels in the block to be encoded are determined.
[0210] In a possible implementation, after determining the prediction value of each pixel included in the block to be encoded based on each pixel value included in the encoded block corresponding to each of the at least one target prediction regions, the apparatus further includes a prediction mode determination module 1905, which is used to:
[0211] Determining the distortion degree of the region matching mode based on the original value of each pixel and the predicted value of each pixel contained in the block to be encoded;
[0212] For each other intra-frame prediction mode, the following operations are respectively performed: using an intra-frame prediction mode to perform prediction processing on the block to be encoded, obtaining a test block corresponding to the block to be encoded, and based on the test block, obtaining prediction values of each other pixel included in the block to be encoded, and determining a distortion degree of an intra-frame prediction mode based on each other pixel prediction value and each pixel original value;
[0213] From the obtained distortion degrees, a target distortion degree that meets the distortion condition is screened out, and a prediction mode corresponding to the target distortion degree is used as a target prediction mode.
[0214] In a possible implementation, after the prediction mode corresponding to the target distortion is used as the target prediction mode, the apparatus further includes a coding module 1906, which is used to:
[0215] Determine the prediction value of each target pixel included in the block to be encoded based on the target prediction mode;
[0216] The block to be coded is coded according to the prediction value of each target pixel and the target prediction mode to obtain a coded bitstream, wherein the coded bitstream at least includes a mode identifier, and the mode identifier is used to indicate the target prediction mode of the block to be coded.
[0217] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.
[0218] After introducing the intra-frame prediction method and apparatus according to an exemplary embodiment of the present application, next, a computing device according to another exemplary embodiment of the present application is introduced.
[0219] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.
[0220] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a computing device. In one embodiment, the computing device may be a server, such as Figure 2 In this embodiment, the structure of the computer device 2000 is as follows: Fig. 20 As shown, it may include at least a memory 2001 , a communication module 2003 , and at least one processor 2002 .
[0221] The memory 2001 is used to store computer programs executed by the processor 2002. The memory 2001 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.
[0222] The memory 2001 may be a volatile memory, such as a random-access memory (RAM); the memory 2001 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 2001 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2001 may be a combination of the above memories.
[0223] The processor 2002 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 2002 is configured to implement the above intra-frame prediction method when calling the computer program stored in the memory 2001 .
[0224] The communication module 2003 is used to communicate with terminal devices and other servers.
[0225] The specific connection medium between the memory 2001, the communication module 2003 and the processor 2002 is not limited in the embodiment of the present application. Fig. 20 In the embodiment, the memory 2001 and the processor 2002 are connected via a bus 2004. The bus 2004 is Fig. 20 The connections between the other components are only for illustration and are not intended to be limiting. The bus 2004 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig. 20 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.
[0226] The memory 2001 stores a computer storage medium, and the computer storage medium stores computer executable instructions, and the computer executable instructions are used to implement the intra-frame prediction method of the embodiment of the present application. The processor 2002 is used to execute the above intra-frame prediction method.
[0227] In another embodiment, the computing device may also be other computing devices, such as Figure 2 The terminal device 210 shown in FIG. 1 is a terminal device 210 shown in FIG. 1 . In this embodiment, the structure of the computing device can be as follows: Fig.21 As shown, it includes: a communication component 2110, a memory 2120, a display unit 2130, a camera 2140, a sensor 2150, an audio circuit 2160, a Bluetooth module 2170, a processor 2180 and other components.
[0228] The communication component 2110 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the object to send and receive information through the WiFi module.
[0229] The memory 2120 can be used to store software programs and data. The processor 2180 executes various functions and data processing of the terminal device 210 by running the software programs or data stored in the memory 2120. The memory 2120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 2120 stores an operating system that enables the terminal device 210 to run. In the present application, the memory 2120 can store an operating system and various application programs, and can also store a computer program for executing the intra-frame prediction method of the embodiment of the present application.
[0230] The display unit 2130 may also be used to display information input by the object or information provided to the object and a graphical user interface (GUI) of various menus of the terminal device 210. Specifically, the display unit 2130 may include a display screen 2132 disposed on the front of the terminal device 210. The display screen 2132 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 2130 may be used to display the intra-frame prediction interface, the video playback interface, etc. in the embodiment of the present application.
[0231] The display unit 2130 can also be used to receive input digital or character information, and generate signal input related to the object setting and function control of the terminal device 210. Specifically, the display unit 2130 may include a touch screen 2131 set on the front of the terminal device 210, which can collect touch operations of objects on or near it, such as clicking a button, dragging a scroll box, etc.
[0232] The touch screen 2131 can be covered on the display screen 2132, or the touch screen 2131 and the display screen 2132 can be integrated to realize the input and output functions of the physical terminal device 210, and the integrated display screen can be referred to as a touch display screen. In this application, the display unit 2130 can display the application and the corresponding operation steps.
[0233] The camera 2140 can be used to capture static images, and the subject can publish the image taken by the camera 2140 through the application. The camera 2140 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the processor 2180 to convert it into a digital image signal.
[0234] The physical terminal device may also include at least one sensor 2150, such as an acceleration sensor 2151, a distance sensor 2152, a fingerprint sensor 2153, and a temperature sensor 2154. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.
[0235] The audio circuit 2160, the speaker 2161, and the microphone 2162 can provide an audio interface between the object and the terminal device 210. The audio circuit 2160 can transmit the electrical signal converted from the received audio data to the speaker 2161, which is converted into a sound signal for output. The physical terminal device 210 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 2162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 2160 and converted into audio data, and then the audio data is output to the communication component 2110 to be sent to, for example, another physical terminal device 210, or the audio data is output to the memory 2120 for further processing.
[0236] The Bluetooth module 2170 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the physical terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 2170 to exchange data.
[0237] The processor 2180 is the control center of the physical terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 2120 and calling data stored in the memory 2120. In some embodiments, the processor 2180 may include one or more processing units; the processor 2180 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 2180. In the present application, the processor 2180 can run the operating system, application programs, user interface display and touch response, as well as the intra-frame prediction method of the embodiment of the present application. In addition, the processor 2180 is coupled to the display unit 2130.
[0238] In some possible implementations, various aspects of the intra-frame prediction method provided by the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on a computing device, the computer program is used to enable the computing device to execute the steps of the intra-frame prediction method according to various exemplary implementations of the present application described above in this specification.
[0239] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0240] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.
[0241] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.
[0242] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0243] The computer program for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The computer program may be executed entirely on the user's computer device, partially on the user's computer device, as a separate software package, partially on the user's computer device and partially on a remote computer device, or entirely on the remote computer device. In the case of a remote computer device, the remote computer device may be connected to the user's computer device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer device (e.g., via the Internet using an Internet service provider).
[0244] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.
[0245] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0246] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0247] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0248] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0249] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0250] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. An intra-frame prediction method, characterized in that: include: Acquire a coding reference area of a block to be coded in a frame to be coded, wherein the coding reference area includes: coded pixels within a set range around the block to be coded; Based on the coding reference area, searching for at least one initial prediction area in the coded pixel area in the frame to be coded, wherein the pixel difference between each of the initial prediction areas and the coding reference area meets a preset difference condition and has the same shape and size as the coding reference area; Selecting at least one target prediction region from the at least one initial prediction region, wherein the texture similarity between each of the target prediction regions and the coding reference region meets a preset texture similarity condition; Based on the pixel values contained in the encoded blocks corresponding to the at least one target prediction area, the prediction values of the pixels contained in the block to be encoded are determined.
2. The method according to claim 1, characterized in that The step of searching and obtaining at least one initial prediction area in an encoded pixel area in the frame to be encoded based on the encoding reference area comprises: According to the shape and size of the coding reference area, searching for each candidate area in the coded pixel area in the frame to be coded; For each candidate region, the following operations are respectively performed: based on each pixel value included in the candidate region and each pixel value included in the coding reference region, first pixel difference values between two pixel points at corresponding positions are respectively determined, and based on each obtained first pixel difference value, pixel difference between the candidate region and the coding reference region is determined; At least one initial prediction region whose pixel difference meets a preset difference condition is selected from the candidate regions.
3. The method according to claim 1, characterized in that The selecting at least one target prediction region from the at least one initial prediction region comprises: Determine the total gradient value of each of the at least one initial prediction region and the encoding reference region based on the pixel values respectively included in the at least one initial prediction region and the encoding reference region; For the at least one initial prediction region, the following operations are respectively performed: based on a total gradient value of the initial prediction region and a total gradient value of the coding reference region, determining a gradient difference between the initial prediction region and the coding reference region; At least one target prediction region whose gradient difference meets a preset difference condition is selected from the at least one initial prediction region.
4. The method according to claim 3, characterized in that The determining, based on the pixel values respectively included in the at least one initial prediction area and the encoding reference area, the total gradient value of each of the at least one initial prediction area and the encoding reference area comprises: The at least one initial prediction region and the coding reference region are used as respective processing regions, and the following operations are performed for each processing region: Determine a second pixel difference between every two adjacent pixels based on the pixel values of each pixel included in a processing area; Based on the obtained second pixel difference values, a total gradient value of the one processing area is determined.
5. The method according to claim 1, characterized in that The determining, based on each pixel value contained in the encoded block corresponding to each of the at least one target prediction area, each pixel prediction value contained in the block to be encoded includes: For each of the encoded blocks corresponding to the at least one target prediction area, the following operations are respectively performed: a pixel value of each pixel point included in an encoded block is respectively used as a pixel intermediate value of a pixel point corresponding to a position in the block to be encoded; Based on the respective intermediate values of the pixels of the pixels in the block to be encoded, respective pixel prediction values of the pixels in the block to be encoded are determined.
6. The method according to any one of claims 1 to 5, characterized in that: After determining the predicted values of each pixel contained in the block to be encoded based on the pixel values contained in the encoded blocks corresponding to the at least one target prediction area, the method further includes: Determining the distortion degree of the region matching mode based on the original value of each pixel contained in the block to be encoded and the predicted value of each pixel; For each other intra-frame prediction mode, the following operations are performed respectively: using an intra-frame prediction mode, predicting the block to be encoded, obtaining a test block corresponding to the block to be encoded, and obtaining, based on the test block, predicted values of other pixels contained in the block to be encoded, and determining the distortion degree of the intra-frame prediction mode based on the predicted values of other pixels and the original values of the pixels; A target distortion degree that meets the distortion condition is screened out from the obtained distortion degrees, and a prediction mode corresponding to the target distortion degree is used as a target prediction mode.
7. The method according to claim 6, characterized in that After the prediction mode corresponding to the target distortion is used as the target prediction mode, the method further includes: Based on the target prediction mode, determining the prediction value of each target pixel included in the block to be encoded; The block to be encoded is encoded according to the target pixel prediction values and the target prediction mode to obtain an encoded bitstream, wherein the encoded bitstream at least includes a mode identifier, and the mode identifier is used to indicate the target prediction mode of the block to be encoded.
8. An intra-frame prediction device, characterized in that: include: An acquisition module is used to acquire a coding reference area of a block to be coded in a frame to be coded, wherein the coding reference area includes: coded pixels within a set range around the block to be coded; A first processing module is configured to search for at least one initial prediction region in an encoded pixel region in the frame to be encoded based on the encoding reference region, wherein a pixel difference between each of the initial prediction regions and the encoding reference region meets a preset difference condition and has the same shape and size as the encoding reference region; A second processing module, configured to select at least one target prediction region from the at least one initial prediction region, wherein the texture similarity between each of the target prediction regions and the coding reference region meets a preset texture similarity condition; The generating module is used to determine the predicted values of each pixel contained in the block to be encoded based on the pixel values contained in the encoded blocks corresponding to each of the at least one target prediction area.
9. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The method comprises a program code, and when the program code is run on a computing device, the program code is used to make the computing device execute the steps of the method according to any one of claims 1 to 7.
11. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 7 when being executed by a processor.
