Multimedia file coding and decoding method and device, processing system and electronic equipment

Abstract

The embodiment of the invention discloses a multimedia file coding method and device, a multimedia file decoding method and device, a processing system and electronic equipment, and relates to the fields of videos, cloud technologies and the like. The method comprises the following steps: pre-coding a parameter value of a first coding unit of a multimedia file under a first pixel parameter to obtain a first coding component, and matching a reference coding unit for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by coding a parameter value of the reference coding unit under the first pixel parameter; acquiring a first parameter value of the reference coding unit under the second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format; and coding the first coding unit according to the first parameter value. By adopting the embodiment of the invention, the coding performance of the coding unit under the second pixel parameter is improved.

Description

Technical Field

[0001] The present application belongs to the field of coding and decoding technology, and may involve fields such as video and cloud technology, and in particular, to a multimedia file coding and decoding method, device, processing system and electronic device. Background Art

[0002] Multimedia files can be generated based on images, such as images or image sequences. Among them, a continuous image sequence can form a video stream, which is composed of continuous frames, and one frame is an image. In practical applications, in order to reduce the amount of data stored or transmitted in multimedia data, it is usually necessary to encode the multimedia files.

[0003] However, in the related art, when encoding multimedia files, a linear model obtained by reconstructing pixels around the encoding unit can be used for encoding prediction, but the linear relationship obtained by this method will have a certain deviation, which affects the accuracy of the encoding result. Therefore, there is still a need to improve the encoding method of multimedia files. Summary of the invention

[0004] The embodiments of the present application provide a multimedia file encoding and decoding method, device, processing system and electronic device method to solve the problem of poor multimedia file encoding performance in the related art.

[0005] Correspondingly, the embodiments of the present application also provide a multimedia file encoding device, a multimedia file decoding method and device, a multimedia file processing system, an electronic device, a computer-readable storage medium and a computer program product to ensure the implementation and application of the above method.

[0006] On the one hand, an embodiment of the present application provides a multimedia file encoding method, the method comprising:

[0007] Pre-encoding a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component;

[0008] According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter;

[0009] Obtaining a first parameter value of a reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0010] The first coding unit is encoded according to the first parameter value.

[0011] On the other hand, an embodiment of the present application provides a multimedia file decoding method, the method comprising:

[0012] Obtaining encoded data of multimedia files;

[0013] Decode the encoded data;

[0014] The encoded data is obtained by encoding the multimedia file in the following manner:

[0015] Performing an encoding operation on a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component;

[0016] According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter;

[0017] Obtaining a first parameter value of a reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0018] The first coding unit is encoded according to the first parameter value.

[0019] On the other hand, an embodiment of the present application provides a multimedia file processing system, the system comprising:

[0020] An encoding module, used for encoding the first multimedia file;

[0021] A decoding module, used for decoding the acquired coded data;

[0022] The encoding module encodes the first multimedia file according to the following method:

[0023] Performing a pre-coding operation on a parameter value of a first coding unit of a first multimedia file under a first pixel parameter to obtain a first coding component;

[0024] According to the first coding component, a first reference coding unit is matched for the first coding unit in the first multimedia file; wherein an error between the first reference coding component of the first reference coding unit and the first coding component meets a preset requirement, and the first reference coding component is obtained by encoding a parameter value of the first reference coding unit under a first pixel parameter;

[0025] Obtaining a first parameter value of a first reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0026] Encoding the first encoding unit according to the first parameter value;

[0027] The encoded data is obtained by encoding the second multimedia file in the following manner:

[0028] Performing a pre-coding operation on the parameter value of the second coding unit of the second multimedia file under the first pixel parameter to obtain a fourth coding component;

[0029] According to the fourth coding component, matching a second reference coding unit for the second coding unit in the second multimedia file; wherein an error between the second reference coding component of the second reference coding unit and the fourth coding component meets a preset requirement, and the second reference coding component is obtained by encoding the parameter value of the second reference coding unit under the first pixel parameter;

[0030] Obtain a third parameter value of the second reference coding unit under the second pixel parameter;

[0031] The fourth coding unit is encoded according to the third parameter value.

[0032] On the other hand, an embodiment of the present application provides a multimedia file encoding device, the device comprising:

[0033] An encoding module, configured to perform a pre-encoding operation on a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component;

[0034] A coding unit matching module, configured to match a reference matching unit for the first coding unit in the multimedia file according to the first coding component; wherein an error between a reference coding component of the reference matching unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter;

[0035] A parameter value acquisition module, used to acquire a first parameter value of a reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0036] The above encoding module is also used to encode the first encoding unit according to the first parameter value.

[0037] Optionally, the coding unit matching module matches the reference coding unit for the first coding unit in the multimedia file, which may include:

[0038] Determining position information of the first encoding unit in the multimedia file;

[0039] Using the above position information as a matching center, determining a matching area in the multimedia file;

[0040] A reference coding unit is matched for the first coding unit in the matching area.

[0041] Optionally, the above-mentioned position information may include pixel position parameters, width and height of the central pixel of the first coding unit;

[0042] The above-mentioned coding unit matching module determines the matching area in the multimedia file, which may include at least one of the following:

[0043] In the multimedia file, the target image frame including the first encoding unit is used as a matching area;

[0044] According to the width, the width threshold, the height and the height threshold, a matching area centered around the pixel position parameter is determined in the target image frame.

[0045] Optionally, the coding unit matching module matches the reference coding unit for the first coding unit in the multimedia file, which may include:

[0046] Determine a first error between the first coding component and the candidate coding component; wherein the candidate coding component is obtained by encoding a parameter value of a candidate coding unit in the matching area under the first pixel parameter;

[0047] A reference coding unit is determined from candidate coding units according to the first error.

[0048] Optionally, the coding unit matching module may determine the reference coding unit from the candidate coding units according to the first error, which may include at least one of the following:

[0049] Determine the candidate coding unit with the smallest first error as a reference coding unit;

[0050] A candidate coding unit whose first error is less than or equal to a preset threshold is determined as a reference coding unit.

[0051] Optionally, the encoding module may encode the first encoding unit according to the first parameter value, including:

[0052] Pre-coding the first parameter value to obtain a second coded component;

[0053] Using the candidate coding mode, and predicting a third coding component of the first coding unit under the second pixel parameter according to the second coding component;

[0054] Determine, according to the third coding component, a target coding component of the first coding unit under the second pixel parameter;

[0055] A first coding unit is encoded according to the first coding component and the target coding component.

[0056] Optionally, the encoding module determines the target encoding component of the first encoding unit under the second pixel parameter according to the third encoding component, which may include:

[0057] Determine a second parameter value of the first coding unit under the second pixel parameter;

[0058] Determine a target coding mode from candidate coding modes according to a second error between the third coding component and the second parameter value, and use the third coding component corresponding to the target coding mode as the target coding component;

[0059] The candidate coding modes include a first coding mode; when the target coding mode is the first coding mode, the second coding component is used as the third coding component.

[0060] Optionally, the encoding module encodes the first encoding unit according to the first parameter value, which may include:

[0061] Pre-encoding the first parameter value using a candidate encoding mode to obtain a second encoding component; the candidate encoding mode includes the first encoding mode;

[0062] Determine a parameter value of the first coding unit under the second pixel parameter;

[0063] determining a target coding mode from candidate coding modes based on a second error between the second coding component and the second parameter value;

[0064] The first coding unit is encoded according to the second coding component corresponding to the target coding mode.

[0065] Optionally, the encoding module encoding the first encoding unit according to the second encoding component corresponding to the target encoding mode may include:

[0066] When the target coding mode is the first coding mode, the second coding component is used as the third coding component of the first coding unit under the second pixel parameter;

[0067] The first coding unit is encoded according to the first coding component and the third coding component.

[0068] Optionally, after encoding the first encoding unit, the encoding module may also be used to:

[0069] Determine a mode identifier corresponding to a first encoding mode;

[0070] The encoded data of the multimedia file is generated according to the mode identifier and the encoding result of the first encoding unit.

[0071] Optionally, the multimedia file includes a video stream captured from a screen.

[0072] On the other hand, an embodiment of the present application provides a multimedia file decoding device, the device comprising:

[0073] A data acquisition module, used for acquiring encoded data of multimedia files;

[0074] A decoding module, used for decoding the above-mentioned encoded data;

[0075] The above-mentioned coded data is obtained by encoding the multimedia file in the following manner:

[0076] Performing a pre-coding operation on a parameter value of a first coding unit of the multimedia file under a first pixel parameter to obtain a first coding component;

[0077] According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter;

[0078] Obtaining a first parameter value of a reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0079] The first coding unit is encoded according to the first parameter value.

[0080] Optionally, the decoding module may decode the encoded data, which may include:

[0081] Determining a target encoding mode according to a mode identifier in the encoding data;

[0082] When it is determined that the target encoding mode is the first encoding mode, the encoded data is decoded according to a decoding mode corresponding to the first encoding mode.

[0083] Optionally, the decoding module decodes the encoded data according to the decoding mode corresponding to the first encoding mode, which may include:

[0084] Determining a target coded component and a first coded component of a first coded unit in the coded data;

[0085] Pre-decoding the target coding component to obtain a first parameter value of the reference coding unit under the second pixel parameter; and determining a parameter value of the first coding unit under the second pixel parameter according to the first parameter value;

[0086] Pre-decoding the first coding component to obtain a parameter value of the first coding unit under the first pixel parameter;

[0087] The target coding component is the same as the second coding component of the reference coding unit under the second pixel parameter.

[0088] On the other hand, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein the processor and the memory are connected to each other;

[0089] The memory is used to store computer programs;

[0090] The above-mentioned processor is configured to execute the multimedia file encoding method and the multimedia file decoding method provided in the embodiments of the present application when calling the above-mentioned computer program.

[0091] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the multimedia file encoding method and the multimedia file decoding method provided by the embodiment of the present application.

[0092] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program. When the above computer program is executed by a processor, it implements the multimedia file encoding method and the multimedia file decoding method provided by the embodiment of the present application.

[0093] In the multimedia file encoding method provided in the embodiment of the present application, the difference between different encoding units in the multimedia file under the first pixel parameter can be combined, and the reference encoding unit of the first encoding unit can be matched according to the error of the encoding component of different encoding units in the multimedia file under the first pixel parameter, and the error between the reference encoding component and the first encoding component meets the preset requirement, which can be regarded as the similarity between the reference encoding unit and the first encoding unit meets the preset requirement. And further, according to the parameter value of the reference encoding unit under the second pixel parameter (the second pixel parameter and the first pixel parameter are in the same pixel format), the first encoding unit is encoded, and compared with the linear model obtained by reconstructing the pixels around the encoding unit, the encoding prediction method of the encoding unit under different pixel parameters can improve the encoding performance of the encoding unit under the second pixel parameter, thereby improving the encoding performance of the multimedia file as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0095] Figure 1 A schematic diagram of a process flow of a multimedia file encoding method provided in an embodiment of the present application is shown;

[0096] Figure 2 One of the application scenario schematic diagrams of the multimedia file encoding method provided in the embodiment of the present application is shown;

[0097] Figure 3 The second schematic diagram of the application scenario of the multimedia file encoding method provided in the embodiment of the present application is shown;

[0098] Figure 4 The third schematic diagram of the application scenario of the multimedia file encoding method provided in the embodiment of the present application is shown;

[0099] Figure 5a A fourth schematic diagram of an application scenario of the multimedia file encoding method provided in an embodiment of the present application is shown;

[0100] Figure 5b The fifth schematic diagram of the application scenario of the multimedia file encoding method provided in the embodiment of the present application is shown;

[0101] Figure 6 A schematic diagram showing a flow chart of a multimedia file decoding method provided in an embodiment of the present application is shown;

[0102] Figure 7 A schematic diagram showing the structure of a multimedia file processing system provided in an embodiment of the present application is shown;

[0103] Figure 8 One of the application scenario schematic diagrams of the multimedia file processing system provided by the embodiment of the present application is shown;

[0104] Fig. 9 A second schematic diagram of an application scenario of the multimedia file processing system provided in an embodiment of the present application is shown;

[0105] Fig.10 A schematic diagram of the structure of a multimedia file encoding device provided in an embodiment of the present application is shown;

[0106] Fig.11 A schematic diagram of the structure of a multimedia file decoding device provided in an embodiment of the present application is shown;

[0107] Fig.12 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0108] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0109] The embodiment of the present application provides a solution for encoding and decoding multimedia files. By adopting this solution, the encoding performance of the encoding unit under the second pixel parameter can be improved, thereby improving the encoding performance of the multimedia file as a whole.

[0110] Optionally, the data processing processes involved in the various methods provided in the embodiments of the present application can be implemented based on cloud technology. For example, the data calculation involved in the multimedia file encoding and decoding process can be implemented using cloud computing technology, and the data storage involved can be implemented in a cloud storage manner, such as storing multimedia files in a cloud storage manner. In some application scenarios, for example, the multimedia file encoding process or the multimedia file decoding process can be performed by the server of the application, which can be a cloud server.

[0111] Among them, cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called "cloud". From the user's point of view, the resources in the "cloud" can be infinitely expanded, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. Cloud storage is a new concept extended and developed from the concept of cloud computing. Distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide data storage and business access functions to the outside world.

[0112] It should be noted that in the optional embodiments of the present application, the object information and other related data involved, when the embodiments in the present application are applied to specific products or technologies, need to obtain the permission or consent of the object, and the collection, use and processing of the relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions. In other words, if the embodiments of the present application involve data related to the object, it needs to be obtained with the authorization and consent of the object, the authorization and consent of the relevant departments, and in compliance with the relevant laws, regulations and standards of the country and region. If personal information is involved in the embodiments (for example, personal information is involved in multimedia files), the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject needs to be obtained, and the embodiments also need to be implemented with the authorization and consent of the object.

[0113] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0114] Figure 1 A flow chart of a multimedia file encoding method provided by an embodiment of the present application is shown, and the method can be executed by any electronic device, such as a user terminal (also referred to as a terminal, a terminal device, or a user device, etc.) or a server. Among them, the user terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device (such as a smart speaker), a wearable electronic device (such as a smart watch), a vehicle terminal, a smart home appliance (such as a smart TV), an AR / VR device, etc., but is not limited to this. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The user terminal and the server can be directly or indirectly connected by wired or wireless communication, and the embodiments of the present application are not limited here.

[0115] The multimedia file encoding method provided in the embodiment of the present application can be applied to any scenario in which multimedia file encoding is performed, for example, it can be applied to any application or mini-program within an application that performs multimedia file encoding. Specifically, it can be applied to scenarios such as video calls, short videos, video websites, and remote conferences. Taking the application in short videos as an example, the method can be applied to the process of background compression of short videos to improve the compression performance of short videos and save bandwidth.

[0116] like Figure 1 As shown, the multimedia file encoding method provided in the embodiment of the present application may include the following steps:

[0117] Step S110: pre-encoding a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component.

[0118] In the embodiment of the present application, the multimedia file may be generated based on an image, such as an image sequence formed by one image or multiple images, wherein the image sequence may be continuous or discontinuous.

[0119] Among them, a continuous image sequence can form a video stream. The video stream can be a video stream that needs to be encoded in any application scenario, such as short video, film and television video, etc. The video can be a real-time video, such as live video, online conference video, online call video, etc., or it can be a non-real-time video, such as a recorded video, a video on a video website (such as a video in a video database in an application that provides video viewing services), etc. The image can be any frame image in the video stream, and its specific acquisition method can refer to the acquisition method of the video stream, which is not limited here. The discontinuous image sequence can be at least two discontinuous frames of images in the same video stream (for example, the first frame and the third frame image in the video stream), or it can be at least two frames of images in different video streams (for example, the first frame image in the first video stream and the first frame image in the second video stream).

[0120] The embodiment of the present application does not limit the specific determination method of the coding unit (CU), and the technology for determining the coding unit in the prior art can be used. For example, the multimedia file can be divided into different frames to be encoded, and the frames to be encoded can be divided into multiple coding units according to the total number of pixels in the frames to be encoded.

[0121] In the embodiment of the present application, the size granularity of the coding unit is not limited. A coding unit can be an image block of a set size in the frame to be coded, and the size of the coding unit can be preconfigured according to the requirements. Optionally, the frame to be coded can be divided once or multiple times, and each sub-region after the division is used as a coding unit, such as dividing the frame to be coded according to a preset size, or dividing the frame to be coded into n×n (n≥2) sub-regions, and each sub-region after the division is used as a coding unit, or each sub-region is divided again, and each region obtained by the further division is used as a coding unit. For example, the coding unit obtained by the division can be a 4*4 coding unit (that is, the width of the coding unit is the width of 4 pixels, the length is the length of 4 pixels, and the number of pixels included in the coding unit can be 16), an 8*8 coding unit (that is, the width of the coding unit is the width of 8 pixels, the length is the length of 8 pixels, and the number of pixels included in the coding unit can be 64), etc.

[0122] When encoding a multimedia file, each encoding unit can be encoded as the first encoding unit in turn according to a preset order until the encoding process of all encoding units in the multimedia file is completed to obtain the encoded multimedia file (hereinafter referred to as the encoded data of the multimedia file).

[0123] The first pixel parameter may be a pixel parameter in any pixel format (i.e., a color space model, a pixel space). The pixel format may be an RGB (R is red, red; G is green, green; B is blue) pixel format, a YUV pixel format (Y represents brightness (Luminance or Luma), that is, a grayscale value; U and V represent chrominance (Chrominance or Chroma), that is, color and saturation), a CMYK (C is Cyan, cyan; M is Magenta, also known as magenta; Y is Yellow, yellow; K is black) pixel format, etc. Different pixel formats may be converted to each other. Taking the YUV pixel format as an example, the first pixel parameter may be a Y pixel parameter (i.e., a brightness pixel parameter), a U pixel parameter, or a V pixel parameter. In the embodiment of the present application, the YUV pixel format, the first pixel parameter is a Y pixel parameter, and the second pixel parameter is a U pixel parameter or a V pixel parameter as an example for explanation.

[0124] The parameter value of the first coding unit under the first pixel parameter can be determined according to the parameter value of each pixel in the first coding unit under the first pixel parameter. For example, the average value, median or mode of the parameter value of each pixel in the first coding unit under the first pixel parameter can be used as the parameter value of the first coding unit under the first pixel parameter.

[0125] When pre-encoding the parameter value of the first coding unit under the first pixel parameter, any coding mode can be used for coding, and the embodiment of the present application does not impose specific restrictions on this. Among them, the coding method adopted can be, for example, the coding mode involved in the H.26x series, MPEG (Moving Pictures Experts Group) series, AVS (Audio Video coding Standard, source coding standard), RMVB (RM is Real Media, VB refers to Variable Bit Rate), VVC standard (Versatile Video Coding) and CCLM prediction mode (Cross Component Linear Model, cross-component linear model) and other standards.

[0126] Step S120: According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under a first pixel parameter.

[0127] It can be understood that the reference coding component is determined in the same manner as the first coding component.

[0128] In the embodiment of the present application, the matching area of ​​the reference coding unit can be matched for the first coding unit according to the actual situation, and no specific limitation is made here. For example, the matching area can be set to all coding units in the multimedia file. The matching area can also be set to the image frame where the first coding unit is located in the multimedia file. The matching area can also be set to a set area in the image frame where the first coding unit is located in the multimedia file.

[0129] The error between the reference coding component and the first coding component includes but is not limited to the absolute error sum, error square sum, mean square error, root mean square error, mean absolute error, etc. between the reference coding component and the first coding component. The specific error determination method can be determined based on actual conditions.

[0130] The preset requirements may include but are not limited to the error between the reference coding component and the first coding component being less than or equal to a preset threshold, the error between the reference coding component and the first coding component being less than the error between the first coding component and the coding components of all coding units within the matching range under the first pixel parameters, etc. The preset requirements may be set according to actual conditions.

[0131] In actual application, the error between the first coding component and the parameter value of the coding unit in the matching area under the first pixel parameter can be determined, and the coding unit whose error meets the coding requirement is used as the reference coding unit. Figure 2 As shown, taking the first pixel parameter as the Y pixel parameter as an example, the first coding unit 21 for determining the first coding component is called the coding unit Luma', and the set area 22 in the image frame 20 where the first coding unit is located in the multimedia file is used as the matching area. The reference coding unit 23, that is, the coding unit P, can be obtained by matching in the matching area.

[0132] Step S130: Obtain a first parameter value of the reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format.

[0133] It is understood that in the embodiment of the present application, the number of the first pixel parameter and the second pixel parameter is a general reference, and the number of the first pixel parameter and the number of the second pixel parameter are not specifically limited, and the number of the first pixel parameter and the number of the second pixel parameter can be determined according to actual conditions. For example, taking the pixel format as the YUV format as an example, if the first pixel parameter is the Y pixel parameter, the second pixel parameter can be the U pixel parameter and the V pixel parameter, that is, the number of the first pixel parameter is 1, and the number of the second pixel parameter is 2.

[0134] Step S140: Encode the first encoding unit according to the first parameter value.

[0135] In this implementation, a specific encoding method for the first encoding unit can be determined according to actual conditions. For example, the first parameter value can be encoded according to the above-mentioned first encoding method, and the encoded component obtained after encoding is used as the encoding component of the first encoding unit under the second pixel parameter, and the encoding result of the first encoding unit is obtained according to the obtained encoding component of the first encoding unit under the second pixel parameter and the first encoding component.

[0136] It should be noted that before matching the reference coding unit for the first coding unit in the multimedia file, the difference between the parameter values ​​of each pixel in the first coding unit under the first pixel parameter and the difference between the parameter values ​​of each pixel in the reference matching unit under the first pixel parameter can also be determined. The smaller the difference between the parameter values ​​of each pixel in the first coding unit under the first pixel parameter, the more accurate the reference matching unit matched according to the method, and thus the more accurate the result of encoding the first coding unit based on the first parameter value of the reference coding unit under the second pixel parameter.

[0137] In the multimedia file encoding method provided in the embodiment of the present application, by matching the reference encoding unit of the first encoding unit according to the error of the encoding component of different encoding units in the multimedia file under the first pixel parameter, and the error between the reference encoding component and the first encoding component meets the preset requirement, it can be regarded that the similarity between the reference encoding unit and the first encoding unit meets the preset requirement. And further, according to the parameter value of the reference encoding unit under the second pixel parameter (the second pixel parameter and the first pixel parameter are in the same pixel format), the first encoding unit is encoded, and the encoding unit in the multimedia file can be encoded according to the parameter value of the matched reference encoding unit under the second pixel parameter in combination with the difference of different encoding units in the multimedia file under the first pixel parameter. Compared with the linear model obtained according to the surrounding reconstructed pixels of the encoding unit, the encoding prediction method of the encoding unit under different pixel parameters can improve the encoding performance of the encoding unit under the second pixel parameter, thereby improving the encoding performance of the multimedia file as a whole.

[0138] Optionally, matching the reference coding unit for the first coding unit in the multimedia file may include:

[0139] Determining position information of the first encoding unit in the multimedia file;

[0140] Using the above position information as a matching center, determining a matching area in the multimedia file;

[0141] A reference coding unit is matched for the first coding unit in the matching area.

[0142] The position information of the first coding unit in the multimedia file may include but is not limited to the frame to be encoded in the multimedia file where the first coding unit is located and the position parameters of the first coding unit in the frame to be encoded, etc. The position parameters of the first coding unit in the frame to be encoded may include but are not limited to the width and height of the first coding unit, the position parameters of each pixel of the first coding unit in the frame to be encoded, the central pixel of the first coding unit, etc. The central pixel of the first coding unit may be determined according to the position parameters of each pixel in the first coding unit in the frame to be encoded.

[0143] like Figure 3 As shown, in the frame to be encoded 30, the lower left corner of the frame to be encoded 30 can be taken as the origin O (0, 0), the width direction is the x-axis, the direction from the origin and parallel to the x-axis is the positive direction of the x-axis, the height direction is the y-axis, the direction from the origin and parallel to the y-axis is the positive direction of the y-axis, the width of each pixel is the unit length of the x-axis, and the height of each pixel is the unit length of the y-axis, and the position parameters of each pixel in the frame to be encoded 30 are determined. Then, according to the position parameters of each pixel in the first encoding unit in the frame to be encoded 30, the position parameters of the central pixel of the first encoding unit are determined. Figure 3 As shown, the first coding unit 31 can be represented by four points x1 (a, b), x2 (a+w, b), y1 (a, b+h), and y2 (a+w, b+h). The width of the first coding unit is w, that is, the width of the first coding unit 31 is w pixels, and the height is h, that is, the height of the first coding unit 31 is h pixels. The pixel position parameters of the central pixel O' are (a+w / 2, b+h / 2).

[0144] When matching a reference coding unit for a first coding unit in a matching region, the first coding unit may be matched with each coding unit in the matching region in turn, and a coding unit that meets a preset requirement in the matching region may be determined as a reference coding unit.

[0145] In an embodiment of the present application, a matching area is determined in the multimedia file based on the position information of the first coding unit in the multimedia file, and a reference coding unit is matched for the first coding unit in the coding unit within the matching area. Compared with matching the reference coding unit for the first coding unit in the coding unit within the multimedia file, the efficiency of determining the reference coding unit can be improved, thereby improving the encoding efficiency of the multimedia file and reducing the amount of data processing required for encoding the multimedia file.

[0146] Optionally, the above-mentioned position information may include pixel position parameters, width and height of the central pixel of the first coding unit;

[0147] The determining of the matching area in the multimedia file may include at least one of the following:

[0148] In the multimedia file, the target image frame including the first encoding unit is used as a matching area;

[0149] According to the width, the width threshold, the height and the height threshold, a matching area centered around the pixel position parameter is determined in the target image frame.

[0150] In the embodiment of the present application, the width threshold and the height threshold can be empirical values ​​or experimental values. The specific value selection method can be determined according to the actual situation and is not specifically limited here. Among them, the width threshold and the height threshold can be the same or different, that is, the setting method of the width threshold and the height threshold is independent of each other and can be determined according to the actual situation. For example, the width threshold and the height threshold can be set to s, that is, the width of the matching area is: Search Range Width = s*W, and the height is: Search Range Height = s*H. Figure 4 As shown, in the frame to be encoded 40, the pixel position parameter O" of the first encoding unit 41 can be taken as the center, s=5 can be set, and a matching area 42 with a width of 5w and a height of 5h can be obtained.

[0151] It can be understood that the central pixel of the matching area centered on the pixel position parameter is the same as the central pixel of the first encoding unit.

[0152] In an embodiment of the present application, by determining a target image frame including a first coding unit in a multimedia file or a set area in the target image frame as a matching area, and matching a reference coding unit for the first coding unit within the matching area, the differences between the image frames in the multimedia file can be combined to accurately match the reference coding unit for the first coding unit within the target image frame to which the first coding unit belongs.

[0153] Optionally, matching the reference coding unit for the first coding unit in the multimedia file may include:

[0154] Determine a first error between the first coding component and the candidate coding component; wherein the candidate coding component is obtained by encoding a parameter value of a candidate coding unit in the matching area under the first pixel parameter;

[0155] A reference coding unit is determined from candidate coding units according to the first error.

[0156] The first error may include but is not limited to the sum of absolute errors, the sum of squared errors, the mean square error, the root mean square error, the mean absolute error, etc. The method for determining the first error may be determined according to actual conditions.

[0157] In the embodiment of the present application, a reference coding unit is matched for a first coding unit in a matching area through a first error. The matching area may also be referred to as a search area, and the first error may be referred to as a search error.

[0158] Taking the first error as the Sum of Absolute Differences (SAD) and the first coding component as the coding component under the Y pixel parameter as an example, the first error can be determined by:

[0159]

[0160] Wherein, loss1(i) represents the first error, w represents the width of the first coding unit, h represents the height of the first coding unit, Luma ′ (x, y) represents the first coded component, S i (x,y) represents the candidate coded component.

[0161] Optionally, the determining of the reference coding unit from the candidate coding units according to the first error may include at least one of the following:

[0162] Determine the candidate coding unit with the smallest first error as a reference coding unit;

[0163] A candidate coding unit whose first error is less than or equal to a preset threshold is determined as a reference coding unit.

[0164] In this implementation, after determining the first errors corresponding to all candidate coding units in the matching area, the candidate coding unit corresponding to the minimum value among the first errors may be determined as the reference matching unit. Alternatively, the first error corresponding to each candidate coding unit may be compared with a preset threshold, and the candidate coding unit that is less than or equal to the preset threshold may be determined as the reference coding unit.

[0165] It is understandable that the above method of determining the reference coding unit according to the first error is only an example, and each condition can be regarded as the above preset requirement. Of course, the reference coding unit can also be determined according to other methods.

[0166] If the number of candidate coding units whose first errors are less than or equal to the preset threshold is greater than 1, the candidate coding unit with the smallest first error among the candidate coding units whose first errors are less than or equal to the preset threshold may be used as a reference coding unit, or a coding unit may be randomly selected from the candidate coding units whose first errors are less than or equal to the preset threshold as a reference coding unit, etc. The specific details may be determined according to actual conditions.

[0167] In an embodiment of the present application, by determining a reference coding unit from candidate coding units based on a first coding component of a first coding unit and a first error of a candidate coding component of a candidate coding unit, the error between the determined reference coding unit and the first coding unit can be effectively reduced, and the similarity between the determined reference coding unit and the first coding unit can be improved.

[0168] Optionally, encoding the first encoding unit according to the first parameter value may include:

[0169] Pre-coding the first parameter value to obtain a second coded component;

[0170] Using the candidate coding mode, and predicting a third coding component of the first coding unit under the second pixel parameter according to the second coding component;

[0171] Determine, according to the third coding component, a target coding component of the first coding unit under the second pixel parameter;

[0172] A first coding unit is encoded according to the first coding component and the target coding component.

[0173] The manner of pre-encoding the first parameter value may be the same as the manner of pre-encoding the parameter value of the first encoding unit under the first pixel parameter.

[0174] The candidate coding modes may include, but are not limited to, a prediction mode based on directional interpolation in the VCC standard (e.g., Direction Interpolation Prediction Variable Length Coding, DIPVLC for short), a CCLM prediction mode, and a first coding mode, etc. Among them, the first coding mode is to use the second coding component as the coding component of the first coding unit under the second pixel parameter.

[0175] In the embodiments of the present application, unless otherwise specified, the candidate coding mode can be regarded as a coding mode that predicts a coding component under another pixel parameter based on a coding component under a certain pixel parameter.

[0176] When the candidate coding mode is adopted to predict the third coding component, the coding basis may be determined according to the specific candidate coding mode.

[0177] As an example, when the third coding component is predicted by using CCLM prediction, the third coding component can be determined based on the first coding component and the linear transformation function. The first coding component can be obtained by downsampling the parameter values ​​of the surrounding pixels of the first coding unit under the first pixel parameter, and the linear transformation function can be pred_(i, j) = · rec_L′ (i,j)+. Where pred_(i,j) represents the third coded component, rec_L ′ (i, j) represents the first coding component. Parameters α and β can be calculated according to actual conditions and are generally constants.

[0178] Specifically, the parameters α and β can be determined in the following manner. Figure 5a As shown, the size of the first coding unit m is N*N, the surrounding pixels of the first coding unit m can be the pixels above and to the left of the first coding unit m, and rec_C(i) represents the pixel value of the surrounding pixel i of the first coding unit m under the first pixel parameter. Figure 5b As shown, rec_L ′ (m) represents the pixel value obtained by downsampling the pixel value of the surrounding pixel i of the first coding unit m under the second pixel parameter. Correspondingly, the size of the first coding unit m after downsampling is 2N*2N. If the first pixel parameter is a brightness parameter (i.e., a Y pixel parameter) and the second pixel parameter is a chrominance parameter (i.e., a U pixel parameter and a V pixel parameter), then Figure 5a The coding unit shown is called a chrominance block Chroma. Figure 5b The coding unit shown is called the luminance block Luma. It can be obtained by least square method according to rec_C(m) and rec_L ′ (m) Get the parameters α and β:

[0179]

[0180] In an embodiment of the present application, by combining different candidate coding modes, the third coding component of the first coding unit under the second pixel parameter is predicted, and then the target coding component is obtained according to each third coding component. According to the diversity of candidate coding modes, the target coding component that meets the actual needs can be obtained, thereby paving the way for improving the coding accuracy.

[0181] Optionally, the determining, according to the third coding component, a target coding component of the first coding unit under the second pixel parameter includes:

[0182] Determine a second parameter value of the first coding unit under the second pixel parameter;

[0183] Determine a target coding mode from candidate coding modes according to a second error between the third coding component and the second parameter value, and use the third coding component corresponding to the target coding mode as the target coding component;

[0184] The candidate coding modes include a first coding mode; when the target coding mode is the first coding mode, the second coding component is used as the third coding component.

[0185] Taking the first pixel parameter as the Y pixel parameter and the second pixel parameter as the U pixel parameter and the V pixel parameter as an example, the pixel value of the reference coding unit S_luma under the second pixel parameter is encoded, and the obtained second coding components are S_U and S_V respectively. Then, the third coding components of the first coding unit under the second pixel parameter can be obtained as follows: pred_U(x, y)=S_U(x, y) under the U pixel parameter, and pred_V(x, y)=S_V(x, y) under the V pixel parameter.

[0186] The second error may include but is not limited to the sum of absolute errors, the sum of squared errors, the mean square error, the root mean square error, the mean absolute error, etc. The method for determining the second error may be determined according to actual conditions.

[0187] Taking the second error as the sum of squared errors (SSE), the second parameter value as orig(,y), and the third coding component as pred(x,y) as an example, the second error dist can be determined as follows:

[0188]

[0189] By determining the second error between the third coding component corresponding to each candidate coding mode and the second parameter value, the coding distortion magnitude when encoding is performed using each candidate coding mode can be determined.

[0190] Optionally, when determining the target coding mode from the candidate coding modes according to the second error between the third coding component and the second parameter value, the smallest second error may be first determined, and the candidate coding mode corresponding to the second error may be determined as the target coding mode. Correspondingly, a preset threshold may also be set, and after determining that the second error is smaller than the preset threshold, the candidate coding mode corresponding to the determined second error may be determined as the target coding mode.

[0191] In an embodiment of the present application, by comparing the coding distortions corresponding to different candidate coding modes, a target coding mode is determined, and the third coding component corresponding to the target coding mode is used as the target coding component. This allows the coding distortion to be minimized when the first coding unit is encoded according to the determined target coding component, thereby effectively improving the coding performance of multimedia files and avoiding the loss of information in multimedia files due to coding operations.

[0192] Taking into account the limitations of the information of the multimedia file itself reflected by the chromaticity information, when encoding the multimedia file using the YUV chromaticity format, it is preferred to use the Y pixel parameter as the first pixel parameter and the U pixel parameter and the V pixel parameter as the second pixel parameter to encode the multimedia file.

[0193] Optionally, after encoding the first encoding unit, the method may further include:

[0194] Determine a mode identifier corresponding to a first encoding mode;

[0195] The encoded data of the multimedia file is generated according to the mode identifier and the encoding result of the first encoding unit.

[0196] Considering that in the related art, mode identifiers corresponding to different candidate encoding modes have been set, when encoding is performed using the encoding mode provided in the embodiment of the present application (i.e., the first encoding mode), it is necessary to set a mode identifier corresponding to the first encoding mode to distinguish it from the candidate encoding modes in the related art, so that the decoding end can parse the encoding data of the received multimedia file and obtain the decoding mode corresponding to the encoding mode, thereby realizing the decoding operation of the encoded data.

[0197] As an example, take the first pixel parameter as the Y pixel parameter, and the second pixel parameter as the U pixel parameter and the V pixel parameter. In order to better transmit the adopted chroma coding mode in the encoded data, after setting the mode identifiers of each chroma prediction mode in VVC, the mode identifiers of each chroma prediction mode can also be represented by binary. As shown in Table 1, the mode identifiers of each chroma prediction mode in VVC and the corresponding binary representation are shown. Among them, mode identifiers 5, 6, and 7 represent LM, LM-L, and LM-A in CCLM mode, respectively, and the remaining modes are traditional chroma prediction modes.

[0198] Chroma prediction mode flag Binary Representation 4 00 0 0100 1 0101 2 0110 3 0111 5 10 6 110 7 111

[0199] Assuming that the mode identifier of the first coding mode is set to 8, the above Table 1 can be modified to obtain Table 2:

[0200]

[0201]

[0202] As shown in Table 2, when Table 1 is revised, the binary representation corresponding to mode identifier 7 is also modified so that the decoding end can determine whether the final encoding mode adopted is the encoding mode corresponding to mode identifier 7 or the encoding mode corresponding to mode identifier 8 based on the fourth bit value "k" in "111k", thereby accurately implementing the decoding operation of the encoded data.

[0203] Optionally, the multimedia file includes a video stream captured from a screen.

[0204] The video stream captured from the screen can be a video stream during instant messaging (such as a video stream generated by real-time communication in an instant messaging application) or a recorded video stream (such as a video stream in a video playback application), and no specific limitation is made here.

[0205] By encoding the multimedia file encoding method provided in the embodiment of the present application, the video stream captured from the screen is encoded, which can improve the encoding performance.

[0206] Based on the same principle as the multimedia file encoding method provided in the embodiment of the present application, a multimedia file decoding method is also provided in the embodiment of the present application. The method can be executed by any electronic device, such as a user terminal (also referred to as a terminal, a terminal device, or a user device, etc.) or a server. The specific implementation form or connection mode of the user terminal and the server can refer to the description in the above-mentioned multimedia file encoding method, which will not be repeated here.

[0207] The multimedia file decoding method provided in the embodiment of the present application can be applied to any scenario for multimedia file decoding. For example, the multimedia file decoding scenario can correspond to the above-mentioned multimedia file encoding scenario, and can be applied to any application for multimedia file decoding or a mini-program within the application. Specifically, it can be applied to scenarios such as video calls, short videos, video websites, and remote conferences. Taking the application in short videos as an example, the method can be applied to the process of playing short videos. After obtaining the compressed short video, the compressed short video is decoded, and then the decoded short video is played, wherein the compressed short video can be encoded based on the above-mentioned multimedia file encoding method, thereby improving the transmission bandwidth for obtaining the compressed short video.

[0208] like Figure 6 As shown, the multimedia file decoding method includes the following steps:

[0209] Step S610: Obtaining encoded data of a multimedia file;

[0210] Step S620: decoding the encoded data;

[0211] The encoded data is obtained by encoding the multimedia file in the following manner:

[0212] Performing an encoding operation on a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component;

[0213] According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter;

[0214] Obtaining a first parameter value of a reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0215] The first coding unit is encoded according to the first parameter value.

[0216] In the embodiment of the present application, there is no restriction on the method of obtaining the coded data. For example, it can be obtained locally or downloaded in real time from the network. For example, during a video call, one video call terminal can obtain the coded data corresponding to the video stream of another video call terminal.

[0217] In the multimedia file decoding method provided in the embodiment of the present application, since in the encoded data, the difference between the first encoding components of different encoding units in the multimedia file and the first encoding unit under the first pixel parameter is combined, and the first encoding unit in the multimedia file is encoded according to the parameter value of the reference encoding unit matching the first encoding unit under the second pixel parameter, compared with the linear model obtained according to the reconstructed pixels around the encoding unit, the encoding prediction method of the encoding unit under different pixel parameters improves the encoding performance of the encoding unit under the second pixel parameter, thereby improving the encoding performance of the multimedia file as a whole, so that a multimedia file with less distortion than the original multimedia file can be decoded at the decoding end.

[0218] Optionally, the decoding of the encoded data may include:

[0219] Determining a target encoding mode according to a mode identifier in the encoding data;

[0220] When it is determined that the target encoding mode is the first encoding mode, the encoded data is decoded according to a decoding mode corresponding to the first encoding mode.

[0221] In this implementation, the mode identifier in the encoded data can refer to the above description of the mode identifier, such as the contents shown in Table 1 and Table 2, and will not be repeated here.

[0222] The decoding mode corresponding to the first encoding mode is a decoding mode symmetrical to the first encoding mode. That is, the target encoding component of the first encoding unit is: the second encoding component obtained by encoding the parameter value of the reference encoding unit under the second pixel parameter. Accordingly, the parameter value obtained by decoding the second encoding component can be used as the parameter value of the first encoding unit under the second pixel parameter.

[0223] It can be understood that the above is only an example, and when the target coding mode is different from the first coding mode, the encoded data can be decoded using a decoding mode that is symmetrical to the target coding mode.

[0224] In an embodiment of the present application, a target encoding mode is determined according to a mode identifier in the encoded data, and the encoded data is encoded based on a decoding mode corresponding to the target encoding mode.

[0225] Optionally, the decoding of the encoded data according to the decoding mode corresponding to the first encoding mode may include:

[0226] Determining a target coded component and a first coded component of a first coded unit in the coded data;

[0227] Pre-decoding the target coding component to obtain a first parameter value of the reference coding unit under the second pixel parameter; and determining a parameter value of the first coding unit under the second pixel parameter according to the first parameter value;

[0228] Pre-decoding the first coding component to obtain a parameter value of the first coding unit under the first pixel parameter;

[0229] The target coding component is the same as the second coding component of the reference coding unit under the second pixel parameter.

[0230] In this implementation, the method of pre-decoding the target coding component and the first coding component may be symmetrical to the above-mentioned pre-coding method, which will not be described in detail herein.

[0231] In the embodiment of the present application, by decoding the target coding component and the first coding component of the first coding unit in the coded data, the parameter value of the first coding unit under each pixel parameter can be obtained, thereby realizing the decoding of the multimedia file.

[0232] The encoding process of the encoded data in the multimedia file decoding method of the embodiment of the present application is similar to the implementation principle of the aforementioned multimedia file encoding method. For a detailed description of the encoding process of the encoded data, please refer to the description in the corresponding multimedia file encoding method shown in the previous text, which will not be repeated here.

[0233] Based on the same principle as the multimedia file encoding method and multimedia file decoding method provided in the embodiment of the present application, a multimedia file processing system is also provided in the embodiment of the present application. The multimedia file processing system can be any electronic device, such as a user terminal (also referred to as a terminal, terminal device or user equipment, etc.) or a server. The specific implementation form or connection mode of the user terminal and the server can refer to the description in the above-mentioned multimedia file encoding method, which will not be repeated here.

[0234] The multimedia file processing system provided in the embodiment of the present application can be applied to any scenario for multimedia file processing. For example, the multimedia file processing scenario can include the above-mentioned multimedia file encoding scenario and the above-mentioned multimedia file encoding scenario corresponding to the above-mentioned multimedia file encoding scenario, and can be applied to any application program or mini-program within the application program for multimedia file encoding and decoding. Specifically, it can be applied to scenarios such as video calls, short videos, video websites, and remote conferences.

[0235] Taking the application in video calling as an example, the multimedia file processing system can be at least two terminals for video calling. Assuming that the terminals currently making the video call are terminal A and terminal B, the real-time video stream captured from the screen of terminal A can be encoded by terminal A, and the encoded data can be sent to terminal B, and the acquired data can be decoded at terminal B, and the real-time video stream captured from the screen of terminal A can be played at terminal B. Similarly, the real-time video stream captured from the screen of terminal B can be encoded by terminal B, and the encoded data can be sent to terminal A, and the acquired data can be decoded at terminal A, and the real-time video stream captured from the screen of terminal B can be played at terminal A, so as to realize the video calling, improve the encoding performance during the video calling process, reduce the transmission bandwidth for transmitting the compressed video stream, and improve the quality of the video calling.

[0236] like Figure 7 As shown, the multimedia file processing system 70 includes:

[0237] The encoding module 71 is used to encode the first multimedia file;

[0238] A decoding module 72, used to decode the acquired coded data;

[0239] The encoding module encodes the first multimedia file according to the following method:

[0240] Performing a pre-coding operation on a parameter value of a first coding unit of a first multimedia file under a first pixel parameter to obtain a first coding component;

[0241] According to the first coding component, a first reference coding unit is matched for the first coding unit in the first multimedia file; wherein an error between the first reference coding component of the first reference coding unit and the first coding component meets a preset requirement, and the first reference coding component is obtained by encoding a parameter value of the first reference coding unit under a first pixel parameter;

[0242] Obtaining a first parameter value of a first reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0243] Encoding the first encoding unit according to the first parameter value;

[0244] The encoded data is obtained by encoding the second multimedia file in the following manner:

[0245] Performing a pre-coding operation on the parameter value of the second coding unit of the second multimedia file under the first pixel parameter to obtain a fourth coding component;

[0246] According to the fourth coding component, matching a second reference coding unit for the second coding unit in the second multimedia file; wherein an error between the second reference coding component of the second reference coding unit and the fourth coding component meets a preset requirement, and the second reference coding component is obtained by encoding the parameter value of the second reference coding unit under the first pixel parameter;

[0247] Obtain a third parameter value of the second reference coding unit under the second pixel parameter;

[0248] The fourth coding unit is encoded according to the third parameter value.

[0249] In the multimedia file processing system of the embodiment of the present application, the operations performed by the encoding module can refer to the description in the aforementioned multimedia file encoding method, and the operations performed by the decoding module can refer to the description in the aforementioned multimedia file decoding method, which will not be repeated here.

[0250] In the multimedia file processing system provided in the embodiment of the present application, by combining the difference between different coding units in the first multimedia file and the first coding component of the first coding unit under the first pixel parameter, the first coding unit in the first multimedia file is encoded according to the parameter value of the reference coding unit matching the first coding unit under the second pixel parameter, and the second multimedia file is decoded using a decoding method symmetrical to the encoding method. Compared with the method of predicting and decoding the coding unit under different pixel parameters based on the linear model obtained by reconstructing the pixels around the coding unit, the encoding performance of the coding unit under the second pixel parameter is improved, thereby improving the encoding performance of the multimedia file as a whole, obtaining an encoded multimedia file with low distortion, and decoding a multimedia file with less distortion than the original multimedia file through a decoding module, thereby improving the processing performance of the multimedia file.

[0251] In order to more clearly illustrate the multimedia file encoding method, multimedia file decoding method, and multimedia file processing system provided in the embodiments of the present application, the following takes the multimedia file as a video as an example and describes it in combination with a specific application scenario:

[0252] Figure 8 A schematic diagram of the system structure of an online video viewing application scenario applicable to an embodiment of the present application is shown. In this scenario, the system architecture may include a video server 10 and a request terminal connected to the video server 10 for communication, such as Figure 8 Terminal 1, ..., terminal N in, wherein the user terminal can be any form of terminal that can watch videos, such as a user's mobile phone, computer, etc., and the video server 10 can be any server of an application that can provide video viewing services to users. The user can watch various types of videos such as movies and TV series provided by the server through the application running on the requesting terminal. The multimedia file encoding method provided in the embodiment of the present application can be deployed in the video server 10. When the video server 10 receives a viewing request for any video sent by the requesting terminal (such as the user clicks on the application interface of its terminal to watch a certain movie), the application server 10 can obtain the target video corresponding to the request from the video database, and can use the multimedia file encoding method provided in the embodiment of the present application to encode the first encoding unit in the video frame of the target video, and send the encoded code stream to the corresponding requesting terminal. After receiving the code stream sent by the application server, the requesting terminal can execute the multimedia file decoding method provided in the embodiment of the present application, and obtain the decoding result of the unit to be decoded in the video frame of the target video by continuously decoding the received code stream, thereby obtaining the reconstructed video frame and playing it to the user.

[0253] Fig. 9A schematic diagram of the system structure of a video call application scenario applicable to an embodiment of the present application is shown. Fig. 9 As shown, the application server 20 is a server of an application program that provides a video call function. The application program can be run on the first terminal 121 of the object 1 and the second terminal 122 of the object 2. The object 1 and the object 2 can make a video call through the video call service provided by the application server 20. In this application scenario, the multimedia file encoding method and the multimedia file decoding method provided by the embodiment of the present application can be deployed on both the first terminal 121 and the second terminal 122. That is, the first terminal 121 and the second terminal 122 can be regarded as the multimedia file processing system provided by the embodiment of the present application. After the first terminal 121 and the second terminal 122 establish a video call through the application server 20, the first terminal 121 collects the video on the side of the object 1 in real time (that is, the video stream captured from the screen of the first terminal 121), and can use the multimedia file encoding method provided by the embodiment of the present application to encode the video frames in the collected video, and send the code stream generated by the encoding to the second terminal 121 through the application server 20. The second terminal 122 uses the multimedia file decoding method provided by the embodiment of the present application to decode the code stream sent by the first terminal 121 to obtain the video screen on the side of the object 1, and display it to the object 2. Similarly, the second terminal 122 adopts the multimedia file encoding method provided in the embodiment of the present application to encode the video stream on the side of object 2 collected by it, and then sends it to the first terminal 121 through the application server 20. The first terminal 121 adopts the multimedia file decoding method provided in the embodiment of the present application to decode the code stream and present the video screen on the side of object 2 to object 1.

[0254] Based on the same principle as the multimedia file encoding method provided in the embodiment of the present application, the embodiment of the present application also provides a multimedia file encoding device. Fig.10 As shown, the multimedia file encoding device 100 includes:

[0255] The encoding module 101 is used to perform a pre-encoding operation on a parameter value of a first encoding unit of a multimedia file under a first pixel parameter to obtain a first encoding component;

[0256] The coding unit matching module 102 is used to match a reference matching unit for the first coding unit in the multimedia file according to the first coding component; wherein the error between the reference coding component of the reference matching unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding the parameter value of the reference coding unit under the first pixel parameter;

[0257] The parameter value acquisition module 103 is used to acquire a first parameter value of the reference coding unit under the second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0258] The encoding module 101 is further configured to encode the first encoding unit according to the first parameter value.

[0259] Optionally, the coding unit matching module 102 matches the reference coding unit for the first coding unit in the multimedia file, which may include:

[0260] Determining position information of the first encoding unit in the multimedia file;

[0261] Using the above position information as a matching center, determining a matching area in the multimedia file;

[0262] A reference coding unit is matched for the first coding unit in the matching area.

[0263] Optionally, the above-mentioned position information may include pixel position parameters, width and height of the central pixel of the first coding unit;

[0264] The coding unit matching module 102 determines the matching area in the multimedia file, which may include at least one of the following:

[0265] In the multimedia file, the target image frame including the first encoding unit is used as a matching area;

[0266] According to the width, the width threshold, the height and the height threshold, a matching area centered around the pixel position parameter is determined in the target image frame.

[0267] Optionally, the coding unit matching module 102 matches the reference coding unit for the first coding unit in the multimedia file, which may include:

[0268] Determine a first error between a first coding component and a candidate coding component; wherein the candidate coding component is obtained by encoding a parameter value of a candidate coding unit under a first pixel parameter;

[0269] A reference coding unit is determined from candidate coding units according to the first error.

[0270] Optionally, the coding unit matching module 102 may determine the reference coding unit from the candidate coding units according to the first error, which may include at least one of the following:

[0271] Determine the candidate coding unit with the smallest first error as a reference coding unit;

[0272] A candidate coding unit whose first error is less than or equal to a preset threshold is determined as a reference coding unit.

[0273] Optionally, the encoding module 101 may encode the first encoding unit according to the first parameter value, including:

[0274] Pre-coding the first parameter value to obtain a second coded component;

[0275] Using the candidate coding mode, and predicting a third coding component of the first coding unit under the second pixel parameter according to the second coding component;

[0276] Determine, according to the third coding component, a target coding component of the first coding unit under the second pixel parameter;

[0277] A first coding unit is encoded according to the first coding component and the target coding component.

[0278] Optionally, the encoding module 101 determines the target encoding component of the first encoding unit under the second pixel parameter according to the third encoding component, which may include:

[0279] Determine a second parameter value of the first coding unit under the second pixel parameter;

[0280] Determine a target coding mode from candidate coding modes according to a second error between the third coding component and the second parameter value, and use the third coding component corresponding to the target coding mode as the target coding component;

[0281] The candidate coding modes include a first coding mode; when the target coding mode is the first coding mode, the second coding component is used as the third coding component.

[0282] Optionally, after encoding the first encoding unit, the encoding module 101 may also be used to:

[0283] Determine a mode identifier corresponding to a first encoding mode;

[0284] The encoded data of the multimedia file is generated according to the mode identifier and the encoding result of the first encoding unit.

[0285] Optionally, the multimedia file includes a video stream captured from a screen.

[0286] In the multimedia file encoding device provided in the embodiment of the present application, a reference encoding unit of the first encoding unit is obtained by matching according to the error of the encoding components of different encoding units in the multimedia file under the first pixel parameter, and the error between the reference encoding component and the first encoding component meets the preset requirement, so that the similarity between the reference encoding unit and the first encoding unit can be regarded as meeting the preset requirement. And further, according to the parameter value of the reference encoding unit under the second pixel parameter (the second pixel parameter and the first pixel parameter are in the same pixel format), the first encoding unit is encoded, and the encoding unit in the multimedia file can be encoded according to the parameter value of the matched reference encoding unit under the second pixel parameter in combination with the difference of different encoding units in the multimedia file under the first pixel parameter. Compared with the linear model obtained according to the surrounding reconstructed pixels of the encoding unit, the encoding prediction method of the encoding unit under different pixel parameters can improve the encoding performance of the encoding unit under the second pixel parameter, thereby improving the encoding performance of the multimedia file as a whole.

[0287] The multimedia file encoding device of the embodiment of the present application can execute the multimedia file encoding method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the multimedia file encoding device of each embodiment of the present application correspond to the steps in the multimedia file encoding method of each embodiment of the present application. For the detailed functional description of each module of the multimedia file encoding device, please refer to the description in the corresponding multimedia file encoding method shown in the previous text, which will not be repeated here.

[0288] Based on the same principle as the multimedia file decoding method provided in the embodiment of the present application, the embodiment of the present application also provides a multimedia file decoding device. Fig.11 As shown, the multimedia file decoding device 110 includes:

[0289] The data acquisition module 111 is used to acquire the encoded data of the multimedia file;

[0290] A decoding module 112, used to decode the above-mentioned encoded data;

[0291] The above-mentioned coded data is obtained by encoding the multimedia file in the following manner:

[0292] Performing a pre-coding operation on a parameter value of a first coding unit of the multimedia file under a first pixel parameter to obtain a first coding component;

[0293] According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter;

[0294] Obtaining a first parameter value of a reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format;

[0295] The first coding unit is encoded according to the first parameter value.

[0296] Optionally, the decoding module 112 may decode the encoded data, which may include:

[0297] Determining a target encoding mode according to a mode identifier in the encoding data;

[0298] When it is determined that the target encoding mode is the first encoding mode, the encoded data is decoded according to a decoding mode corresponding to the first encoding mode.

[0299] Optionally, the decoding module 112 decodes the encoded data according to the decoding mode corresponding to the first encoding mode, which may include:

[0300] Determining a target coded component and a first coded component of a first coded unit in the coded data;

[0301] Pre-decoding the target coding component to obtain a first parameter value of the reference coding unit under the second pixel parameter; and determining a parameter value of the first coding unit under the second pixel parameter according to the first parameter value;

[0302] Pre-decoding the first coding component to obtain a parameter value of the first coding unit under the first pixel parameter;

[0303] The target coding component is the same as the second coding component of the reference coding unit under the second pixel parameter.

[0304] In the multimedia file decoding device provided in the embodiment of the present application, since in the encoded data, the difference between the first encoding components of different encoding units in the multimedia file and the first encoding unit under the first pixel parameter is combined, and the first encoding unit in the multimedia file is encoded according to the parameter value of the reference encoding unit matching the first encoding unit under the second pixel parameter, compared with the linear model obtained according to the reconstructed pixels around the encoding unit, the encoding prediction method of the encoding unit under different pixel parameters improves the encoding performance of the encoding unit under the second pixel parameter, thereby improving the encoding performance of the multimedia file as a whole, so that a multimedia file with less distortion than the original multimedia file can be decoded at the decoding end.

[0305] The multimedia file decoding device of the embodiment of the present application can execute the multimedia file decoding method provided by the embodiment of the present application, and its implementation principle is similar. The actions performed by each module in the multimedia file decoding device of each embodiment of the present application correspond to the steps in the multimedia file decoding method of each embodiment of the present application. For the detailed functional description of each module of the multimedia file decoding device, please refer to the description in the corresponding multimedia file decoding method shown in the previous text, which will not be repeated here.

[0306] Based on the same principles as the various methods and devices provided in the embodiments of the present application, an electronic device (such as a server) is also provided in the embodiments of the present application. The electronic device may include a memory, a processor, and a computer program stored in the memory. The processor executes the above-mentioned computer program to implement the method provided in any optional embodiment of the present application.

[0307] See also Fig.12 , Fig.12 FIG. 1 is a schematic diagram showing the structure of an electronic device provided by an embodiment of the present application. Fig.12 As shown, the electronic device 1200 in this embodiment may include: a processor 1201, a network interface 1204 and a memory 1205. In addition, the above-mentioned electronic device 1200 may also include: an object interface 1203, and at least one communication bus 1202. Among them, the communication bus 1202 is used to realize the connection and communication between these components. Among them, the object interface 1203 may include a display screen (Display), a keyboard (Keyboard), and the object interface 1203 may optionally include a standard wired interface and a wireless interface. The network interface 1204 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1205 may be a high-speed RAM memory, or it may be a non-volatile memory (NVM), such as at least one disk storage. The memory 1205 may optionally also be at least one storage device located away from the aforementioned processor 1201. As Fig.12 As shown, the memory 1205 as a computer-readable storage medium may include an operating system, a network communication module, an object interface module, and a device control application program.

[0308] exist Fig.12 In the electronic device 1200 shown, the network interface 1204 can provide a network communication function; the object interface 1203 is mainly used to provide an input interface for the object; and the processor 1201 can be used to call the device control application stored in the memory 1205 to achieve:

[0309] In some feasible implementations, the processor 1201 is configured to:

[0310] It should be understood that in some feasible implementations, the processor 1201 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the type of device.

[0311] In a specific implementation, the electronic device 1200 can execute the above-mentioned functions through its built-in functional modules. Figure 1 or Figure 6 For the implementation methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0312] The present application also provides a computer-readable storage medium, which stores a computer program and is executed by a processor to implement Figure 1 or Figure 6 For the methods provided in each step, please refer to the implementation methods provided in the above steps for details, which will not be repeated here.

[0313] The computer-readable storage medium may be an internal storage unit of any device or electronic device provided in any of the aforementioned embodiments, such as a hard disk or memory of an electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the electronic device. The computer-readable storage medium may also include a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. Further, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0314] The present invention provides a computer program product, which includes a computer program. The computer program is executed by a processor. Figure 1 or Figure 6 The methods provided in each step.

[0315] The terms "first", "second" and the like in the claims, the specification and the drawings of the present application are used to distinguish different objects rather than to describe a specific order.

[0316] In addition, those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. The terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or electronic devices.

[0317] Mentioning "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. Displaying the phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0318] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0319] The above disclosure is only the preferred embodiment of the present application, which cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A multimedia file encoding method, characterized in that: The method comprises: Pre-encoding a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component; According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter; Acquire a first parameter value of the reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format; The first encoding unit is encoded according to the first parameter value.

2. The method according to claim 1, characterized in that: The matching of the reference coding unit for the first coding unit in the multimedia file includes: Determine position information of the first encoding unit in the multimedia file; Determine a matching area in the multimedia file using the location information as a matching center; The reference coding unit is matched for the first coding unit within the matching region.

3. The method according to claim 2, characterized in that The position information includes pixel position parameters, width and height of a central pixel of the first coding unit; The determining of the matching area in the multimedia file includes at least one of the following: In the multimedia file, taking the target image frame including the first encoding unit as the matching area; A matching area centered around the pixel position parameter is determined in the target image frame according to the width, the width threshold, the height, and the height threshold.

4. The method according to any one of claims 2 or 3, characterized in that The matching of the reference coding unit for the first coding unit in the multimedia file includes: Determine a first error between the first coding component and a candidate coding component; wherein the candidate coding component is obtained by encoding a parameter value of a candidate coding unit in a matching area under the first pixel parameter; The reference coding unit is determined from the candidate coding units according to the first error.

5. The method according to claim 4, characterized in that The determining, according to the first error, the reference coding unit from the candidate coding units comprises at least one of the following: Determine the candidate coding unit with the smallest first error as the reference coding unit; A candidate coding unit whose first error is less than or equal to a preset threshold is determined as the reference coding unit.

6. The method according to claim 1, characterized in that The step of encoding the first encoding unit according to the first parameter value includes: Precoding the first parameter value to obtain a second coded component; Using the candidate coding mode, and predicting, according to the second coding component, a third coding component of the first coding unit under the second pixel parameter; Determining, according to the third coding component, a target coding component of the first coding unit under the second pixel parameter; The first coding unit is encoded according to the first coding component and the target coding component.

7. The method according to claim 6, characterized in that The determining, according to the third coding component, a target coding component of the first coding unit under the second pixel parameter includes: Determine a second parameter value of the first encoding unit under the second pixel parameter; determining a target coding mode from the candidate coding modes according to a second error between the third coding component and the second parameter value, and taking the third coding component corresponding to the target coding mode as the target coding component; The candidate coding mode includes a first coding mode; when the target coding mode is the first coding mode, the second coding component is used as the third coding component.

8. The method according to claim 7, characterized in that After encoding the first encoding unit, the method further includes: Determining a mode identifier corresponding to the first encoding mode; Generate encoding data of the multimedia file according to the mode identifier and the encoding result of the first encoding unit.

9. The method according to claim 1, characterized in that: The multimedia file includes a video stream captured from the screen.

10. A multimedia file decoding method, characterized in that: The method comprises: Obtaining encoded data of multimedia files; Decoding the encoded data; The encoded data is obtained by encoding the multimedia file in the following manner: Performing a pre-coding operation on a parameter value of a first coding unit of the multimedia file under a first pixel parameter to obtain a first coding component; According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter; Acquire a first parameter value of the reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format; The first encoding unit is encoded according to the first parameter value.

11. The decoding method according to claim 10, characterized in that: The decoding of the encoded data comprises: Determining a target encoding mode according to a mode identifier in the encoding data; When it is determined that the target encoding mode is the first encoding mode, the encoded data is decoded according to a decoding mode corresponding to the first encoding mode.

12. A multimedia file processing system, characterized in that: The system comprises: An encoding module, used for encoding the first multimedia file; A decoding module, used for decoding the acquired coded data; The encoding module encodes the first multimedia file in the following manner: Performing a pre-coding operation on a parameter value of a first coding unit of the first multimedia file under a first pixel parameter to obtain a first coding component; According to the first coding component, a first reference coding unit is matched for the first coding unit in the first multimedia file; wherein an error between a first reference coding component of the first reference coding unit and the first coding component meets a preset requirement, and the first reference coding component is obtained by encoding a parameter value of the first reference coding unit under the first pixel parameter; Obtaining a first parameter value of the first reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format; Encoding the first encoding unit according to the first parameter value; The coded data is obtained by encoding the second multimedia file in the following manner: performing a pre-coding operation on a parameter value of a second coding unit of the second multimedia file under the first pixel parameter to obtain a fourth coding component; According to the fourth coding component, matching a second reference coding unit for the second coding unit in the second multimedia file; wherein an error between the second reference coding component of the second reference coding unit and the fourth coding component meets the preset requirement, and the second reference coding component is obtained by encoding a parameter value of the second reference coding unit under the first pixel parameter; Obtain a third parameter value of the second reference coding unit under a second pixel parameter; The fourth encoding unit is encoded according to the third parameter value.

13. A multimedia file encoding device, characterized in that: The device comprises: An encoding module, configured to perform a pre-encoding operation on a parameter value of a first encoding unit of the multimedia file under a first pixel parameter to obtain a first encoding component; a coding unit matching module, configured to match a reference matching unit for the first coding unit in the multimedia file according to the first coding component; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter; A parameter value acquisition module, used to acquire a first parameter value of the reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format; The encoding module is further used to encode the first encoding unit according to the first parameter value.

14. A multimedia file decoding device, characterized in that: The device comprises: A data acquisition module, used for acquiring encoded data of multimedia files; A decoding module, used for decoding the encoded data; The encoded data is obtained by encoding the multimedia file in the following manner: Performing a pre-coding operation on a parameter value of a first coding unit of the multimedia file under a first pixel parameter to obtain a first coding component; According to the first coding component, a reference coding unit is matched for the first coding unit in the multimedia file; wherein an error between a reference coding component of the reference coding unit and the first coding component meets a preset requirement, and the reference coding component is obtained by encoding a parameter value of the reference coding unit under the first pixel parameter; Acquire a first parameter value of the reference coding unit under a second pixel parameter; wherein the first pixel parameter and the second pixel parameter are in the same pixel format; The first encoding unit is encoded according to the first parameter value.

15. An electronic device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected to each other; The memory is used to store computer programs; The processor is configured to execute the method of any one of claims 1 to 9 and 10 to 11 when calling the computer program.

Images