Image processing method and related device

Abstract

The embodiment of the invention discloses an image processing method and a related device, and the method comprises the steps: obtaining a target animation file, carrying out the decoding processing of the target animation file, obtaining the transparency configuration information of the target animation file and a plurality of original image frames of a target animation effect, and performing transparency processing on each original image frame of the target animation effect based on the transparency configuration information to obtain a transparent image frame corresponding to each original image frame, and playing the transparent image frame corresponding to each original image frame. Transparent configuration indicated by the transparency configuration information is utilized to perform transparent processing on each original image frame of the target animation effect to obtain a transparent image frame corresponding to each original image frame, and each transparent image frame is played, so that the effect of playing the transparent animation video can be realized. And the animation display effect is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method and related devices. Background Art

[0002] Nowadays, many applications play animation effects at specific times to provide users with a fresh and interesting visual experience. For example, a live broadcast application will play the corresponding gift animation when a gift is given in the live broadcast room. Another example is that a game application will play the corresponding game result animation when showing the game result of a game match, and so on.

[0003] Currently, MP4 (MEPG-4, Moving Picture Experts Group 4) files are usually used as video files carrying the above animations because of their good compatibility, high compression rate, high hardware decoding efficiency, etc. However, MP4 files cannot support transparent backgrounds, and the playback effect of the animations they carry is usually not ideal. Summary of the invention

[0004] The embodiments of the present application provide an image processing method and related devices to improve the display effect of animation.

[0005] The first aspect of the present application provides an image processing method, the method comprising:

[0006] Obtain a target animation file; the target animation file is a video file corresponding to a target animation effect;

[0007] Decoding the target animation file to obtain transparency configuration information and a plurality of original image frames of the target animation effect included in the target animation file; the transparency configuration information is used to indicate the transparency configuration of the original image frame;

[0008] Based on the transparency configuration information, each original image frame of the target animation effect is subjected to transparency processing to obtain a transparent image frame corresponding to each original image frame;

[0009] Play the transparent image frames corresponding to the original image frames.

[0010] A second aspect of the present application provides an image processing device, the device comprising:

[0011] An acquisition module is used to acquire a target animation file; the target animation file is a video file corresponding to a target animation effect;

[0012] A decoding module, used for decoding the target animation file to obtain transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect; the transparency configuration information is used to indicate the transparency configuration of the original image frame;

[0013] A transparency module, used for performing transparency processing on each original image frame of the target animation effect based on the transparency configuration information to obtain a transparent image frame corresponding to each original image frame;

[0014] The playing module is used to play the transparent image frames corresponding to the original image frames.

[0015] Optionally, the transparency module includes:

[0016] A determination unit, used to determine a transparency configuration image according to the transparency configuration information;

[0017] The transparency unit is used to determine the transparency value of each pixel in the original image frame according to the color channel value of each pixel in the transparency configuration image for each original image frame; and determine the transparent image frame corresponding to the original image frame based on the color channel value and transparency value of each pixel in the original image frame.

[0018] Optionally, the transparency configuration information includes a correspondence between a color channel value and a pixel area; and the determining unit includes:

[0019] A first determining subunit is used to determine a basic transparency configuration image having the same size as the original image frame;

[0020] The second determining subunit is used to determine the color channel value of each pixel in the basic transparency configuration image according to the correspondence between the color channel value and the pixel area included in the transparency configuration information, so as to obtain the transparency configuration image.

[0021] Optionally, the transparency unit is specifically used to: for each pixel in the original image frame, determine a pixel in the transparency configuration image that is at the same position as the pixel as a reference pixel corresponding to the pixel; and calculate the average of the color channel values ​​of the reference pixels as the transparency value of the pixel.

[0022] Optionally, the decoding module includes:

[0023] A first transmission unit, used for transmitting a first buffer object to a decoder; the first buffer object carries an information unit in a target animation file;

[0024] A decoding unit, used to decode the content carried by the information unit through a decoder to obtain a decoding result; the decoding result is transparency configuration information or an original image frame;

[0025] The second transmission unit is used to transmit the decoding result to the second buffer object by outputting the callback function.

[0026] Optionally, the target animation file is a video file encrypted using a target key; the device further comprises:

[0027] A third transmission unit, used for transmitting a key encryption factor to a video server;

[0028] The receiving unit is used to receive a target encryption key; the target encryption key is obtained by encrypting the target key by the video server using a key encryption factor.

[0029] Optionally, before the decoding module, the device further includes:

[0030] A first decryption unit, configured to use a key encryption factor to decrypt a target encryption key to obtain a target key;

[0031] The second decryption unit is used to use the target key to decrypt the target animation file.

[0032] A third aspect of the present application provides a computer device, the device comprising a processor and a memory:

[0033] The memory is used to store computer programs;

[0034] The processor is used to execute the steps of the image processing method as described in the first aspect according to the computer program.

[0035] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the steps of the image processing method described in the first aspect above.

[0036] In a fifth aspect, the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of the image processing method described in the first aspect above.

[0037] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0038] The embodiment of the present application obtains the target animation file and decodes the target animation file to obtain the transparency configuration information and multiple original image frames of the target animation effect included in the target animation file, and then performs transparency processing on each original image frame of the target animation effect based on the transparency configuration information to obtain the transparent image frame corresponding to each original image frame, and plays the transparent image frame corresponding to each original image frame. In the embodiment of the present application, the configuration information in the target animation file in MP4 format is innovatively used to carry the transparency configuration content for the original image frame to obtain the transparency configuration information; then, by using the transparency configuration indicated by the transparency configuration information, each original image frame of the target animation effect is transparently processed to obtain the transparent image frame corresponding to each original image frame, and each transparent image frame is played to achieve the effect of playing a transparent animation video, that is, the MP4 file can achieve a transparent animation effect, which improves the display effect of the animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of playing a game result animation provided in an embodiment of the present application;

[0040] Figure 2 A schematic diagram of another method of playing a game result animation provided in an embodiment of the present application;

[0041] Figure 3 A schematic diagram of a scene of an image processing method provided in an embodiment of the present application;

[0042] Figure 4 A flowchart of an image processing method provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of a container box in an MP4 file provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of a transparent image frame provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of a transparency configuration image provided by an embodiment of the present application;

[0046] Figure 8 A schematic diagram of an encryption and decryption process of a target animation file provided in an embodiment of the present application;

[0047] Fig. 9 A schematic diagram of decoding a target animation file provided in an embodiment of the present application;

[0048] Fig.10 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0049] Fig.11 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0050] Fig.12 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] Currently, many applications will play animation effects at specific times to provide users with a fresh and interesting visual experience. For example, a live broadcast application will play the corresponding gift animation when a gift is given in the live broadcast room. Another example is that a game application will play the corresponding game result animation when showing the game result of a game match, and so on. For details, please refer to Figure 1 , which is a schematic diagram of a game result animation provided in an embodiment of the present application.

[0054] At present, MP4 files are usually used as video files carrying the above animations because of their advantages of good compatibility, high compression rate, high hardware decoding efficiency, etc. However, MP4 files cannot support transparent backgrounds, and the playback effect of the animations they carry is usually not ideal.

[0055] Combination Figure 1As shown, the animation A displays the word "Victory" and other elements are displayed around the word. However, the blank area between the word and other elements cannot be made transparent and can only be rendered in a color similar to the page after the game is over.

[0056] In order to solve the above technical problems, an embodiment of the present application provides an image processing method, in which a target animation file is obtained, and the target animation file is decoded to obtain the transparency configuration information included in the target animation file and multiple original image frames of the target animation effect, and then each original image frame of the target animation effect is transparently processed based on the transparency configuration information to obtain the transparent image frame corresponding to each original image frame, and the transparent image frame corresponding to each original image frame is played. In the embodiment of the present application, the configuration information in the target animation file in MP4 format is innovatively used to carry the transparency configuration content for the original image frame to obtain the transparency configuration information; then, by using the transparency configuration indicated by the transparency configuration information, each original image frame of the target animation effect is transparently processed to obtain the transparent image frame corresponding to each original image frame, and each transparent image frame is played, so that the effect of playing a transparent animation video can be achieved, that is, the MP4 file can achieve a transparent animation effect, which improves the display effect of the animation.

[0057] See also Figure 2 , this figure is a schematic diagram of another method of playing a game result animation provided in an embodiment of the present application.

[0058] Combination Figure 2 As shown, as an example, the target animation file is an MP4 file corresponding to animation A. By decoding the target animation file, the transparency configuration information and multiple original image frames of animation A can be obtained. Then, based on the transparency configuration information, each original image frame of animation A is transparently processed to obtain a transparent image frame corresponding to each original image frame. The transparent animation A can be played by playing each transparent image frame.

[0059] See also Figure 3 , this figure is a scene schematic diagram of an image processing method provided in an embodiment of the present application, and the application scene includes a terminal device 301.

[0060] The terminal device 301 obtains the target animation file, wherein the target animation file is a video file corresponding to the target animation effect. As an example, assuming that a video file of a victory animation effect needs to be played after the game ends, the video file is obtained as the target animation file.

[0061] The terminal device 301 decodes the target animation file to obtain the transparency configuration information included in the target animation file and multiple original image frames of the target animation effect; the transparency configuration information is used to indicate the transparency configuration of the original image frame. As an example, assuming that the target animation file includes 10 original image frames, it is necessary to obtain these 10 original image frames and the transparency configuration information corresponding to these 10 original image frames; it should be understood that, under normal circumstances, the transparency effects of each image frame in an animation effect are basically the same, and the transparency configuration information carried by the target animation file can be used to perform transparency processing on each original image frame.

[0062] The terminal device 301 performs transparency processing on each original image frame of the target animation effect based on the transparency configuration information to obtain a transparent image frame corresponding to each original image frame. As an example, after obtaining 10 original image frames and the transparency configuration information, each original image frame can be transparently processed, that is, a specific area in the original image frame is transparent, and a transparent image frame corresponding to each original image frame can be obtained.

[0063] The terminal device 301 plays the transparent image frames corresponding to the original image frames. As an example, after the 10 original image frames of the target animation file are processed into 10 transparent image frames, the transparent image frames are played in the order of the original image frames. At this time, the playback effect presented by the target animation file is a transparent animation playback effect.

[0064] The image processing method provided in the embodiments of the present application can be applied to terminal devices with image display functions, including but not limited to mobile phones, tablets, computers, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc., but not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0065] The target animation file in the embodiment of the present application can be provided by a server, which can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0066] The image processing method provided in the embodiment of the present application involves cloud technology.

[0067] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0068] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, each item may have its own identification mark in the future, and all need to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.

[0069] The relevant data collection and processing in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, obtain the informed consent or separate consent of the subject of personal information, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the subject of personal information.

[0070] See also Figure 4 , which is a flow chart of an image processing method provided by an embodiment of the present application. Figure 4 As shown, the image processing method provided in the embodiment of the present application may include:

[0071] S401: Obtain a target animation file.

[0072] The target animation file refers to the video file corresponding to the target animation effect. In the embodiment of the present application, the target animation file is a file in MP4 format; the target animation effect refers to the effect of the target animation displayed by the object. As an example, the target animation effect is Figure 2 If you want to achieve the playback effect of animation A in , then the target animation can be animation A, and the target animation file is the video file corresponding to animation A.

[0073] S402: Decode the target animation file to obtain transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect.

[0074] The original image frame refers to each original image frame of the target animation file. It can be understood that a video file is composed of multiple image frames, and the original image frame does not include the transparency effect. The original image frame usually includes the red, green, and blue (RGB) channel values ​​of each pixel therein, but does not include the transparency (Alpha) channel value of each pixel.

[0075] The transparency configuration information is used to indicate the transparency configuration of the original image frame. It should be understood that the target animation file is a video file in MP4 format. There are multiple container boxes in the MP4 file. The container box can be used to carry relevant configuration information of the video and can also be used to carry image frames. In this embodiment of the present application, at least one container box in the MP4 file can be used to carry configuration information indicating the transparent area, that is, there is a container box in the target animation file, which is used to carry the transparency configuration of the original image frame in the target animation file, that is, the target animation file contains transparency configuration information, and the transparency configuration information is the container box that carries the transparency configuration of the original image frame.

[0076] Multimedia encapsulation format (also called container format) refers to putting video data, audio data, etc. into a file according to certain rules. MP4 is one of the most common encapsulation formats and is widely used because of its cross-platform characteristics. MP4 files are composed of multiple container boxes, each of which stores different information, and the container boxes are in a tree structure. For details, see Figure 5 As shown, this figure is a schematic diagram of a container box in an MP4 file provided in an embodiment of the present application.

[0077] Each container box has two attributes: type and length. The container box can be understood as a data object block, which itself consists of a header and data / body. The header contains the length size and type type of the entire container box. When size == 0, it means that this is the last container box in the file; when size == 1, it means that the length of the container box requires more bytes to describe, and a 64-byte largesize (large length) will be defined later to describe the length of the container box. Data / body is the actual data of the container box, which can be pure data or more sub-container boxes. As an example, the header can be the transparency of the original image frame, and the data can be the transparency data corresponding to the transparency of the original image frame.

[0078] S403: Based on the transparency configuration information, each original image frame of the target animation effect is subjected to transparency processing to obtain a transparent image frame corresponding to each original image frame.

[0079] The transparent image frame refers to an image frame obtained by making the original image frame transparent. The set of the transparent image frames is an animation video with a transparent effect.

[0080] It should be understood that after obtaining the transparency configuration information, each original image frame in the target animation effect can be made transparent based on the transparency configuration indicated by the transparency configuration information, that is, a specific area in the original image frame is made transparent, and animation elements are displayed based on the transparent area.

[0081] S404: Play the transparent image frames corresponding to the original image frames.

[0082] It should be understood that by converting each original image frame into a corresponding transparent playback effect based on the corresponding fixed transparency configuration, the effect of playing a transparent animation video is achieved.

[0083] The image processing method provided in the embodiment of the present application obtains the target animation file and decodes the target animation file to obtain the transparency configuration information and multiple original image frames of the target animation effect included in the target animation file, and then performs transparency processing on each original image frame of the target animation effect based on the transparency configuration information to obtain the transparent image frame corresponding to each original image frame, and plays the transparent image frame corresponding to each original image frame. In the embodiment of the present application, the configuration information in the target animation file in MP4 format is innovatively used to carry the transparency configuration content for the original image frame to obtain the transparency configuration information; then, by using the transparency configuration indicated by the transparency configuration information, each original image frame of the target animation effect is transparently processed to obtain the transparent image frame corresponding to each original image frame, and each transparent image frame is played to achieve the effect of playing a transparent animation video, that is, the MP4 file can achieve a transparent animation effect, which improves the display effect of the animation.

[0084] Based on the image processing method provided in the above embodiment, in order to further illustrate the process of transparency processing, in some possible implementations, step S403 may specifically include:

[0085] A1: Determine a transparency configuration image according to the transparency configuration information.

[0086] The transparency configuration image refers to an image corresponding to the original image frame and used to indicate the transparency configuration, and can be used to indicate the transparency channel value of each pixel in the original image frame.

[0087] A2: For each original image frame, determine the transparency value of each pixel in the original image frame according to the color channel value of each pixel in the transparency configuration image; and determine the transparent image frame corresponding to the original image frame based on the color channel value and transparency value of each pixel in the original image frame.

[0088] The color channel value refers to the value of the color channel corresponding to the pixel. There are a few color spaces that can represent images, such as RGB, BGR, HSV, CMYK, etc., which are channels used by these color spaces to form images together.

[0089] Color digital images are made up of pixels, which are composed of a combination of primary colors represented by a series of codes. In this case, a channel is a grayscale image of the same size as the color image, composed of only one of these primary colors. That is, each raw image frame is made up of pixels, and each pixel is composed of a combination of colors. A channel is a grayscale image of a color image, which is composed of only one of the primary colors that make up the color image.

[0090] In the embodiment of the present application, an RGB image can be taken as an example. The RGB image is a three-channel image. Each pixel is composed of three channels, each channel represents a color, that is, the three channels represent the three color channels of red (R), green (G), and blue (B). For example, the color channel value corresponding to a pixel can be [R: 255, G: 255, B: 255].

[0091] The transparency value corresponds to the color channel value, and is used to indicate the transparency of the pixel corresponding to the color channel value. It can be understood that the transparency value is generally any decimal value or integer value between 0 and 1. If the transparency value is 0, the pixel is transparent regardless of the color represented by the color channel value.

[0092] It should be understood that by determining the color and transparency ultimately represented by each pixel, the color and transparency of the transparent image frame corresponding to the original image frame can be determined, that is, the effect of a transparent animation video can be achieved, thereby improving the display effect of the animation.

[0093] See also Figure 6 , which is a schematic diagram of a transparent image frame provided by an embodiment of the present application. Figure 6 A in is the original image frame, Figure 6 B in is the transparency configuration image corresponding to the original image frame. Figure 6 C in is the configuration image corresponding to the original image frame, Figure 6 D in is the transparent image frame. Figure 6 E in FIG. 1 is the display effect of the transparent image frame.

[0094] Combination Figure 6 As shown, in the embodiment of the present application, the transparency configuration information includes the corresponding relationship between the color channel value and the pixel area, specifically Figure 6For example, assuming that the color channel value of a pixel is [R:128, G:128, B:128], and the pixel area is divided from left to right, from top to bottom, horizontally as rows, and vertically as columns, then the coordinates of the first row (that is, the first column) can be (1,1,) and the color channel value [R:128, G:128, B:128] in the pixel area (that is, the position) can be (1,1) and (3,3). It should be understood that through the bet-win relationship between the color channel value and the pixel area, the amount of information carried by the transparency configuration information can be reduced without causing information loss, saving resource usage.

[0095] The process of determining the transparent image frame may be:

[0096] B1: Determines a base transparency profile image with the same frame size as the original image.

[0097] The basic transparency configuration image means an image having the same size as the original image frame and used to represent the transparency configuration.

[0098] B2: According to the correspondence between the color channel value and the pixel area included in the transparency configuration information, the color channel value of each pixel in the basic transparency configuration image is determined to obtain the transparency configuration image.

[0099] It can be understood that there is a corresponding relationship between the color channel value included in the transparency configuration information and the pixel area, so each pixel area in the basic transparency configuration image can be represented by the corresponding color channel value to obtain the transparency configuration image. Specifically, for each pixel in the basic transparency configuration image, according to the pixel area to which the pixel belongs, the color channel value corresponding to the pixel area carried in the transparency configuration information is determined as the color channel value of the pixel, so that the transparency configuration image is restored based on the transparency configuration information carrying less information.

[0100] For details, please refer to Figure 7 , which is a schematic diagram of a transparency configuration image provided by an embodiment of the present application, wherein: Figure 7 The X in represents the original image frame image, Figure 7 The Y in the image represents the transparency configuration image.

[0101] B3: For each pixel in the original image frame, determine a pixel in the transparency configuration image that is at the same position as the pixel as a reference pixel corresponding to the pixel; and calculate the average of the color channel values ​​of the reference pixels as the transparency value of the pixel.

[0102] Combination Figure 6As shown in C, the figure is the configuration image corresponding to the original image frame, that is, the position and color channel value of each pixel in the original image frame. Taking pixels a, b, and c as examples, in the transparent configuration image B, the pixels corresponding to pixels a, b, and c are a1, b1, and c1, respectively. Pixels a1, b1, and c1 are used as reference pixels of pixels a, b, and c, respectively. By determining the average value of the color channel value of each pixel a1, b1, and c1, that is, determining the average value of the R channel value, G channel value, and B channel value of each pixel a1, b1, and c1, the transparency values ​​corresponding to pixels a, b, and c are K1, K2, and K3, respectively. For example, assuming that the color channel value of the reference pixel a1 is [R: 128, G: 128, B: 128], then the average value of the color channel value of the pixel a1 is 128, and [K1: 128] is used as the transparency value of the pixel a in the original image frame.

[0103] It should be noted that, in this embodiment, the color channel value of each pixel in the basic transparency configuration image is determined by the correspondence between the color channel value and the pixel area included in the transparency configuration information, and the transparency configuration image is obtained. This can reduce the resolution of the video file and improve the compatibility of different device types without affecting the target animation effect displayed by the target animation file.

[0104] Based on the image processing method provided in the above embodiment, in order to further illustrate the decoding process of the target animation file, step S402 may include:

[0105] C1: Transfer the first buffer object to the decoder.

[0106] Among them, the first buffer object means a compressed object containing one or more animation videos, that is, the first buffer object carries the information unit in the target animation file; the information unit means a unit for storing relevant data of the target animation file, which can be specifically a container box.

[0107] As a possible implementation method, the first buffer object can be represented by CMSampleBuffer (Controller Memory Sample Buffer), and when the target animation file needs to be decoded, the target animation file will be transmitted through CMSampleBuffer. Each CMSampleBuffer contains one or more encoded H.264 image frames (referring to the original image frames in this application), called NALU (Network Abstraction Layer Unit), which is also the information unit in the embodiment of this application.

[0108] Among them, H.264 is the latest encoding format defined by the MPEG-4 standard. Its video format is lossy compressed, but technically it tries to reduce the storage volume as much as possible to achieve better image quality and fast transmission of low-bandwidth images.

[0109] A CMSampleBuffer is a buffer containing zero or more compressed or uncompressed samples of a specific media type, such as audio, video, or multiplexed.

[0110] CMSampleBuffer can contain CMBlockBuffer (Controller MemoryBlockBuffer) and CVImageBuffer (Computer VisionImageBuffer).

[0111] A CMBlockBuffer contains one or more media samples.

[0112] CVImageBuffer is a reference to a CMSampleBuffer stream format description, containing the size and timing information of each media sample, as well as the buffer level and sample base attachments.

[0113] A SampleBuffer can contain sample-level and buffer-level attachments. Sample-level attachments are associated with each sample (frame) of the buffer and contain information such as timestamps and image frame related information. Buffer-level attachments provide information about the buffer as a whole, such as playback speed and the operations performed when consuming the buffer.

[0114] It should be understood that when acquiring the target animation file, the target animation file is transmitted based on the first buffer object.

[0115] C2: Decode the content carried by the information unit through a decoder to obtain a decoding result.

[0116] The decoding result is transparency configuration information or an original image frame.

[0117] It should be understood that, since the relevant data of the target animation file is stored based on the information unit, after the information unit is acquired, the information unit needs to be decoded to obtain the transparency configuration information or the original image frame.

[0118] Among them, after step C2, it is also necessary to set the output callback function of the decoder, which can contain a pointer to the CVPixelBuffer (Computer PixelBuffer) containing the decoded data. The callback function can also be used to process the decoded CVPixelBuffer, such as display, processing, export, etc.

[0119] C3: The decoding result is transmitted to the second buffer object through the output callback function.

[0120] The output callback function refers to a function called after the information unit is decoded, which is mainly used for display, processing, export and other operations after decoding. That is, through the callback function, the decoding result can be exported and transmitted to the second buffer object.

[0121] Among them, the second buffer object means an object used to encapsulate the decoded decoding result, and the second buffer object can save the saved pixels, image width and height, pointer and other information of the target animation file. As an example, when the output callback function is called, a pointer to the CVPixelBuffer containing the decoded image data will be obtained. The CVPixelBuffer refers to the second buffer object in the embodiment of the present application, which can be used for subsequent image processing and rendering, such as using CoreImage (image processing framework) or Metal (image processing engine) for image processing, or rendering directly on a layer.

[0122] Based on the image processing method provided in the above embodiment, in the related art, for the security of the target animation file, the video file corresponding to the target animation is generally encrypted to ensure security and non-tamperability. However, the key used in the encryption process is in a public state, which may cause the target animation file to be decrypted, resulting in poor security.

[0123] In order to solve the above technical problems, some possible implementations may also include:

[0124] D1: Transmit the key encryption factor to the video server.

[0125] The key encryption factor means an element used to encrypt the target key corresponding to the target animation file.

[0126] The video server refers to a server that encrypts the target animation file.

[0127] D2: Receive the target encryption key; the target encryption key is obtained by encrypting the target key by the video server using the key encryption factor.

[0128] After receiving the key encryption factor, the video server can use the key encryption factor to encrypt the target key, which is the basic key used to encrypt the target animation file.

[0129] It should be understood that by using the key encryption factor to encrypt the target key by the video server, the target key can be prevented from being leaked, thereby further causing the target animation file to be leaked, thereby improving security.

[0130] After the target encryption key is obtained through steps D1 and D2, before step S402, the following may also be included:

[0131] E1: Use the key encryption factor to decrypt the target encryption key to obtain the target key.

[0132] E2: Use the target key to decrypt the target animation file.

[0133] It should be understood that the target animation file will be encrypted by the target key. At the same time, the target key will be encrypted by the key encryption factor to obtain the target encryption key. Therefore, before decoding the target animation file, it is also necessary to first use the key encryption factor to decrypt the target encryption key, and use the decrypted target key to decrypt the target animation file to ensure the consistency of the transmitted file.

[0134] In this way, the target animation file is encrypted with the target key, and the target key is further encrypted with the key encryption factor. Through this secondary encryption mechanism, the security of the target key can be effectively guaranteed, thereby further ensuring the security of the target animation file, thereby improving the security of the target animation file during the transmission process as a whole, and more effectively preventing the target animation file from being stolen and tampered with.

[0135] See also Figure 8 , which is a schematic diagram of the encryption and decryption process of a target animation file provided in an embodiment of the present application.

[0136] Combination Figure 8 As shown, the encryption and decryption process of the target animation file provided in the embodiment of the present application can be:

[0137] Step 1: The COS service encrypts the video file using the target key.

[0138] Among them, the COS service (Cloud Object Storage) can call the cloud key management service to generate a target key according to the needs of the user. For example, the cloud key management service can be called to generate a target key according to the name, number and other information selected by the user.

[0139] Step 2: Encrypt the video file using the generated target key, the video file is the video file corresponding to the target animation, and obtain the target animation file.

[0140] Step 3: The client sends the defined key encryption factors (overlaykey and overlayiv) to the server through the CGI interface, and the server uses the key encryption factors to encrypt the target key to obtain the target encryption key.

[0141] Step 4: The server sends the target animation file and the target encryption key to the client.

[0142] Step 5: The client decrypts the target encryption key according to the key encryption factor stored locally to obtain the target key.

[0143] Step 6: Decrypt the target animation file using the target key to obtain the decrypted target animation file.

[0144] Step 7: Demultiplex the audio file and the video file of the target animation file respectively to obtain video compression data and audio compression data.

[0145] Step 8: Decompress the video compressed data and the audio compressed data respectively to obtain the original video data and the original audio data.

[0146] Step 9: Synchronize the original video data and the original audio data.

[0147] Step 10: Perform cropping, modification, super-resolution and other processing on the original video data to obtain processed video data.

[0148] Step 11: Resample and denoise the original audio data to obtain processed audio data.

[0149] Step 12: Render the processed video data and audio data respectively.

[0150] Step 13: Display the rendered video data through the transparency area.

[0151] The encryption and decryption process of the target animation file provided in the embodiment of the present application avoids data leakage and improves data security by performing secondary encryption on the target key.

[0152] Based on the image processing method provided in the above embodiment, the present application embodiment also provides a decoding process of a target animation file, see Fig. 9 , which is a schematic diagram of decoding a target animation file provided in an embodiment of the present application.

[0153] Combination Fig. 9As shown, the target animation file decoding process provided by the embodiment of the present application may include:

[0154] The decoding process provided in the embodiment of the present application can be implemented based on the capabilities of the player. Its underlying layer relies on the player hardware to decode the target animation file. After the original image frame is decoded, Metal (image processing engine) and the pre-set transparency processing are used to obtain a transparent image frame, and multiple transparent image frames are displayed through Metal in the order of the original image frames.

[0155] Among them, Metal is a set of programming APIs (Application Programming Interface) for operating GPU (Graphics Processing Unit, also known as graphics card).

[0156] F1: Get CMSampleBuffers.

[0157] F2: Create VTDecompressionSession (a session object that manages the decompression of incoming video data).

[0158] Specifically, a VTDecompressionSession can be created using the VTDecompressionSessionCreate() function. During the creation, the output callback function of the video decoder and the codec parameters (such as resolution, bit rate, etc.) need to be specified.

[0159] F3: Set the decoder's output callback function.

[0160] This output callback function can contain a pointer to a CVPixelBuffer (ComputerPixelBuffer) containing the decoded data. The callback function can also be used to process the decoded CVPixelBuffer, such as display, processing, export, etc.

[0161] F4: Pass CMSampleBuffer to VTDecompressionSession.

[0162] When receiving CMSampleBuffer, call VTDecompressionSessionDecodeFrame() function. This function can pass CMSampleBuffer to the decoder and decode it. After decoding is completed, the output callback function will be called.

[0163] F5: Get the CVPixelBuffer in the output callback.

[0164] When the output callback function is called, it will get a pointer to a CVPixelBuffer containing the decoded image data. The CVPixelBuffer can be used for subsequent image processing and rendering, such as using Core Image (image processing framework) or Metal (image processing engine) for image processing, or rendering directly on a layer.

[0165] See also Fig.10 , which is a structural schematic diagram of an image processing device provided in an embodiment of the present application.

[0166] Combination Fig.10 As shown, the image processing device 1000 provided in the embodiment of the present application may include:

[0167] The acquisition module 1001 is used to acquire a target animation file; the target animation file is a video file corresponding to a target animation effect;

[0168] The decoding module 1002 is used to decode the target animation file to obtain the transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect; the transparency configuration information is used to indicate the transparency configuration of the original image frame;

[0169] A transparency module 1003 is used to perform transparency processing on each original image frame of the target animation effect based on the transparency configuration information to obtain a transparent image frame corresponding to each original image frame;

[0170] The playing module 1004 is used to play the transparent image frames corresponding to the original image frames.

[0171] As an example, the transparency module 1003 includes:

[0172] A determination unit, used to determine a transparency configuration image according to the transparency configuration information;

[0173] The transparency unit is used to determine the transparency value of each pixel in the original image frame according to the color channel value of each pixel in the transparency configuration image for each original image frame; and determine the transparent image frame corresponding to the original image frame based on the color channel value and transparency value of each pixel in the original image frame.

[0174] As an example, the transparency configuration information includes a correspondence between a color channel value and a pixel area; the determining unit includes:

[0175] A first determining subunit is used to determine a basic transparency configuration image having the same size as the original image frame;

[0176] The second determining subunit is used to determine the color channel value of each pixel in the basic transparency configuration image according to the correspondence between the color channel value and the pixel area included in the transparency configuration information, so as to obtain the transparency configuration image.

[0177] As an example, the transparency unit is specifically used to: for each pixel in the original image frame, determine the pixel at the same position as the pixel in the transparency configuration image as the reference pixel corresponding to the pixel; and calculate the average of the color channel values ​​of the reference pixels as the transparency value of the pixel.

[0178] As an example, the decoding module 1002 includes:

[0179] A first transmission unit, used for transmitting a first buffer object to a decoder; the first buffer object carries an information unit in a target animation file;

[0180] A decoding unit, used to decode the content carried by the information unit through a decoder to obtain a decoding result; the decoding result is transparency configuration information or an original image frame;

[0181] The second transmission unit is used to transmit the decoding result to the second buffer object by outputting the callback function.

[0182] As an example, the target animation file is a video file encrypted using a target key; the device further includes:

[0183] A third transmission unit, used for transmitting a key encryption factor to a video server;

[0184] The receiving unit is used to receive a target encryption key; the target encryption key is obtained by encrypting the target key by the video server using a key encryption factor.

[0185] As an example, before the decoding module 1002, the device further includes:

[0186] A first decryption unit, configured to use a key encryption factor to decrypt a target encryption key to obtain a target key;

[0187] The second decryption unit is used to use the target key to decrypt the target animation file.

[0188] The image processing device provided in the embodiment of the present application obtains the target animation file through the acquisition module, and decodes the target animation file through the decoding module to obtain the transparency configuration information included in the target animation file and multiple original image frames of the target animation effect, and then performs transparency processing on each original image frame of the target animation effect based on the transparency configuration information through the transparency module to obtain the transparent image frame corresponding to each original image frame, and finally plays the transparent image frame corresponding to each original image frame through the playback module. In the embodiment of the present application, the configuration information in the target animation file in MP4 format is innovatively used to carry the transparency configuration content for the original image frame to obtain the transparency configuration information; then, by using the transparency configuration indicated by the transparency configuration information, each original image frame of the target animation effect is transparently processed to obtain the transparent image frame corresponding to each original image frame, and each transparent image frame is played to achieve the effect of playing the transparent animation video, that is, the MP4 file can achieve the transparent animation effect, which improves the display effect of the animation.

[0189] The embodiment of the present application also provides a computer device, which may specifically be a terminal device or a server. The terminal device and the server provided in the embodiment of the present application will be introduced below from the perspective of hardware entity.

[0190] See also Fig.11 , Fig.11 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Fig.11 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant, a point of sales (POS), a car computer, etc., taking the terminal as a computer as an example:

[0191] Fig.11 FIG. 1 is a block diagram showing a partial structure of a computer related to a terminal provided in an embodiment of the present application. Fig.11 The computer includes: a radio frequency (RF) circuit 1210, a memory 1220, an input unit 1230 (including a touch panel 1231 and other input devices 1232), a display unit 1240 (including a display panel 1241), a sensor 1250, an audio circuit 1260 (which can be connected to a speaker 1261 and a microphone 1262), a wireless fidelity (WiFi) module 1270, a processor 1280, and a power supply 1290. Those skilled in the art can understand that Fig.11The computer structure shown in the figure does not constitute a limitation of the computer, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0192] The memory 1220 can be used to store software programs and modules. The processor 1280 executes various functional applications and data processing of the computer by running the software programs and modules stored in the memory 1220. The memory 1220 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the computer (such as audio data, a phone book, etc.), etc. In addition, the memory 1220 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0193] The processor 1280 is the control center of the computer, which uses various interfaces and lines to connect various parts of the entire computer, and executes various functions of the computer and processes data by running or executing software programs and / or modules stored in the memory 1220, and calling data stored in the memory 1220. Optionally, the processor 1280 may include one or more processing units; preferably, the processor 1280 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1280.

[0194] In the embodiment of the present application, the processor 1280 included in the terminal also has the following functions:

[0195] Obtain a target animation file; the target animation file is a video file corresponding to a target animation effect;

[0196] Decoding the target animation file to obtain transparency configuration information and a plurality of original image frames of the target animation effect included in the target animation file; the transparency configuration information is used to indicate the transparency configuration of the original image frame;

[0197] Based on the transparency configuration information, each original image frame of the target animation effect is subjected to transparency processing to obtain a transparent image frame corresponding to each original image frame;

[0198] Play the transparent image frames corresponding to the original image frames.

[0199] Optionally, the processor 1280 is also used to execute the steps of any implementation method of the image processing method provided in the embodiments of the present application.

[0200] See also Fig.12 , Fig.12 A schematic diagram of the structure of a server 1300 provided for an embodiment of the present application. The server 1300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1322 (for example, one or more processors) and a memory 1332, and one or more storage media 1330 (for example, one or more mass storage devices) storing application programs 1342 or data 1344. Among them, the memory 1332 and the storage medium 1330 may be temporary storage or permanent storage. The program stored in the storage medium 1330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1322 may be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the server 1300.

[0201] The server 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358, and / or one or more operating systems, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0202] The steps performed by the server in the above embodiment can be based on the Fig.12 The server structure shown.

[0203] The CPU 1322 is used to execute the following steps:

[0204] Obtain a target animation file; the target animation file is a video file corresponding to a target animation effect;

[0205] Decoding the target animation file to obtain transparency configuration information and a plurality of original image frames of the target animation effect included in the target animation file; the transparency configuration information is used to indicate the transparency configuration of the original image frame;

[0206] Based on the transparency configuration information, each original image frame of the target animation effect is subjected to transparency processing to obtain a transparent image frame corresponding to each original image frame;

[0207] Play the transparent image frames corresponding to the original image frames.

[0208] Optionally, the CPU 1322 may also be used to execute steps of any implementation method of the image processing method provided in the embodiments of the present application.

[0209] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute any one of the implementations of an image processing method described in the aforementioned embodiments.

[0210] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes any one of the implementations of the image processing method described in the above embodiments.

[0211] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0212] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0215] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store computer programs.

[0216] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0217] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Obtain a target animation file; the target animation file is a video file corresponding to a target animation effect; Decoding the target animation file to obtain transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect; the transparency configuration information is used to indicate the transparency configuration of the original image frame; Based on the transparency configuration information, each of the original image frames of the target animation effect is subjected to transparency processing to obtain a transparent image frame corresponding to each of the original image frames; The transparent image frames corresponding to the original image frames are played.

2. The method according to claim 1, characterized in that The method of performing transparency processing on each of the original image frames of the target animation effect based on the transparency configuration information to obtain a transparent image frame corresponding to each of the original image frames includes: Determining a transparency configuration image according to the transparency configuration information; For each of the original image frames, the transparency value of each pixel in the original image frame is determined according to the color channel value of each pixel in the transparency configuration image; and based on the color channel value and transparency value of each pixel in the original image frame, the transparent image frame corresponding to the original image frame is determined.

3. The method according to claim 2, characterized in that The transparency configuration information includes a correspondence between a color channel value and a pixel area; and determining a transparency configuration image according to the transparency configuration information includes: Determine a base transparency configuration image having the same size as the original image frame; According to the correspondence between the color channel values ​​and the pixel areas included in the transparency configuration information, the color channel value of each pixel in the basic transparency configuration image is determined to obtain the transparency configuration image.

4. The method according to claim 2 or 3, characterized in that: Determining the transparency value of each pixel in the original image frame according to the color channel value of each pixel in the transparency configuration image includes: For each pixel in the original image frame, a pixel in the transparency configuration image that is at the same position as the pixel is determined as a reference pixel corresponding to the pixel; and an average of the color channel values ​​of the reference pixels is calculated as the transparency value of the pixel.

5. The method according to claim 1, characterized in that: The decoding process of the target animation file to obtain the transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect includes: Transmitting a first buffer object to a decoder; the first buffer object carries the information unit in the target animation file; By means of the decoder, the content carried by the information unit is decoded to obtain a decoding result; the decoding result is the transparency configuration information or the original image frame; The decoding result is transmitted to the second buffer object through the output callback function.

6. The method according to claim 1, characterized in that The target animation file is a video file encrypted using a target key; the method further comprises: Transmit key encryption factors to the video server; Receive a target encryption key; the target encryption key is obtained by the video server encrypting the target key using the key encryption factor.

7. The method according to claim 6, characterized in that Before decoding the target animation file to obtain the transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect, the method further includes: Decrypting the target encryption key using the key encryption factor to obtain the target key; The target animation file is decrypted using the target key.

8. An image processing device, characterized in that: The device comprises: An acquisition module is used to acquire a target animation file; the target animation file is a video file corresponding to a target animation effect; A decoding module, used for decoding the target animation file to obtain transparency configuration information included in the target animation file and a plurality of original image frames of the target animation effect; the transparency configuration information is used to indicate the transparency configuration of the original image frame; A transparency module, configured to perform transparency processing on each of the original image frames of the target animation effect based on the transparency configuration information, to obtain a transparent image frame corresponding to each of the original image frames; The playing module is used to play the transparent image frames corresponding to the original image frames.

9. A computer device, characterized in that: The computer device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the image processing method according to any one of claims 1 to 7 according to the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the image processing method according to any one of claims 1 to 7.

Images