Method and apparatus for special-effects generation, and device and storage medium

Abstract

According to the embodiments of the present disclosure, provided are a method and apparatus for special-effects generation, and a device and a storage medium. The method comprises: acquiring a special-effects image sequence, wherein the special-effects image sequence comprises one or more special-effects images for applying a special-effects style; executing redundancy detection on the special-effects image sequence, in order to detect at least one special-effects image and / or at least one image region having redundancy in a feature image sequence; on the basis of the redundancy in the at least one special-effects image and / or the at least one image region, executing encoding on image information of the special-effects image sequence to obtain an encoding result corresponding to the special-effects image sequence; and at least on the basis of the encoding result, generating a special-effects package corresponding to the special-effects style. Thus, the memory consumption and the computational load can be reduced, thereby improving the efficiency of generating a special-effects package corresponding to a special-effects style, and reducing the cost.

Description

Method, device, equipment and storage medium for special effect generation

[0001] This application claims priority to the Chinese invention patent application entitled “Method, device, apparatus and storage medium for special effects generation” filed on February 2, 2024, with application number 202410153962.4, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for generating special effects. Background Art

[0003] Currently, more and more applications are designed to provide various services to users. For example, users can publish, browse, and view various types of content within applications, including multimedia content such as videos, images, image collections, and sounds. Users can also apply special effects styles to images within applications. In some applications, the generation of special effects packages corresponding to certain special effects styles is performed on the server device. However, server-side devices are limited by power consumption and computing power, which can affect the user experience. Therefore, it is desirable to improve the user experience. Summary of the Invention

[0004] In a first aspect of the present disclosure, a method for generating special effects is provided. The method comprises: obtaining a special effects image sequence, the special effects image sequence comprising one or more special effects images for applying a special effects style; performing redundancy detection on the special effects image sequence to detect at least one special effects image and / or at least one image region that is redundant in a feature image sequence; encoding image information of the special effects image sequence based on the redundancy of the at least one special effects image and / or at least one image region to obtain an encoding result corresponding to the special effects image sequence; and generating a special effects package corresponding to the special effects style based at least on the encoding result.

[0005] In a second aspect of the present disclosure, a device for generating special effects is provided. The device includes: an acquisition module configured to acquire a special effect image sequence, the special effect image sequence including one or more special effect images for applying a special effect style; a detection module configured to perform redundancy detection on the special effect image sequence to detect at least one redundant special effect image and / or at least one image region in a feature image sequence; an encoding module configured to encode image information of the special effect image sequence based on the redundancy of the at least one special effect image and / or at least one image region to obtain an encoding result corresponding to the special effect image sequence; and a generation module configured to generate a special effect package corresponding to the special effect style based at least on the encoding result.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0008] In a fifth aspect of the present disclosure, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions, which, when executed by a device, cause the device to perform the method of the first aspect.

[0009] It should be understood that the content described in this summary section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG2 is a schematic diagram showing an example architecture for special effect generation according to some embodiments of the present disclosure;

[0013] 3A to 3C are schematic diagrams illustrating redundant special effect images in a special effect image sequence according to some embodiments of the present disclosure;

[0014] 4A to 4C are schematic diagrams showing redundant special effect images in a special effect image sequence according to other embodiments of the present disclosure;

[0015] 5A to 5D are schematic diagrams showing redundant image regions in a special effect image sequence according to some embodiments of the present disclosure;

[0016] FIG6 shows a flowchart of a process of special effect interaction according to some embodiments of the present disclosure;

[0017] FIG7 shows a block diagram of an apparatus for generating special effects according to some embodiments of the present disclosure; and

[0018] FIG8 illustrates a block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below.

[0021] Herein, unless explicitly stated otherwise, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0022] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0023] As used herein, the term "model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, corresponding outputs can be generated for given inputs. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", and these terms are used interchangeably in this article. A model can also include different types of processing units or networks.

[0024] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0025] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information, so that the user can independently choose whether to provide personal information to the electronic device, application, server or storage medium and other software or hardware that performs the operation of the technical solution of the present disclosure based on the prompt message.

[0026] As an optional but non-limiting implementation, in response to receiving a user's active request, a prompt message may be sent to the user, for example, in the form of a pop-up window, in which the prompt message may be presented in text form. Furthermore, the pop-up window may also include a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0028] FIG1 shows a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented.

[0029] In the example environment 100, an application 120 is installed in a terminal device 110. A user 140 can interact with the application 120 via the terminal device 110 and / or a device attached to the terminal device 110. In some embodiments, the terminal device 110 communicates with a server device 130 to provide services for the application 120.

[0030] In the environment 100 , a terminal device 105 communicates with a server device 130 , and an application 125 is installed in the terminal device 105 . A user 135 can interact with the application 125 via the terminal device 105 and / or an attached device of the terminal device 105 .

[0031] In some examples, application 125 may be a special effects editing application that can provide various services related to special effects content editing to user 135. In some examples, application 125 includes an editor. Server device 130 can use application 125 to provide services for application 120 installed on terminal device 110. For example, server device 130 can use application 125 to generate a special effects package corresponding to a special effects style and provide services to application 120 installed on terminal device 110.

[0032] In the example environment 100, the user 135 may be referred to as a special effects design user or designer. In some embodiments, the application 125 may be deployed locally on the terminal device of the user 135 and / or may be supported by the server device 130. For example, the terminal device of the user 135 may be running a client of the application 125, which may support the interaction between the user and the application 125 provided by the server device 130. In the case where the application 125 is running on the server device 130, the server device 130 may implement service provision for the client running in the terminal device based on the communication connection with the terminal device. The application 125 may present a corresponding page 145 to the user 135 based on the operation of the user 135 to output and / or receive information related to special effects editing to the user 135 and / or from the user 135.

[0033] In some embodiments, application 120 may be a content sharing application (e.g., a video application primarily focused on video sharing) that can provide various services related to media content items to user 140, including browsing, commenting, forwarding, creation (e.g., filming and / or editing), and publishing of content. In some embodiments, application 120 may also be any other appropriate application that can display media content items.

[0034] In the environment 100 of Figure 1 , if application 120 is active, terminal device 110 may present a page 150 of application 120. Page 150 may include various pages provided by application 120, such as a content presentation page, a content creation page, a content publishing page, a message page, a personal homepage, and so on. Application 120 may provide a content viewing function to view various types of content published in application 120. Through application 120, user 140 may also use a generated special effects package to create various types of content. User 140 may be referred to as a special effects user or special effects user.

[0035] In some embodiments, the terminal device 105 or 110 can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 105 or 110 can also support any type of interface for the user (such as a "wearable" circuit, etc.). The server device 130 can be various types of computing systems / servers that can provide computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0036] It should be understood that the structure and function of the various elements in the environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the present disclosure.

[0037] As briefly mentioned above, in the production process of special effects packages, some image materials are often used to achieve visual effects such as animation and interaction. The specific processing process can be divided into two steps: decoding and post-processing. Decoding refers to restoring image compression files such as png and jpg into ordinary pixel textures that can be directly read by the graphics processing unit (GPU) hardware according to the decompression algorithm and saving them in the system memory / video memory. Post-processing is to sample the pixel texture in the GPU and calculate the final pixel value according to the special effects design and display it on the screen. For example, each frame decompresses an image and displays it at a specified position on the screen. Several consecutive frames in this area can form an animation effect.

[0038] Currently, special effects packages are mostly used on mobile devices, which are constrained by power consumption and computing power. For example, excessive power consumption and intensive computation can lead to overheating, device frequency throttling, and frame drops, severely impacting the user experience. Correspondingly, the decoding and post-processing of special effects image material involve intensive computations, which significantly impacts heat generation due to memory read and transfer. Excessive memory usage also reduces stability, making it prone to crashes on some low-end machines.

[0039] By reducing the size / memory size of image materials, memory consumption and computational complexity can be reduced. Conventionally, by reducing the size, low-resolution materials can be used or the white area at the edges of the image can be reduced. For example, texture compression technology (Adaptive scalable texture compression, ASTC for short) can be used to reduce memory size. However, these technologies are single-point technologies and require a lot of manual participation in judgment. Although texture compression technology can reduce a certain amount of memory usage, it can process all data. However, a large amount of existing and future incremental special effects props image materials in the current business lack a unified automated processing process.

[0040] According to an embodiment of the present disclosure, an improved special effects generation scheme is proposed. According to the scheme of the embodiment of the present disclosure, a special effects image sequence including one or more special effects images for applying a special effects style is obtained. Redundancy detection is performed on the special effects image sequence to detect at least one special effects image and / or at least one image area that is redundant in the feature image sequence. Based on the redundancy of at least one special effects image and / or at least one image area, the image information of the special effects image sequence is encoded to obtain an encoding result corresponding to the special effects image sequence. Then, a special effects package corresponding to the special effects style is generated at least based on the encoding result. In this way, memory consumption and computational complexity can be reduced, thereby improving the efficiency of generating special effects packages corresponding to special effects styles and reducing costs.

[0041] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0042] FIG2 shows a schematic diagram of an example architecture 200 for special effect generation according to some embodiments of the present disclosure. For ease of discussion, these embodiments will be described with reference to the environment 100 of FIG1 .

[0043] As shown in Figure 2, a user 135 (e.g., a special effects designer) designs a special effects image sequence based on an editor 212 in a terminal device 105, and encodes the special effects image sequence to obtain a special effects package 213. The special effects package 213 can be provided to a server device 130 or a terminal device 110. The server device 130 or the terminal device 110 decodes and renders the special effects image sequence from the special effects package 213 based on a rendering engine and algorithm engine layer 215. The hardware 216 is used to support the editing and encoding of special effects image sequences, and the decoding and rendering of special effects image sequences. In some examples, the editor 212 and part of the hardware layer 216 can be included on the special effects design side, such as in the application 125 of the terminal device 105. The rendering engine and algorithm engine layer 215 and part of the hardware layer 216 can be included on the special effects application side, such as in the terminal device 110.

[0044] The material optimization module 214 is configured to perform the optimization operation for special effect generation proposed in the embodiments of the present disclosure. In some embodiments, the material optimization module 214 can be implemented in various appropriate devices.

[0045] In some embodiments, the material optimization module 214 can be implemented in the application 125. For example, the material optimization module 214 can be implemented together with the editor 212 at the terminal device 105, so that the designed special effects can be optimized and packaged during the special effects design process. Although shown as two separate modules in Figure 2, in some embodiments, the material optimization module 214 can be integrated into the editor 212 as part of the editor 212.

[0046] In some embodiments, the material optimization module 214 can be integrated into the server device 130. In this way, the special effects being designed can be optimized and packaged during the special effects design process of the terminal device 105, and the special effects packages that have been generated can also be optimized and repackaged.

[0047] In some embodiments, the material optimization module 214 may be implemented in the terminal device 105. In this way, before the special effect package is applied, the special effect package may be optimized and repackaged, and then the repackaged special effect package may be used to perform special effect rendering.

[0048] In the following, for the convenience of description, the case where the material optimization module is integrated into the server device 130 is taken as an example for explanation, and thus the material optimization process is described from the perspective of the server device 130.

[0049] For ease of understanding, the following first describes the overall process of the material optimization module 214 being configured to perform the optimization operation for special effect generation proposed in the embodiment of the present disclosure with reference to FIG1 and FIG2 .

[0050] If the material optimization module 214 is triggered or automatically optimized, the material optimization module 214 first performs redundancy detection. If the server device 130 finds that there is at least one redundant special effect image and / or at least one image area (for example, redundant image material) in the special effect image sequence based on the material optimization module 214, redundancy optimization processing is performed in the redundancy optimization module 218. For example, the redundancy optimization module 218 includes redundant area cropping 218-1, redundant area replacement 218-2, etc. After the server device 130 performs redundancy optimization on the special effect image sequence based on the material optimization module 214, the corresponding size and organizational form of the special effect image sequence will change. Therefore, in order not to change the prop effect designed by the user, the server device 130 performs corresponding reconfiguration of the rendering parameters based on the rendering parameter reconfiguration module 219. The server device 130 packages the reconfigured rendering parameters based on the repackaging module 220, and outputs the modified special effects package.

[0051] The following continues to describe in detail the solution for generating special effects according to an embodiment of the present disclosure with reference to FIG1 and FIG2 .

[0052] When performing material optimization, the server device 130 obtains a special effect image sequence, which includes one or more special effect images for applying a special effect style. Each special effect image is also called a special effect frame. In some examples, the server device 130 obtains a special effect image sequence including one or more special effect images for applying a special effect style from the terminal device 110.

[0053] In some embodiments, the server device 130 performs redundancy detection on the special effect image sequence to detect at least one redundant special effect image and at least one redundant image region in the feature image sequence. In some examples, the server device 130 performs redundancy detection on the special effect image sequence via the material optimization module 214. It will be appreciated that the material optimization module 214 can be applied to both static special effects (including a single special effect image) and dynamic special effects (e.g., special effect animations).

[0054] In some examples, the server device 130 can perform redundancy detection on the special effect image sequence through the material optimization module 214 to detect at least one redundant special effect image and at least one image region in the feature image sequence. The embodiments of the present disclosure are described below with reference to Figures 3A to 3C, Figures 4A to 4C, and Figures 5A to 5D, illustrating schematic diagrams corresponding to the detection of redundant special effect images and image regions in the present disclosure. However, this is merely exemplary and is not a limitation of the present disclosure.

[0055] Figures 3A to 3C illustrate schematic diagrams of redundant special effect images in a special effect image sequence according to some embodiments of the present disclosure. The server device 130, through the material optimization module 214, performs redundancy detection on the special effect image sequence to detect at least one redundant special effect image in the special effect image sequence. For example, schematic diagram 301 shown in Figure 3A, schematic diagram 302 shown in Figure 3B, and schematic diagram 303 shown in Figure 3C illustrate redundant special effect images in the special effect image sequence, such as black images in the special effect image sequence.

[0056] Figures 4A to 4C illustrate schematic diagrams of redundant special effect images in a special effect image sequence according to other embodiments of the present disclosure. The server device 130, through the material optimization module 214, performs redundancy detection on the special effect image sequence to detect at least one redundant special effect image in the special effect image sequence. For example, schematic diagram 401 shown in Figure 4A, schematic diagram 402 shown in Figure 4B, and schematic diagram 403 shown in Figure 4C illustrate redundant special effect images in the special effect image sequence, such as duplicate images in the special effect image sequence.

[0057] Figures 5A to 5D are schematic diagrams showing redundant image areas in a special effects image sequence according to some embodiments of the present disclosure. As shown in Figures 5A to 5D, the server-side device 130 performs redundancy detection on the special effects image sequence through the material optimization module 214 to detect at least one redundant image area in the special effects image sequence. For example, schematic diagram 501 shown in Figure 5A, schematic diagram 502 shown in Figure 5B, schematic diagram 503 shown in Figure 5C, and schematic diagram 504 shown in Figure 5D are redundant image areas in the special effects image sequence. For example, the effective picture in the special effects image sequence is only in the lower half. As shown in Figure 501, the effective picture is in interface 510. In Figure 502, the effective picture is in interface 520. In Figure 503, the effective picture is in interface 530. In Figure 504, the effective picture is in interface 540.

[0058] In some embodiments, the server device 130 encodes the image information of the special effect image sequence based on the redundancy of at least one special effect image and / or at least one image region. Furthermore, the server device 130 determines an encoding result corresponding to the special effect image sequence. Based on the encoding result, the server device 130 generates a special effect package corresponding to the special effect style.

[0059] In some embodiments, the server device 130 configures rendering parameters for each special effect image in the special effect image sequence based at least on the encoding result of the special effect image sequence. The server device 130 generates a special effect package corresponding to the special effect style based on the encoding result and the rendering parameters.

[0060] In some examples, because the server device 130 performs redundant optimization on the special effects image sequence based on the material optimization module 214, the size and organization of the special effects image sequence will change. Therefore, in the special effects package corresponding to the special effects style, the parameters must be reconfigured when referencing them. For example, the algorithm in the prop package references the pixel at the upper left corner of frame 1. After optimization, the pixel at this position disappears. In this case, the rendering parameters are modified so that the rendering starts sampling from the middle position, and the parameters at the upper half are required to be assigned a default value (0, 0, 0). The rendering parameters (reference position) are included in the configuration file of the special effects package.

[0061] For example, the material optimization module 214 can be integrated into the special effects editor 212. In this way, the generated special effects package (also called a prop package) can be optimized in real time during the process of the user 135 (special effects designer) designing the special effects. In other examples, the material optimization module 214 can be configured between the editor 212 and the rendering / algorithm engine layer 215. After the user 135 (special effects designer) designs the special effects, the special effects are packaged based on the existing scheme to obtain the corresponding special effects package. Then, the material optimization module 214 performs optimization on the basis of the existing special effects package, and provides the optimized special effects package for rendering and output by the rendering / algorithm engine layer 215. If it is a special effects package that has been generated, the material optimization module 214 can also perform batch detection and automatic optimization according to the above-mentioned optimization steps.

[0062] The following describes redundancy detection performed by the server device 130 on a special effect image sequence with reference to FIG2 . In some embodiments, the server device 130 performs monochrome detection on each special effect image in the special effect image sequence to detect whether there are image regions with the same color in each special effect image. In some examples, the server device 130 performs monochrome detection on each special effect image in the special effect image sequence based on the monochrome detection module 217-1 in the redundancy detection module 217 in the material optimization module 214.

[0063] In some embodiments, the server device 130 may perform monochrome detection on each special effect image in the special effect image sequence in the following manner: the server device 130 slides a first sliding window of a predetermined area shape across each special effect image in the special effect image sequence, and then determines whether the image blocks defined by the first window have the same color.

[0064] In one example, server device 130 detects the special effect image sequence using a first sliding window of a block-like region. For example, the first sliding window of the block-like region is 64*64, and determines whether there are images of the same color within the image blocks defined by the window. By sliding the first sliding window sequentially across the special effect image, image blocks of the same color detected in successive sliding windows are identified as first image regions with monochromatic redundancy.

[0065] In some embodiments, if a first image region having the same color as the first special effect image in the special effect image sequence is detected, the server device 130 encodes the first image region based on the single pixel value corresponding to the color of the first image region. In some embodiments, the encoding result refers to the range of the first image region in the first special effect image, and also refers to the encoding result of the single pixel value applied to the first image region.

[0066] In some examples, if a region of an image has the same color, all pixels in the region can be represented by a single pixel value c(r, g, b, a). If the entire special effect image is a single color value, the special effect image can be represented by only a single pixel value.

[0067] In some examples, if an image of the same color exists in the first image area, the server device 130 encodes a single pixel value in the first image area and records the range (L, T, R, B) of the single pixel value in the original special effect image. For example, if the first image area has the same color, the server device 130 encodes the single pixel value. Then, the server device 130 directly references the encoded single pixel value for other pixel values ​​contained in the image of the same color.

[0068] In some examples, the server device 130 performs inter-frame duplication detection on each special effect image in the special effect image sequence based on the inter-frame duplication detection module 217 - 2 in the redundancy detection module 217 in the material optimization module 214 .

[0069] In some examples, the server device 130 may perform inter-frame duplication detection on each special effect image in the special effect image sequence in the following manner. In some embodiments, the server device 130 determines whether corresponding pixel positions of at least two consecutive special effect images in the special effect image sequence have the same pixel value. The server device 130 then determines that redundancy exists between the at least two consecutive special effect images based on the determination that the corresponding pixel positions of the at least two consecutive special effect images have the same pixel value.

[0070] In some embodiments, the server device 130 slides a second sliding window through the special effect image sequence to determine whether at least two consecutive special effect images defined in the second sliding window are identical. For example, for scenes such as sequential frame animation, the server device 130 uses the second sliding window to compare pixel values ​​of preceding and following frames to determine whether at least two consecutive special effect images defined in the second sliding window are identical. By sliding the second sliding window sequentially, identical special effect images detected in consecutive sliding windows are determined to be a group of special effect images with inter-frame redundancy.

[0071] In some embodiments, if the server device 130 determines that at least two consecutive special effect images in the special effect image sequence are redundant, encoding is performed on a single special effect image from the at least two consecutive special effect images. The encoding result may indicate the positions of the at least two consecutive special effect images in the special effect image sequence, and may also indicate that the encoding result of the single special effect image is applied to decode the at least two consecutive special effect images.

[0072] In some examples, if the server device 130 determines that at least two consecutive special effect images are redundant, only a single special effect image from the at least two consecutive special effect images is retained and the frame information of the image is recorded. For example, for other special effect images that have the same color as the image corresponding to the single pixel value being encoded in the at least two consecutive special effect images defined by the second sliding window, the server device 130 directly references the encoded single pixel value.

[0073] For example, if there are at least two consecutive special effect images in the special effect image sequence that are redundant, only a single special effect image from the at least two consecutive special effect images is retained, and during decoding, only the frame information corresponding to the single special effect image is decoded.

[0074] The embodiments of the present disclosure can reduce memory consumption and computational complexity, thereby improving the efficiency of generating special effects packages corresponding to special effects styles and reducing costs.

[0075] Example Process

[0076] FIG6 shows a flow chart of a process 600 of special effect interaction according to some embodiments of the present disclosure. The process 600 may be implemented at the server device 130. The process 600 is described below with reference to FIG1.

[0077] As shown in FIG6 , in block 610 , the server device 130 obtains a special effect image sequence, where the special effect image sequence includes one or more special effect images for applying a special effect style.

[0078] In block 620 , the server device 130 performs redundancy detection on the special effect image sequence to detect at least one special effect image and / or at least one image region that is redundant in the feature image sequence.

[0079] In block 630 , the server device 130 encodes the image information of the special effect image sequence based on the redundancy of at least one special effect image and / or at least one image region to obtain an encoding result corresponding to the special effect image sequence.

[0080] In block 640 , the server device 130 generates a special effect package corresponding to the special effect style based at least on the encoding result.

[0081] In some embodiments, performing redundancy detection on the special effect image sequence includes performing monochrome detection on each special effect image in the special effect image sequence to detect whether there is an image region with the same color in each special effect image.

[0082] In some embodiments, performing monochrome detection on each special effect image in the special effect image sequence includes: sliding a first sliding window with a predetermined area shape in each special effect image in the special effect image sequence, and determining whether an image block defined by the first sliding window has the same color.

[0083] In some embodiments, encoding image information of the special effect image sequence includes: if a first image region having the same color is detected from a first special effect image in the special effect image sequence, encoding the first image region based on a single pixel value corresponding to the color of the first image region, wherein the encoding result indicates a range of the first image region in the first special effect image and further indicates that the encoding result of the single pixel value is applied to the first image region.

[0084] In some embodiments, performing redundancy detection on the special effect image sequence includes: determining whether corresponding pixel positions of at least two consecutive special effect images in the special effect image sequence have the same pixel value; and determining that redundancy exists between the at least two consecutive special effect images based on determining that the corresponding pixel positions of the at least two consecutive special effect images have the same pixel value.

[0085] In some embodiments, determining whether at least two consecutive special effect images in the special effect image sequence are identical includes sliding a second sliding window in the special effect image sequence to determine whether at least two consecutive special effect images defined in the second sliding window are identical.

[0086] In some embodiments, encoding image information of a special effect image sequence includes: if it is determined that redundancy exists between at least two consecutive special effect images, encoding a single special effect image from the at least two consecutive special effect images, wherein the encoding result indicates positions of the at least two consecutive special effect images in the special effect image sequence and further indicates that the encoding result of the single special effect image is applied to decoding the at least two consecutive special effect images.

[0087] In some embodiments, generating a special effects package corresponding to a special effects style based at least on the encoding results includes: configuring rendering parameters for each special effects image in a special effects image sequence based at least on the encoding results; and generating a special effects package including at least the encoding results and the rendering parameters.

[0088] Example devices and equipment

[0089] 7 shows a schematic structural block diagram of an apparatus 700 for generating special effects according to certain embodiments of the present disclosure. Apparatus 700 may be implemented as or included in server device 130. Each module / component in apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0090] As shown in the figure, the apparatus 700 includes an acquisition module 710 configured to acquire a special effect image sequence, where the special effect image sequence includes one or more special effect images for applying a special effect style.

[0091] The apparatus 700 further includes a detection module 720 configured to perform redundancy detection on the special effect image sequence to detect at least one special effect image and / or at least one image region that is redundant in the feature image sequence.

[0092] The apparatus 700 further includes an encoding module 730 configured to encode image information of a special effect image sequence based on redundancy of at least one special effect image and / or at least one image region to obtain an encoding result corresponding to the special effect image sequence.

[0093] The apparatus 700 further includes a generating module 740 configured to generate a special effect package corresponding to the special effect style based at least on the encoding result.

[0094] In some embodiments, the detection module 720 is further configured to perform monochrome detection on each special effect image in the special effect image sequence to detect whether there is an image region with the same color in each special effect image.

[0095] In some embodiments, the detection module 720 is further configured to slide a first sliding window of a predetermined area shape in each special effect image in the special effect image sequence to determine whether the image blocks defined by the first sliding window have the same color.

[0096] In some embodiments, the encoding module 730 is further configured to, if a first image area having the same color is detected from a first special effect image in the special effect image sequence, perform encoding on the first image area based on a single pixel value corresponding to the color of the first image area, wherein the encoding result indicates a range of the first image area in the first special effect image, and further indicates that the encoding result of the single pixel value is applied to the first image area.

[0097] In some embodiments, the detection module 720 is further configured to determine whether corresponding pixel positions of at least two consecutive special effect images in the special effect image sequence have the same pixel value; and determine that redundancy exists in the at least two consecutive special effect images based on determining that the corresponding pixel positions of the at least two consecutive special effect images have the same pixel value.

[0098] In some embodiments, the detection module 720 further includes a determination module configured to slide the special effect image sequence according to the second sliding window to determine whether at least two consecutive special effect images defined in the second sliding window are identical.

[0099] In some embodiments, the encoding module 730 is further configured to, if it is determined that redundancy exists between the at least two consecutive special effect images, perform encoding on a single special effect image among the at least two consecutive special effect images, wherein the encoding result indicates positions of the at least two consecutive special effect images in the special effect image sequence, and further indicates that the encoding result of the single special effect image is applied to decoding the at least two consecutive special effect images.

[0100] In some embodiments, the generation module 740 is further configured to configure rendering parameters for each special effect image in the special effect image sequence based at least on the encoding result; and generate a special effect package including at least the encoding result and the rendering parameters.

[0101] FIG8 illustrates a block diagram of an electronic device 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 800 shown in FIG8 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 800 shown in FIG8 may be used to implement the server device 130 of FIG1 or the apparatus 700 of FIG7 .

[0102] As shown in FIG8 , electronic device 800 is a general-purpose electronic device. Components of electronic device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 800.

[0103] The electronic device 800 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data (e.g., training data for training) and can be accessed within the electronic device 800.

[0104] The electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG8 , a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0105] The communication unit 840 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 800 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 800 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0106] The input device 850 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 860 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 800 may also communicate with one or more external devices (not shown) via the communication unit 840 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 800, or with any device that allows the electronic device 800 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0107] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0108] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0109] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0110] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0111] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0112] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating special effects, comprising: Acquire a special effect image sequence, where the special effect image sequence includes one or more special effect images for applying a special effect style; performing redundancy detection on the special effect image sequence to detect at least one special effect image and / or at least one image region that is redundant in the feature image sequence; Based on the redundancy of the at least one special effect image and / or the at least one image region, encoding the image information of the special effect image sequence to obtain an encoding result corresponding to the special effect image sequence; as well as A special effects package corresponding to the special effects style is generated at least based on the encoding result.

2. The method according to claim 1, wherein performing redundancy detection on the special effect image sequence comprises: A monochrome detection is performed on each special effect image in the special effect image sequence to detect whether there is an image region with the same color in each special effect image.

3. The method according to claim 1 or 2, wherein performing monochrome detection on each special effect image in the special effect image sequence comprises: A first sliding window with a predetermined area shape slides in each special effect image in the special effect image sequence to determine whether image blocks defined by the first sliding window have the same color.

4. The method according to claim 1 or 2, wherein encoding the image information of the special effect image sequence comprises: If a first image region having the same color is detected from a first special effect image in the special effect image sequence, encoding is performed on the first image region based on a single pixel value corresponding to the color of the first image region, The encoding result indicates a range of the first image area in the first special effect image, and further indicates that the encoding result of the single pixel value is applied to the first image area.

5. The method according to any one of claims 1 to 4, wherein performing redundancy detection on the special effect image sequence comprises: determining whether corresponding pixel positions of at least two consecutive special effect images in the special effect image sequence have the same pixel value; as well as Based on determining that corresponding pixel positions of at least two consecutive special effect images have the same pixel value, it is determined that redundancy exists between the at least two consecutive special effect images.

6. The method according to claim 5, wherein determining whether at least two consecutive special effect images in the special effect image sequence are the same comprises: A second sliding window is slid in the special effect image sequence to determine whether at least two consecutive special effect images defined in the second sliding window are the same.

7. The method according to claim 5, wherein encoding the image information of the special effect image sequence comprises: If it is determined that the at least two consecutive special effect images have redundancy, encoding is performed on a single special effect image in the at least two consecutive special effect images. The encoding result indicates positions of the at least two consecutive special effect images in the special effect image sequence, and further indicates that the encoding result of the single special effect image is applied to decoding the at least two consecutive special effect images.

8. The method according to any one of claims 1 to 7, wherein generating a special effect package corresponding to the special effect style based at least on the encoding result comprises: configuring rendering parameters for each special effect image in the special effect image sequence based at least on the encoding result; as well as Generate a special effects package that at least includes the encoding result and the rendering parameters.

9. A device for generating special effects, comprising: an acquisition module configured to acquire a special effect image sequence, wherein the special effect image sequence includes one or more special effect images for applying a special effect style; a detection module configured to perform redundancy detection on the special effect image sequence to detect at least one special effect image and / or at least one image region that is redundant in the feature image sequence; an encoding module configured to encode image information of the special effect image sequence based on redundancy of the at least one special effect image and / or the at least one image region, to obtain an encoding result corresponding to the special effect image sequence; as well as A generating module is configured to generate a special effect package corresponding to the special effect style based at least on the encoding result.

10. An electronic device comprising: at least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 8 when executed by the at least one processing unit.

11. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 8.

Images