Special effect adding method, apparatus, computer device and medium
By determining the data object type and selecting the target effect template, and by adopting parallel processing and collaborative work, the problems of time-consuming, labor-intensive, and power-consuming effect addition in existing technologies are solved, and efficient and flexible effect addition is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2021-06-30
- Publication Date
- 2026-07-03
AI Technical Summary
Existing technologies involve time-consuming and labor-intensive special effects processing with high CPU power consumption, making it difficult to efficiently coordinate the CPU, GPU, and DSP to add special effects.
By determining the data object type to be added with the special effects, selecting the target special effects template, and establishing the corresponding special effects adding process, the special effects are added in a parallel processing and collaborative manner.
It improves the efficiency of adding special effects, reduces the power consumption of the device, and achieves flexibility and high efficiency in adding special effects.
Smart Images
Figure CN115546352B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of special effects processing technology, and in particular to methods, apparatus, computer devices and media for adding special effects. Background Technology
[0002] Special effects processing technology adds popular elements to the original audio or video, making the video or audio richer and more realistic in terms of visuals or sound. To improve the efficiency of special effects processing and reduce the power consumption of devices, the encoder and decoder need to work in conjunction with CPU (Central Processing Unit), GPU (Graphics Processing Unit), and DSP (Digital Signal Processing).
[0003] In related technologies, adding special effects to video, image, or audio objects is usually done manually based on user needs, adding effects to specific parts of the object to be added. For example, adding effects to specific actions in a video or specific sounds in an audio file. Adding effects is time-consuming and laborious, and also results in high power consumption of the central processing unit. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, computer device, and medium for adding special effects.
[0005] According to a first aspect of the present disclosure, a method for adding special effects is provided, the method comprising:
[0006] Determine the target data type of the data object to which the effect will be added;
[0007] Based on the target data type, the target effect template is determined from the preset effect templates corresponding to different data types;
[0008] Establish a process for adding special effects corresponding to the target data type;
[0009] Run the effect-adding process and call the effect plugin that stores the target effect template to add effects to the data object.
[0010] Optionally, there are multiple target effect templates, and the process of creating effect additions corresponding to the target data type includes:
[0011] Based on the number of target effect templates and the relationships between them, determine the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process.
[0012] Based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
[0013] Optionally, the association relationship includes: a first association relationship, which is used to restrict the order of adding special effects between target special effects templates, and the same subprocess is used to add special effects to target special effects templates with the first association relationship.
[0014] Optionally, the step of running the effect-adding process and calling the effect plugin storing the target effect template to add effects to the data object includes:
[0015] Each of the subprocesses in the effect addition process is run in parallel so that each subprocess can call the effect plugin that stores the target effect template corresponding to the subprocess in parallel to add effects to the data object.
[0016] Optionally, the data object is a plurality of data objects to be concatenated, and the step of determining the target effect template from a preset set of effect templates corresponding to different data types according to the target data type includes:
[0017] Determine the concatenation order among the multiple data objects;
[0018] For each data object, based on the target data type of the data object and the target data type of the next data object in the splicing order, a target transition effect template corresponding to the data object is determined from the preset transition effect templates. The target effect template includes the target transition effect template.
[0019] The step of calling the effects plugin storing the target effects template to add effects to the data object includes:
[0020] According to the splicing order, the special effects plugins that store the target transition effect templates corresponding to each of the data objects are called in sequence to add special effects to the data objects.
[0021] Optionally, the step of running the effect-adding process and calling the effect plugin storing the target effect template to add effects to the data object includes:
[0022] Determine the source of the data object;
[0023] When the source of the data object is a camera captured by the terminal, the target processor for adding special effects to the data object is determined based on the current memory usage of the terminal's central processing unit and the current cache state of the graphics processor.
[0024] The effects-adding process runs on the target processor, and the effects plugin storing the target effects template is invoked to add effects to the data object.
[0025] Optionally, determining the target special effects template from a preset set of special effects templates corresponding to different data types based on the target data type includes:
[0026] Determine the target scene to which the data object belongs from the preset scene classification;
[0027] From a plurality of preset special effect templates corresponding to the target scene, the special effect template corresponding to the target data type of the data object is selected as a candidate special effect template;
[0028] The target effect template is determined from the candidate effect templates.
[0029] Optionally, determining the target effect template from the candidate effect templates includes:
[0030] Display the candidate effect templates;
[0031] When a user selects any effect template from the candidate effect templates, the selected candidate effect template is determined as the target effect template.
[0032] According to a second aspect of the present disclosure, a special effects adding device is provided, the device comprising:
[0033] The first determination module is configured to determine the target data type of the data object to which the special effect is to be added;
[0034] The second determining module is configured to determine the target special effect template from a preset set of special effect templates corresponding to different data types based on the target data type.
[0035] The module is configured to create a special effects addition process corresponding to the target data type.
[0036] The calling module is configured to run the effect adding process and call the effect plugin storing the target effect template to add effects to the data object.
[0037] Optionally, there are multiple target effect templates, and the creation module is configured to determine the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process based on the number of target effect templates and the association relationships between them; and
[0038] Based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
[0039] Optionally, the association relationship includes: a first association relationship, which is used to restrict the order of adding special effects between target special effects templates, and the same subprocess is used to add special effects to target special effects templates with the first association relationship.
[0040] Optionally, the calling module is configured to run each of the sub-processes in the effect adding process in parallel, so that each sub-process can call the effect plugin storing the target effect template corresponding to the sub-process in parallel to add effects to the data object.
[0041] Optionally, the data objects are multiple data objects to be concatenated, and the second determining module is configured to determine the concatenation order among the multiple data objects; and,
[0042] For each data object, based on the target data type of the data object and the target data type of the next data object in the splicing order, a target transition effect template corresponding to the data object is determined from the preset transition effect templates. The target effect template includes the target transition effect template.
[0043] The calling module is configured to sequentially call the effects plugin that stores the target transition effect template corresponding to each of the data objects according to the splicing order, and add effects to the data object.
[0044] Optionally, the calling module is configured to determine the source of the data object;
[0045] When the source of the data object is a camera captured by the terminal, the target processor for adding special effects to the data object is determined based on the current memory usage of the terminal's central processing unit and the current cache state of the graphics processor.
[0046] The effects-adding process runs on the target processor, and the effects plugin storing the target effects template is invoked to add effects to the data object.
[0047] Optionally, the second determining module is configured to determine the target scene to which the data object belongs from a preset scene classification;
[0048] From a plurality of preset special effect templates corresponding to the target scene, the special effect template corresponding to the target data type of the data object is selected as a candidate special effect template;
[0049] The target effect template is determined from the candidate effect templates.
[0050] Optionally, the second determining module is configured to display the candidate effect templates;
[0051] When a user selects any effect template from the candidate effect templates, the selected candidate effect template is determined as the target effect template.
[0052] According to a third aspect of the present disclosure, a computer apparatus is provided, comprising:
[0053] processor;
[0054] Memory used to store processor-executable instructions;
[0055] The processor is configured as follows:
[0056] Determine the target data type of the data object to which the effect will be added;
[0057] Based on the target data type, the target effect template is determined from the preset effect templates corresponding to different data types;
[0058] Establish a process for adding special effects corresponding to the target data type;
[0059] Run the effect-adding process and call the effect plugin that stores the target effect template to add effects to the data object.
[0060] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the method described in any one of the first aspects.
[0061] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: determining the target data type of the data object to which special effects are to be added; determining the target special effects template from a preset set of special effects templates corresponding to different data types based on the target data type; establishing a special effects adding process corresponding to the target data type; running the special effects adding process; and calling the special effects plugin storing the target special effects template to add special effects to the data object. Thus, by determining the target special effects template from a preset set of special effects templates corresponding to different data types, and calling the special effects plugin storing the target special effects template to add special effects to the data object, the efficiency of special effects addition can be improved.
[0062] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0063] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0064] Figure 1 This is a flowchart illustrating a method for adding special effects according to an exemplary embodiment.
[0065] Figure 2 This is a schematic diagram illustrating the addition of a special effect according to an exemplary embodiment.
[0066] Figure 3 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart for step S13.
[0067] Figure 4 This is a schematic diagram illustrating the addition of a parallel effect according to an exemplary embodiment.
[0068] Figure 5 This is a flowchart illustrating another method for adding special effects according to an exemplary embodiment.
[0069] Figure 6 This is a schematic diagram illustrating the addition of a splicing effect according to an exemplary embodiment.
[0070] Figure 7 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart for step S14.
[0071] Figure 8 This is a schematic diagram illustrating another effect addition according to an exemplary embodiment.
[0072] Figure 9 This is a schematic diagram illustrating a special effects addition system according to an exemplary embodiment.
[0073] Figure 10 This is a schematic diagram illustrating another effect addition according to an exemplary embodiment.
[0074] Figure 11 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart for step S12.
[0075] Figure 12 This is a block diagram illustrating a special effects adding device according to an exemplary embodiment.
[0076] Figure 13 This is a block diagram illustrating an apparatus for adding special effects according to an exemplary embodiment. Detailed Implementation
[0077] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0078] Figure 1 This is a flowchart illustrating a method for adding special effects according to an exemplary embodiment, such as... Figure 1 As shown, this method can be used in terminals, such as mobile phones, tablets, personal terminals, and other terminal devices, or in servers. The method includes the following steps.
[0079] In step S11, the target data type of the data object to which the effect is to be added is determined.
[0080] In step S12, the target effect template is determined from the preset effect templates corresponding to different data types, based on the target data type.
[0081] In step S13, a process for adding special effects corresponding to the target data type is established.
[0082] In step S14, the special effects addition process is run, and the special effects plugin storing the target special effects template is called to add special effects to the data object.
[0083] In practice, the data objects to which special effects are to be added can be not only users' personal videos, images, and audio on the terminal, but also educational, film and television, cultural, and commercial videos, images, and audio. Furthermore, the data objects to which special effects are to be added can be objects captured by the terminal, or data objects uploaded locally via, for example, Bluetooth, near-field communication, or data networks.
[0084] Among them, different data types in the data object to which the special effect is to be added can be distinguished by different storage formats, and then the target data type of the data object to which the special effect is to be added can be determined according to the storage format. Alternatively, they can be distinguished by adding data type identifiers in specific positions, and then the target data type of the data object to which the special effect is to be added can be determined according to the data type identifiers.
[0085] Furthermore, for video files in the target data type, image data containing luminance and chrominance signals can be decoded using either software or hardware decoding. Alternatively, the hardware-decoded data can be rendered into Android's core rendering component (surface texture) to obtain a video data object. For audio files in the target data type, audio data objects can be obtained from the audio files using either software or hardware decoding. For image files in the target data type, graphic data objects can be obtained using open-source tools such as PNG or JPEG formats (libpng, libjpeg).
[0086] Furthermore, a process for adding video effects is established for video data objects, an process for adding audio effects is established for audio data objects, and a process for adding graphic effects is established for graphic data objects.
[0087] The special effects plugins are created by breaking down special effects functionality into its smallest unit and then encapsulating these sub-elements. Each plugin contains only one of these smallest units, allowing for plugin reuse. Furthermore, new effects can be added using plugins that are the smallest units. These plugins can also be flexibly combined with other plugins, significantly increasing the flexibility of effect addition and greatly shortening project development time for adding effects.
[0088] See Figure 2 The diagram shown illustrates the process of adding effects. For the data object V1 to which effects are to be added, the corresponding type of effect adding process is run. The effect plugins storing the target effect templates are called sequentially. For example, the first effect plugin storing effect template A is called to add effects to the data object V1. After the effect is added, the second effect plugin storing effect template B is called to add effects to the data object V1, resulting in the data object with added effects.
[0089] Furthermore, the data object after adding effects can be previewed in real time through the core rendering component window (GLSurfaceView), allowing users to preview the effect in real time. For example, for Android terminals, 2D and 3D image data can be rendered based on the display open graphics library, and effects can be added internally through an independent rendering sub-thread, so that adding effects does not affect the main thread operation.
[0090] Furthermore, in response to a save command, the data object with added effects is encoded and compressed in a specific format for saving. For example, in response to a manual save command generated after the user clicks save, the video data object with added effects is encoded and compressed in MP4 format for saving. Another example is that after a preset duration following preview display, an automatic save command is generated, and the video data object with added effects is encoded and compressed in MP4 format for saving. Yet another example is that in response to a manual save command generated after the user clicks save, the audio data object with added effects is encoded and compressed in MP3 format for saving.
[0091] Optionally, encoding and synthesis can be performed through software encoding or hardware encoding. Software encoding uses an H264 encoding tool to encode and compress the video data objects of luminance and chrominance signals into an H264 raw bitstream and write it into an encapsulated MP4 container. Hardware encoding, on the other hand, inputs the rendered video data objects output by the open graphics library to a hardware encoder for encoding, compression, and synthesis.
[0092] The above technical solution can improve the efficiency of adding effects by determining the target data type of the data object to which the effect is to be added; determining the target effect template from a set of preset effect templates corresponding to different data types based on the target data type; establishing an effect adding process corresponding to the target data type; running the effect adding process; and calling the effect plugin storing the target effect template to add effects to the data object.
[0093] Based on the above embodiments, there are multiple target effect templates. Figure 3 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart of step S13 shows that the process of establishing the special effects addition process corresponding to the target data type includes the following steps.
[0094] In step S131, based on the number of target effect templates and the relationship between them, the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process are determined.
[0095] In step S132, based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
[0096] Optionally, the association relationship includes: a first association relationship, which is used to restrict the order of adding special effects between target special effects templates, and the same subprocess is used to add special effects to target special effects templates with the first association relationship.
[0097] The association relationship also includes a second association relationship, which indicates that the addition of special effects to target special effects templates cannot be performed simultaneously. For target special effects templates with the second association relationship, different sub-processes are used to add special effects. For target special effects templates without an association relationship, the same sub-process can be used to add special effects to them, or different sub-processes can be used, depending on the sub-process with the largest number of target special effects templates.
[0098] For example, there are 4 target effect templates. The first target effect template has a first association with the second target effect template, the first target effect template has a second association with the third target effect template, and the fourth target effect template has no association with the first, second, and third target effect templates.
[0099] Furthermore, the effects of the first target effect template and the second target effect template are added through the first sub-process, and the effects of the third target effect template and the fourth target effect template are added through the second sub-process.
[0100] The above technical solution allows for the addition of effects to multiple target effect templates in multiple sub-processes, which can improve the efficiency of effect addition and reduce device power consumption.
[0101] Based on the above embodiments, in step S14, running the special effects adding process and calling the special effects plugin storing the target special effects template to add special effects to the data object includes:
[0102] The effects are added in parallel to each of the child processes, so that each child process can call the effects plugin that stores the target effects template of the corresponding child process to add effects to the data object.
[0103] See Figure 4 The diagram shown illustrates the parallel effects addition process. The first sub-process and the second sub-process run in parallel. If the time taken to add effects to each target effect template is the same, the first effect plugin storing target effect template AI1 and the third effect plugin storing target effect template AI3 are called in parallel to add effects to the data object. After the effects are added, the second effect plugin storing target effect template AI2 and the fourth effect plugin storing target effect template AI4 are called in parallel to add effects to the data object.
[0104] The above technical solution can run each subprocess in the special effects addition process in parallel, and add special effects to data objects at the same time. In particular, it can effectively improve the efficiency of special effects addition and reduce the power consumption of the device when adding AI special effects.
[0105] Based on the above embodiments, the data object is a plurality of data objects to be concatenated. Figure 5 This is a flowchart illustrating another method for adding special effects according to an exemplary embodiment. In step S12, determining the target special effect template from preset special effect templates corresponding to different data types according to the target data type includes the following steps.
[0106] In step S121, the splicing order between multiple data objects is determined.
[0107] In step S122, for each data object, based on the target data type of the data object and the target data type of the next data object in the splicing order, a target transition effect template corresponding to the data object is determined from the preset transition effect templates. The target effect template includes the target transition effect template.
[0108] In step S14, the step of calling the special effects plugin storing the target special effects template to add special effects to the data object includes the following steps.
[0109] In step S141, according to the splicing order, the special effects plugins that store the target transition effect templates corresponding to each data object are called in sequence to add special effects to the data object.
[0110] For specific implementation, see Figure 6 The diagram showing the addition of special effects indicates that, for data objects V1, V2, I1, I2, and A1, the splicing order of the aforementioned data objects is determined as follows: data object V1 is spliced with data object V2, data object I1 is spliced after data object V1 and data object V2 are spliced, data object I2 is spliced after data object I1 is spliced, and data object A1 is spliced after data object I2 is spliced.
[0111] Furthermore, based on the target data types of data object V1 and data object V2, the target normal transition effect and the first target transition effect template for data object V1 are determined from the preset transition effect templates; based on the target data types of data object V2 and data object I1, the target normal transition effect and the second target transition effect template for data object V2 are determined from the preset transition effect templates; based on the target data types of data object I1 and data object I2, the target normal transition effect and the third target transition effect template for data object I1 are determined from the preset transition effect templates; based on the target data types of data object I2 and data object A1, the target normal transition effect and the fourth target transition effect template for data object I2 are determined from the preset transition effect templates; and based on the target data type of data object A1, the target normal transition effect for data object A1 is determined from the preset transition effect templates.
[0112] Furthermore, the special effects addition process is run, and the special effects plugin that stores the target ordinary special effects template of the data object is called to add ordinary special effects to the data object. After the ordinary special effects of the data object are added, the special effects plugin that stores the target transition special effects template is called to add transition special effects to the data object.
[0113] For example, the effects plugin that stores the first target effect template of data object V1 is called to add normal effects to data object V1. After the normal effects of data object V1 are added, the first transition effect plugin that stores the first target transition effect template is called to add transition effects to data object V1. And so on, the process of adding other effects will not be described in detail.
[0114] This allows not only the concatenation of multiple data objects, but also the concatenation of data objects of different data types.
[0115] It is worth noting that multiple data objects to be added with special effects can be the same data object. For example, for multiple data objects V1 to be added with special effects, multiple data objects V1 can be spliced together using different target transition effect templates.
[0116] Based on the above embodiments, Figure 7 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart of step S14 shows that in step S14, the special effects adding process is run, and the special effects plugin storing the target special effects template is called to add special effects to the data object, including the following steps.
[0117] In step S1401, the source of the data object is determined.
[0118] In step S1402, when the source of the data object is a camera captured by the terminal, the target processor for adding special effects to the data object is determined based on the current memory usage of the terminal's central processing unit and the current cache state of the graphics processor.
[0119] In step S1403, an effects-adding process is run on the target processor, and an effects plugin storing the target effects template is invoked to add effects to the data object.
[0120] In practice, the data object is determined to originate from either a camera captured on the terminal or uploaded locally. Furthermore, in the case of camera capture, to improve the efficiency and speed of adding effects, if the CPU's current memory usage exceeds a preset threshold, the target processor for adding effects to the data object can be determined based on the current cache state of the graphics processor (GPU). (Refer to...) Figure 8 As shown, effects are added to data objects by calling an effects plugin that stores the target effects template through the graphics processor. If the current memory usage of the CPU does not exceed a preset threshold, the CPU is determined as the target processor for adding effects to the data objects based on the current cache state of the graphics processor.
[0121] In one implementation, the graphics processor employs a double buffering mechanism. For example, while the first buffer area is running, the effects are added to the data objects to which effects are to be added through the second buffer area. This allows the graphics processor's graphics processing data and graphics rendering to run in a coordinated and efficient manner, ensuring a smoother picture during preview playback.
[0122] See Figure 9 The diagram shown illustrates the special effects addition module. The input data object for adding special effects can be captured by the terminal camera or uploaded locally as images, videos, and audio. The special effects addition process runs on the central processing unit or the graphics processing unit, which then calls the special effects plugin that stores the target special effects template to add special effects to the data object, resulting in an output preview object or a saved object.
[0123] In one implementation, if the graphics processor is currently in a cached state, based on Android's native link buffer, a graphBuffer is created, and an imageEGL is obtained from the graphBuffer. The image's EGL object is then bound to the core rendering component corresponding to the open graphics library. In this way, the central processing unit can directly obtain data from the graphics processor, thereby shortening the time spent on data delivery between the central processing unit and the graphics processor and improving the speed and efficiency of adding effects.
[0124] Among them, reference Figure 10 As shown, the CPU uses NEON assembly. NEON is a 128-bit SIMD (Single Instruction Multiple Data) extension architecture for ARM Cortex-A series processors. Through improvements to assembly instructions, the performance of adding special effects can be improved by 4.8 times. For example, the CPU first runs the corresponding special effects adding process. The CPU calls the first special effects plugin, which stores special effects template A, to add special effects to data object V1. Based on ION MEMORY, the GPU runs the corresponding special effects adding process and calls the second special effects plugin, which stores special effects template B, to add special effects to the data object V1 after adding the effects. The data object with added effects is then output.
[0125] Furthermore, the special effects added for Android all run on Qualcomm's digital signal processing (DSP). Qualcomm's HVX (Hexagon Vector Extensions) is a typical feature of the Hexagon 680 DSP, capable of handling computational loads in image processing applications such as virtual reality, augmented reality, image processing, video processing, and computer vision. This achieves greater efficiency and lower power consumption; compared to NEON optimization, performance can be improved by 1 to 3 times, while power consumption can be reduced to 1 / 18 to 1 / 4 of the original.
[0126] Based on the above embodiments, Figure 11 This is an implementation illustrated according to an exemplary embodiment. Figure 1 The flowchart of step S12 shows that, in step S12, determining the target special effect template from the preset special effect templates corresponding to different data types based on the target data type includes the following steps.
[0127] In step S1201, the target scene to which the data object belongs is determined from the preset scene classification.
[0128] In step S1202, from multiple preset effect templates corresponding to the target scene, the effect template corresponding to the target data type of the data object is selected as the candidate effect template.
[0129] In step S1203, the target effect template is determined from the candidate effect templates.
[0130] In practical implementation, semantic retrieval, image feature retrieval, and other methods can be used to determine the target scene to which the data object belongs from the preset scene categories. For example, through image feature retrieval, if the captured image contains people, the target scene is determined to be a portrait scene. From multiple preset effect templates for portrait scenes, the effect template corresponding to the image data type is selected as a candidate effect template, and the target effect template is determined from the candidate effect templates.
[0131] For example, if the shooting time determines the shooting scene to be evening, then the target scene to which the data object belongs is determined to be a night shooting scene from the preset scene categories. From the multiple preset special effects templates for night shooting scenes, the special effects template corresponding to the target data type is selected as a candidate special effects template, and the target special effects template is determined from the candidate special effects templates.
[0132] Based on the above embodiments, in step S1203, determining the target effect template from the candidate effect templates includes the following steps.
[0133] Showcase candidate special effects templates;
[0134] When a user selects any effect template from the candidate effect templates, the selected candidate effect template is determined as the target effect template.
[0135] In practice, candidate effect templates can be displayed on the device's user interface, allowing users to preview the effects corresponding to each template. Then, based on the user's selection—for example, a motor selection action within any effect template—the selected candidate effect template is designated as the target effect template.
[0136] In one implementation, scene-related effect template groups within a preset scene category can be used. Specifically, when a candidate effect template corresponding to a target scene is selected, all effect templates associated with that group can be used as the target effect template. Alternatively, all effect templates associated with a single effect template group can be displayed, and the effect template selected by the user can be used as the target effect template. Preset template groups corresponding to different preset scenes, each containing at least one effect template, allow for quick selection of effect templates, improving the efficiency of effect processing and reducing power consumption.
[0137] Based on the same inventive concept, this disclosure also provides a special effects adding device for performing the steps of the special effects adding method provided in the above embodiments. The device can implement the special effects adding method in software, hardware or a combination of both. Figure 12 This is a block diagram illustrating a special effects adding device 100 according to an exemplary embodiment, such as... Figure 12As shown, the device 100 includes: a first determining module 110, a second determining module 120, an establishing module 130, and a calling module 140.
[0138] The first determining module 110 is configured to determine the target data type of the data object to which the special effect is to be added;
[0139] The second determining module 120 is configured to determine a target special effect template from a preset set of special effect templates corresponding to different data types based on the target data type.
[0140] The creation module 130 is configured to create a special effects addition process corresponding to the target data type;
[0141] The calling module 140 is configured to run the special effects adding process and call the special effects plugin storing the target special effects template to add special effects to the data object.
[0142] Optionally, there are multiple target effect templates, and the creation module 130 is configured to determine the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process based on the number of target effect templates and the association between them; and
[0143] Based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
[0144] Optionally, the association relationship includes: a first association relationship, which is used to restrict the order of adding special effects between target special effects templates, and the same subprocess is used to add special effects to target special effects templates with the first association relationship.
[0145] Optionally, the calling module 140 is configured to run each of the sub-processes in the special effects adding process in parallel, so that each sub-process can call the special effects plugin storing the target special effects template corresponding to the sub-process in parallel to add special effects to the data object.
[0146] Optionally, the data objects are multiple data objects to be concatenated, and the second determining module 120 is configured to determine the concatenation order among the multiple data objects; and,
[0147] For each data object, based on the target data type of the data object and the target data type of the next data object in the splicing order, a target transition effect template corresponding to the data object is determined from the preset transition effect templates. The target effect template includes the target transition effect template.
[0148] The calling module 140 is configured to sequentially call the special effects plugin that stores the target transition effect template corresponding to each of the data objects according to the splicing order, and add special effects to the data object.
[0149] Optionally, the calling module 140 is configured to determine the source of the data object;
[0150] When the source of the data object is a camera captured by the terminal, the target processor for adding special effects to the data object is determined based on the current memory usage of the terminal's central processing unit and the current cache state of the graphics processor.
[0151] The effects-adding process runs on the target processor, and the effects plugin storing the target effects template is invoked to add effects to the data object.
[0152] Optionally, the second determining module 120 is configured to determine the target scene to which the data object belongs from a preset scene classification;
[0153] From a plurality of preset special effect templates corresponding to the target scene, the special effect template corresponding to the target data type of the data object is selected as a candidate special effect template;
[0154] The target effect template is determined from the candidate effect templates.
[0155] Optionally, the second determining module 120 is configured to display the candidate special effects template;
[0156] When a user selects any effect template from the candidate effect templates, the selected candidate effect template is determined as the target effect template.
[0157] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0158] Furthermore, it is worth noting that, for the sake of convenience and brevity, the embodiments described in the specification are all preferred embodiments, and the parts involved are not necessarily essential to this disclosure. For example, the creation module 130 and the calling module 140 can be independent devices or the same device in specific implementations, and this disclosure does not limit them.
[0159] According to embodiments of this disclosure, a computer device is also provided, comprising:
[0160] processor;
[0161] Memory used to store processor-executable instructions;
[0162] The processor is configured as follows:
[0163] Determine the target data type of the data object to which the effect will be added;
[0164] Based on the target data type, the target effect template is determined from the preset effect templates corresponding to different data types;
[0165] Establish a process for adding special effects corresponding to the target data type;
[0166] Run the effect-adding process and call the effect plugin that stores the target effect template to add effects to the data object.
[0167] According to embodiments of this disclosure, a computer-readable storage medium is also provided, on which computer program instructions are stored, which, when executed by a processor, implement the steps of the special effects addition method described in any of the preceding claims.
[0168] Figure 13 This is a block diagram illustrating an apparatus 800 for adding special effects according to an exemplary embodiment. Optionally, the apparatus 800 may be configured as a computer device for performing the steps of the aforementioned special effects adding method. For example, the apparatus 800 may be a mobile phone, computer, digital broadcasting terminal, streaming media or multimedia transceiver, game console, tablet device, fitness equipment, personal digital assistant, etc.
[0169] Reference Figure 13 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.
[0170] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, data communication, camera operation, and recording operation. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the aforementioned special effects addition method. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.
[0171] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, audio, images, video, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0172] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 800.
[0173] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user, such as a touch input signal from the user selecting any effect template from candidate effect templates. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0174] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as shooting mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.
[0175] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0176] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0177] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0178] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described special effects addition method.
[0179] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to complete the aforementioned special effects addition method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0180] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described special effects addition method when executed by the programmable device.
[0181] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0182] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A method of special effects addition, characterized by, The method includes: Determine the target data type of the data object to which the effect will be added; Based on the target data type, the target effect template is determined from the preset effect templates corresponding to different data types; Establish a process for adding special effects corresponding to the target data type; Run the special effects adding process and call the special effects plugin that stores the target special effects template to add special effects to the data object; wherein, the special effects plugin is an independent module obtained by splitting the special effects function into the smallest split unit and then encapsulating the split special effects, and each special effects plugin contains only one special effects of the smallest split unit. The target effect templates are multiple, and the process of creating effect additions corresponding to the target data type includes: Based on the number of target effect templates and the relationships between them, the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process are determined. The relationships include a first relationship and a second relationship. The first relationship restricts the order of effect addition between the target effect templates. For target effect templates with the first relationship, the same sub-process is used for effect addition. The second relationship indicates that effect addition between target effect templates cannot be performed simultaneously. For target effect templates with the second relationship, different sub-processes are used for effect addition. Based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
2. The method of claim 1, wherein, The process of running the special effects addition process and calling the special effects plugin storing the target special effects template to add special effects to the data object includes: Each of the subprocesses in the effect addition process is run in parallel so that each subprocess can call the effect plugin that stores the target effect template corresponding to the subprocess in parallel to add effects to the data object.
3. The method of claim 1, wherein, The data objects are multiple data objects to be concatenated. The step of determining the target effect template from a set of preset effect templates corresponding to different data types, based on the target data type, includes: Determine the concatenation order among the multiple data objects; For each data object, based on the target data type of the data object and the target data type of the next data object in the splicing order, a target transition effect template corresponding to the data object is determined from the preset transition effect templates. The target effect template includes the target transition effect template. The step of calling the effects plugin storing the target effects template to add effects to the data object includes: According to the splicing order, the special effects plugins that store the target transition effect templates corresponding to each of the data objects are called in sequence to add special effects to the data objects.
4. The method according to claim 1, characterized in that, The process of running the special effects addition process and calling the special effects plugin storing the target special effects template to add special effects to the data object includes: Determine the source of the data object; When the source of the data object is a camera captured by the terminal, the target processor for adding special effects to the data object is determined based on the current memory usage of the terminal's central processing unit and the current cache state of the graphics processor. The effects-adding process runs on the target processor, and the effects plugin storing the target effects template is invoked to add effects to the data object.
5. The method according to claim 1, characterized in that, The step of determining the target special effect template from a preset set of special effect templates corresponding to different data types based on the target data type includes: Determine the target scene to which the data object belongs from the preset scene classification; From a plurality of preset special effect templates corresponding to the target scene, the special effect template corresponding to the target data type of the data object is selected as a candidate special effect template; The target effect template is determined from the candidate effect templates.
6. The method according to claim 5, characterized in that, Determining the target effect template from the candidate effect templates includes: Display the candidate effect templates; When a user selects any effect template from the candidate effect templates, the selected candidate effect template is determined as the target effect template.
7. A special effects adding device, characterized in that, The device includes: The first determination module is configured to determine the target data type of the data object to which the special effect is to be added; The second determining module is configured to determine the target special effect template from a preset set of special effect templates corresponding to different data types based on the target data type. The module is configured to create a special effects addition process corresponding to the target data type. The calling module is configured to run the effect adding process and call the effect plugin storing the target effect template to add effects to the data object; wherein, the effect plugin is an independent module obtained by splitting the effect function into the smallest split unit and then encapsulating the split effect, and each effect plugin contains only one effect of the smallest split unit. There are multiple target effect templates, and the creation module is configured as follows: Based on the number of target effect templates and the relationships between them, the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process are determined. The relationships include a first relationship and a second relationship. The first relationship restricts the order of effect addition between the target effect templates. For target effect templates with the first relationship, the same sub-process is used for effect addition. The second relationship indicates that effect addition between target effect templates cannot be performed simultaneously. For target effect templates with the second relationship, different sub-processes are used for effect addition. Based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
8. A computer device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Determine the target data type of the data object to which the effect will be added; Based on the target data type, the target effect template is determined from the preset effect templates corresponding to different data types; Establish a process for adding special effects corresponding to the target data type; Run the special effects adding process and call the special effects plugin that stores the target special effects template to add special effects to the data object; wherein, the special effects plugin is an independent module obtained by splitting the special effects function into the smallest split unit and then encapsulating the split special effects, and each special effects plugin contains only one special effects of the smallest split unit. The target effect templates are multiple, and the process of creating effect additions corresponding to the target data type includes: Based on the number of target effect templates and the relationships between them, the number of sub-processes in the effect addition process and the target effect template corresponding to each sub-process are determined. The relationships include a first relationship and a second relationship. The first relationship restricts the order of effect addition between the target effect templates. For target effect templates with the first relationship, the same sub-process is used for effect addition. The second relationship indicates that effect addition between target effect templates cannot be performed simultaneously. For target effect templates with the second relationship, different sub-processes are used for effect addition. Based on the number of subprocesses and the target effect template corresponding to each subprocess, an effect addition process corresponding to the target data type is established.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 6.