Special effect processing method, system and equipment and storage medium

Abstract

The embodiment of the invention provides a special effect processing method, system and device and a storage medium. The method comprises the following steps: receiving a special effect triggering operation of a target special effect through an operation interaction layer; responding to the special effect triggering operation through the following steps of a special effect processing engine: pulling a to-be-processed image frame from a data cache region constructed relative to the target special effect to carry out special effect processing, forming a target special effect image frame with the target special effect, and backfilling the target special effect image frame to the data cache region; and when a display condition is satisfied, pulling the target special effect image frame from the data cache region to display the target special effect image frame. By means of the method, integrated presentation of image frame collection, special effect processing and target special effect image display after special effect processing is achieved. The functions of image acquisition, special effect processing of the acquired image and display of the special effect image after the special effect processing can be realized directly based on the special effect processing engine, so that the processing efficiency and the use performance are improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of special effect processing technology, and in particular, to a special effect processing method, system, device and storage medium. Background Art

[0002] In the field of computer vision, integrated controls that combine acquisition controls and display controls are usually used as the underlying support tools for realizing image data acquisition and display in visual applications. Among them, acquisition controls mainly support controls for image data acquisition, and display controls mainly support controls for displaying the acquired image data.

[0003] With the emergence and continuous development of special effect technology, there is often a need to perform special effect processing on the collected image data in practical applications. In the existing underlying support for special effect processing, acquisition controls can still be used to collect image data, and then additional special effect processing controls are needed to add special effects to the collected data.

[0004] At present, in the implementation of special effect processing using the above existing methods, since the additional special effect processing control cannot be integrated into the acquisition control, the special effect image data after the special effect processing by the additional special effect processing control cannot be directly obtained and displayed by the display control, but the data after the special effect processing needs to be introduced into the additionally programmed display control for display. The above implementation method of special effect processing greatly increases the complexity of special effect processing and also increases the computing resource occupation of special effect processing. Summary of the invention

[0005] The present disclosure provides a special effects processing method, system, device and storage medium, which realize the integrated presentation of image frame acquisition, special effects processing and target special effects image display after special effects processing, simplify the special effects processing process, thereby reducing the computing resource occupation of special effects processing and improving processing efficiency and usage performance.

[0006] In a first aspect, an embodiment of the present disclosure provides a special effect processing method, the method comprising:

[0007] Receive the special effect triggering operation of the target special effect through the operation interaction layer;

[0008] The special effect processing engine responds to the special effect triggering operation through the following steps:

[0009] Pulling the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area;

[0010] When the display condition is met, the target special effect image frame is pulled from the data buffer area to display the target special effect image frame.

[0011] In a second aspect, the embodiment of the present disclosure further provides a special effect processing system, which is used to execute the method described in the embodiment of the first aspect, and the system includes: an operation interaction layer and a special effect processing engine,

[0012] The operation interaction layer is used to receive a special effect triggering operation of a target special effect;

[0013] The special effect processing engine is used to execute the following steps in response to the special effect triggering operation:

[0014] Pulling the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area;

[0015] When the display condition is met, the target special effect image frame is pulled from the data buffer area to display the target special effect image frame.

[0016] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0017] one or more processors;

[0018] a storage device for storing one or more programs,

[0019] When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects processing method provided in the first aspect of the embodiment of the present disclosure.

[0020] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to implement the special effects processing method provided in the first aspect of the embodiments of the present disclosure.

[0021] The disclosed embodiments provide a special effect processing method, system, device and storage medium. Through the special effect processing method, first, the special effect triggering operation of the target special effect is received through the operation interaction layer; then the special effect triggering operation is responded to through the following steps of the special effect processing engine: pull the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, form a target special effect image frame with the target special effect and fill it back into the data buffer area; pull the target special effect image frame from the data buffer area to display the target special effect image frame when the display condition is met. The above technical scheme, by constructing a data buffer area relative to the target special effect, the data buffer area can be regarded as a common area for caching image frames, so that the special effect processing engine can store the image frame in the data buffer area, and can also pull the image frame from the data buffer area, thereby ensuring that the special effect processing engine can collect the image frame to be processed, perform special effect processing on the collected image frame to be processed, and display the target special effect image frame after special effect processing. The reason why the special effects processing engine can perform special effects processing on the image frames to be processed and display the target special effects image frames after special effects processing is because it is associated with a data cache area. The data can fall into the data cache area and can be obtained when the next logic is executed. In this way, the image frame acquisition, special effects processing, and the integrated presentation of the target special effects image display after special effects processing are realized. The above technical solution only needs to be directly based on the special effects processing engine to realize the functions of image acquisition, special effects processing on the acquired images, and displaying the target special effects image frames after special effects processing. It no longer needs to use additional special effects processing controls to add special effects to the collected data, realizes the processing and display of special effects, simplifies the special effects processing process, thereby reducing the computing resources occupied by special effects processing and improving processing efficiency and performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0023] Figure 1 A flowchart of a special effect processing method provided by an embodiment of the present disclosure;

[0024] Figure 2 A structural schematic diagram of a special effect processing system provided by an embodiment of the present disclosure;

[0025] Figure 3 Another structural schematic diagram of a special effect processing system provided by an embodiment of the present disclosure;

[0026] Figure 4A schematic diagram of another structure of a special effect processing system provided by an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of another structure of a special effect processing system provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of another structure of a special effect processing system provided by an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of another structure of a special effect processing system provided by an embodiment of the present disclosure;

[0030] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0032] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0033] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0034] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0035] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0036] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0037] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0038] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0039] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0040] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0041] With the emergence and continuous development of special effects technology, there is often a need to perform special effects processing on the collected image data in practical applications. For example, special effects processing is performed on the image collected by the camera, and then the image after special effects processing is output and displayed. An integrated control that combines acquisition controls and display controls is usually used as the underlying support tool for realizing image data acquisition and display in visual applications. Among them, the acquisition control mainly supports controls for image data acquisition, and the display control mainly supports controls for displaying the collected image data. For scenarios where special effects processing is required for the collected image before display, it is not possible to directly use the integrated control that combines acquisition controls and display controls to realize special effects processing. In the existing underlying support for realizing special effects processing, acquisition controls can still be used to collect image data, and then additional special effects processing controls are needed to add special effects to the collected data.

[0042] In the implementation of special effect processing using the above existing method, since the additional special effect processing control cannot be integrated into the acquisition control, the special effect image data after the special effect processing by the additional special effect processing control cannot be directly obtained and displayed by the display control, but the data after the special effect processing needs to be introduced into the additionally programmed display control for display. The above implementation method of special effect processing greatly increases the complexity of special effect processing and also increases the computing resource occupation of special effect processing.

[0043] Based on this, this embodiment provides a special effects processing method, which realizes the functions of image frame acquisition, special effects processing, and integrated presentation of target special effects images after special effects processing.

[0044] Specifically, Figure 1 A flow chart of a special effects processing method provided in an embodiment of the present disclosure is applicable to the situation of centralized processing of image acquisition, special effects processing and display. The method can be executed by a special effects processing system, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device as an execution terminal, which can be a mobile terminal, a PC or a server, etc.

[0045] like Figure 1 As shown, a special effect processing method provided by an embodiment of the present disclosure specifically includes the following operations:

[0046] S110: Receive a special effect triggering operation of a target special effect through the operation interaction layer.

[0047] The application scenario of this embodiment can be understood as follows: an electronic device having a special effect processing method provided by this embodiment is provided with a special effect application function, and many special effects can be provided at the visualization level, and users can select special effects. In this embodiment, the visualization level is recorded as the operation interaction layer, and the operation performed by the user can be received through the operation interaction layer. There are many special effects provided on the operation interaction layer for users to choose from. In this embodiment, the special effect selected by the user is recorded as the target special effect, and the selection trigger operation of the target special effect is recorded as the special effect trigger operation.

[0048] In this embodiment, there is no specific restriction on the target special effect. The user can select the special effect that he wants to add according to his own needs. The user triggers the special effect by the target special effect that he wants to apply. The form of the special effect triggering operation is not specifically limited. For example, the special effect triggering operation can be performed by dragging or clicking. Correspondingly, this execution subject can receive the triggering operation of the target special effect through the operation interaction layer. This process can be understood as that after receiving the special effect triggering operation of the target special effect, the behavior of the special effect triggering operation can be analyzed to determine what kind of special effect the user has selected as the target special effect.

[0049] S120, responding to the special effect triggering operation through the following steps of the special effect processing engine:

[0050] a1) Pulling an image frame to be processed from a data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area.

[0051] Different from the prior art that requires the use of additional special effects processing controls to add special effects to the collected data, in this embodiment, the special effects processing engine can be regarded as an underlying integrated architecture, which can be considered to include acquisition controls for image acquisition, display controls for image frame display, and processing controls for special effects processing, integrating the functions of image acquisition, special effects processing, and display of images after special effects processing.

[0052] The reason why the special effects processing engine can perform special effects processing on the image frames to be processed and display the target special effects image frames after special effects processing is because it is associated with a data cache area. The data can fall into the data cache area and can be retrieved when the next logic is executed. In this way, the integrated presentation of image frame acquisition, special effects processing, and target special effects image display after special effects processing is realized.

[0053] It should be noted that in this embodiment, a data cache area is pre-constructed. Through the constructed data cache area, the captured image frames can be filled into the data cache area. At the same time, the image frames can be pulled from the data cache area for special effects processing. The image frames after special effects processing can also be backfilled into the data cache area so that the image frames in the data cache area can be read and displayed by the upper layer.

[0054] It can be considered that the data buffer is a public area, and the captured image frames can be stored in the data buffer, and the image frames stored in the data buffer can be pulled out for special effect processing, and the image frames after special effect processing can be filled back into the data buffer. Furthermore, the image frames after special effect processing can be pulled out from the data buffer for final display.

[0055] This step can be considered as a response to the received special effect trigger operation. This step mainly realizes that the special effect processing engine can pull the image frame from the data buffer area constructed by the target special effect, and record the image frame to be processed in the data buffer area as the image frame to be processed. After the special effect processing engine pulls the image frame to be processed, it performs special effect processing on the image frame to be processed according to the target special effect, and records the image frame after special effect processing as the target special effect image frame with the target object. After obtaining the target special effect image frame, the target special effect image frame is backfilled into the data buffer area.

[0056] In this embodiment, before performing special effects processing on the image frame to be processed, the special effects processing engine also executes the step of constructing a data buffer relative to the target special effects. Specifically, after receiving a special effects trigger operation for a target special effect, it is possible to determine which types of objects the target special effect consists of through analysis, and each type of object needs to have a corresponding special effects processing control to process it. In this way, when it is determined which types of objects the target special effect contains, the special effects processing control corresponding to each object can be obtained from the pre-built special effects processing control set. Each special effects processing control can create a data buffer corresponding to itself, which can store image frames sent by the previous control, and can also fill the special effects image frames after the special effects processing control is processed by the special effects processing control back into the data buffer.

[0057] It should be clear that the target special effect may be composed of one or more objects. If the target special effect is composed of one object, only one data buffer needs to be built, the image frame to be processed will be stored in the data buffer, the image frame to be processed will be pulled from the data buffer, and the special effect processing will be performed on the image frame to be processed, and the processed target special effect image frame will be filled back into the data buffer.

[0058] When the target special effect is composed of multiple objects, each object corresponds to a special effect processing control. It is necessary to determine how many special effect processing controls are needed for special effect processing according to the multiple objects corresponding to the target special effect, as well as the execution order between the special effect processing controls and the execution direction. In this case, each special effect processing control corresponds to its own data buffer. From the perspective of a special effect processing control, after receiving the image frame processed by the previous special effect, the image frame processed by the previous special effect can be pulled from its own data buffer, and the image frame processed by the previous special effect can be processed by the special effect, and the processed image frame can be filled back into its own data buffer. The image frame processed by the special effect processing control backfilled by the special effect processing control is passed to the next special effect processing control. This process can be considered as a process of performing special effect processing on the collected image frames to be processed in turn based on multiple special effect processing controls. This process is a process of continuously pulling image frames and filling them back, and the pulling and filling processes are for different data buffers.

[0059] b1) when a display condition is met, pulling the target special effect image frame from the data buffer area to display the target special effect image frame.

[0060] Among them, satisfying the display condition can be understood as the special effect image frames in the data cache area satisfying the requirements of the target special effect, or it can be considered that all the target special effect image frames have been backfilled in the data cache area. Whether the display condition is satisfied can be actively monitored or passively informed. When it is detected or monitored that the data cache area has stored the special effect image frames involved in the target special effect and has completed the target special effect processing, it is considered that the display condition is satisfied.

[0061] Continuing with the above description, when the display conditions are met, the target special effect image frame can be pulled from the data buffer area. The target special effect image frame can be considered as an image frame obtained by performing special effect processing of the collected image frame to be processed with the target special effect, and the target special effect image frame is displayed.

[0062] It is clear that the target special effect may be composed of one or more objects. If the target special effect is composed of only one object, after detecting that the backfill of the target special effect image frame to the corresponding data buffer is completed, the target special effect image frame is pulled from the data buffer to display the target special effect image frame when the display conditions are met. If the target special effect is composed of multiple objects, it is also necessary to determine whether the special effect processing of multiple special effect processing controls associated with all target special effects has been performed and whether there are backfilled image frames in the data buffer. Only when the above conditions are met, the target special effect image frame will be further pulled from the data buffer to display the target special effect image frame when the display conditions are met.

[0063] The disclosed embodiment provides a special effect processing method, which first receives a special effect triggering operation of a target special effect through an operation interaction layer; then responds to the special effect triggering operation through the following steps of a special effect processing engine: pulling an image frame to be processed from a data cache area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame having the target special effect and backfilling it into the data cache area; pulling the target special effect image frame from the data cache area to display the target special effect image frame when a display condition is met. The above technical scheme, by constructing a data cache area relative to the target special effect, the data cache area can be regarded as a common area for caching image frames, so that the special effect processing engine can store image frames in the data cache area, and can also pull image frames from the data cache area, thereby ensuring that the special effect processing engine can collect image frames to be processed, perform special effect processing on the collected image frames to be processed, and display the target special effect image frames after special effect processing. The reason why the special effects processing engine can perform special effects processing on the image frames to be processed and display the target special effects image frames after special effects processing is because it is associated with a data cache area. The data can fall into the data cache area and can be obtained when the next logic is executed. In this way, the image frame acquisition, special effects processing, and the integrated presentation of the target special effects image display after special effects processing are realized. The above technical solution only needs to be directly based on the special effects processing engine to realize the functions of image acquisition, special effects processing on the acquired images, and displaying the target special effects image frames after special effects processing. It no longer needs to use additional special effects processing controls to add special effects to the collected data, realizes the processing and display of special effects, simplifies the special effects processing process, thereby reducing the computing resources occupied by special effects processing and improving processing efficiency and performance.

[0064] On the basis of the above embodiment, as a first optional embodiment of this embodiment, the special effect processing engine may be optimized to include a processing scheduling control and a special effect processing control set.

[0065] The special effect processing engine includes a processing scheduling control and a special effect processing control set, which can be understood as a set of algorithms and functions for special effect processing. It can be considered that the special effect processing control set contains several special effect processing controls, and different special effect processing requires different special effect processing controls.

[0066] Further, the method may be optimized to include:

[0067] a2) Determine at least one target special effect processing control associated with the target special effect from the special effect processing control set through the processing scheduling control and according to the special effect component object of the target special effect, wherein one target special effect processing control corresponds to the special effect processing of one special effect component object.

[0068] It can be considered that before the special effect processing is performed on the frame to be processed, it actually includes analyzing the target special effect itself to determine which special effect component objects the target special effect is composed of. This step is performed by processing the scheduling control. The special effect component object can be understood as an object that constitutes the target special effect, and the target special effect can be composed of one or more special effect component objects.

[0069] For example, assuming that the target special effect is only a dynamic rain scene, it is equivalent to that the target special effect only includes one special effect component object. If the target special effect is a dynamic rain scene and a cat head sticker special effect, it is equivalent to that the target special effect includes two special effect component objects, one of which is a dynamic rain scene and the other is a cat head sticker special effect.

[0070] Continuing with the above description, after determining which special effect component objects the target special effect contains, it is necessary to determine one or more target special effect processing controls associated with the target special effect from the special effect processing control set. It can be understood that a target special effect processing control can contain a method or function required for special effect processing, which is equivalent to a target special effect processing control corresponding to the special effect processing of a special effect component object. It can also be considered that when the target special effect contains a special effect component object, it is necessary to determine the target special effect processing control corresponding to the special effect processing of this special effect component object from the special effect processing control set, which is called the target special effect processing control associated with the target special effect. When the target special effect contains multiple special effect component objects, it is necessary to determine the target special effect processing control corresponding to the special effect processing of each special effect component object from the special effect processing control set, and these determined target special effect processing controls are called target special effect processing controls associated with the special effect. Based on this, it is equivalent to determining which target special effect processing controls are needed to achieve the target special effect.

[0071] b2) constructing a data buffer area relative to the target special effect through the target special effect processing control, wherein the data buffer area corresponds one-to-one to the target special effect processing control associated with the target special effect.

[0072] In this embodiment, after determining the target special effect processing control associated with the target special effect through the processing scheduling control, the target special effect processing control can be used to establish a corresponding data cache area for the target special effect. The data cache area is associated with the target special effect processing control, and the construction of the data cache area is performed by the target special effect processing control, and the data cache area corresponds to the target special effect processing control associated with the target special effect one by one, that is, after determining how many target special effect processing controls are associated with the target special effect, it is necessary to construct several data cache areas, and each target special effect processing control corresponds to one data cache area.

[0073] This first optional embodiment adds a pre-processing step based on the target special effects. First, it is necessary to determine which target special effects processing controls are needed to achieve the target special effects processing, and use these target special effects processing controls to construct corresponding data cache areas, which provide a basis for the storage of image frames before subsequent special effects processing and image frames after processing.

[0074] Considering that the number of target processing controls associated with a target special effect may be one or more, the following describes the cases where the number of target processing controls associated with a target special effect is one and more respectively.

[0075] This optional embodiment describes how to form a target special effect image frame and fill it back into the data buffer area when the number of target special effect processing controls associated with the target special effect is 1. Based on the above first optional embodiment, as a second optional embodiment of this embodiment, when the number of target special effect processing controls associated with the target special effect is 1, the image frame to be processed can be pulled from the data buffer area constructed relative to the target special effect for special effect processing, and the target special effect image frame with the target special effect can be formed and filled back into the data buffer area to achieve the optimization of the following steps:

[0076] a3) Using the image acquisition control in the special effect processing engine, the acquired image frames are filled into the data buffer area corresponding to the target special effect processing control as the image frames to be processed.

[0077] In this embodiment, the special effect processing engine provides an image acquisition function, and this function is implemented by the image acquisition control in the special effect processing engine. The image acquisition control can be used to acquire image frames, and the acquired image frames are used as image frames to be processed. In this embodiment, the target special effect processing control associated with the target special effect is 1. When the target special effect processing control is 1, the corresponding data buffer area is also only 1. Accordingly, the image frame to be processed is filled into the data buffer area corresponding to the target special effect processing control.

[0078] b3) extracting the image frame to be processed from the data buffer area through the target special effect processing control for special effect processing, and recording the image frame after special effect processing as a target special effect image frame with the target special effect and filling it back into the data buffer area.

[0079] Specifically, the target special effect processing control includes algorithms and functions required for special effect processing. After the image frame to be processed is stored in the data buffer area, the target special effect processing control can extract the image frame to be processed from the data buffer area, and perform special effect processing on the image frame to be processed. After the processing is completed, the image frame obtained is recorded as the target special effect image frame with the target special effect. The target image frame is further filled back into the data buffer area.

[0080] This second optional embodiment specifies the steps of performing special effects processing on the image frame to be processed and obtaining the target special effects image frame when the number of target special effects processing controls associated with the target special effects is 1, and the acquisition and special effects processing of the image frame can be realized based on the special effects processing engine. Different from the prior art that requires the use of additional special effects processing controls to perform special effects on the collected data, this technical solution realizes the integrated realization of image frame acquisition and special effects processing on the collected image frames.

[0081] On the basis of the above second optional embodiment, as a third optional embodiment of this embodiment, the implementation of pulling the target special effect image frame from the data buffer area to display the target special effect image frame when the display condition is met can be optimized into the following steps:

[0082] a4) After detecting, through the target special effect processing control, that the backfilling of the target special effect image frame to the corresponding data buffer area is completed, a special effect image frame transfer request is sent to the data transfer control in the special effect processing engine.

[0083] In this embodiment, the special effect processing engine further includes a data transfer control. After detecting completion of backfilling of the target special effect image frame to the corresponding data buffer area through the target special effect processing control, an image frame transfer request is sent to the data transfer control.

[0084] Exemplarily, the target special effect processing control includes at least several method functions, one method function is used to receive the image frame collected by the image acquisition control and put it into the data buffer area, another method function is used to process the image frame in the data buffer area and fill the processed image frame back into the data buffer area. Another method function can be, when the target special effect image frame is filled back into the corresponding data buffer area, to inform the data transfer control that the backfilling of the target special effect image frame to the corresponding data buffer area has been completed.

[0085] The special effect image frame transfer request can be specifically understood as a request to the data transfer control to inquire whether to pull the target special effect image frame from the data buffer area. Exemplarily, the special effect image frame transfer request can include the pull address information of the data buffer area, the inquiry request information, such as whether the target special effect image frame needs to be pulled from the data buffer area, etc. In this embodiment, there is no specific limitation on the special effect image frame transfer request.

[0086] b4) After receiving the special effect image frame transfer request through the data transfer control, it is determined that the display conditions are met, the target special effect image frame is pulled from the data buffer area and passed to the special effect display control in the special effect processing engine.

[0087] The special effect processing engine also includes a special effect display control for displaying image frames. Specifically, after receiving a special effect image transfer request through the data transfer control, it is necessary to determine whether the display conditions are met. If the display conditions are met, the target special effect image frame is pulled from the data buffer and the target special effect image frame is passed to the special effect display control.

[0088] c4) displaying the target special effect image frame through the special effect display control.

[0089] Specifically, the special effect display control can display the target special effect image frame.

[0090] This third optional embodiment specifies the steps of how to pull the target special effect image frame from the data buffer and display the target special effect image frame when the number of target processing controls associated with the target special effect is 1. Different from the prior art, the special effect image data after special effect processing by additional special effect processing controls cannot be directly obtained and displayed by the display control, but the special effect processed data needs to be introduced into the additionally programmed display control for display; this technical solution realizes the integrated realization of special effect processing of image frames and display of image frames after special effect processing.

[0091] It should be noted that a target special effect may also be associated with multiple target special effect processing controls. It can also be considered that to achieve a target special effect, the image frames captured by the image acquisition control need to be processed with special effects based on multiple target special effect processing controls in sequence.

[0092] This optional embodiment describes how to form a target special effect image frame and fill it back into the data buffer when there are multiple target special effect processing controls associated with the target special effect. Based on the above first optional embodiment, as the fourth optional embodiment of this embodiment, when there are multiple target special effect processing controls associated with the target special effect, the image frame to be processed can be pulled from the data buffer constructed relative to the target special effect for special effect processing, and the target special effect image frame with the target special effect can be formed and filled back into the data buffer to achieve the optimization of the following steps:

[0093] a5) determining the special effect processing global table of the target special effect through the processing scheduling control and sending it to multiple target special effect processing controls, wherein the special effect processing global table includes the processing order of multiple target special effect processing controls.

[0094] In this embodiment, the case where there are multiple target special effect processing controls associated with the target special effect is targeted. When there are multiple target special effect processing controls, there are also multiple corresponding data buffers. Multiple target special effect processing controls are required to cooperate, and then it is necessary to determine the order of processing of each target special effect processing control associated with the target. In this embodiment, the processing order of multiple target special effect processing controls can be determined first, and the table containing the processing order of multiple target special effect processing controls is recorded as a special effect processing global table.

[0095] Continuing with the above description, in this embodiment, a relationship can be established between multiple target special effect processing controls by processing the scheduling control, which is equivalent to establishing a special effect processing global table for multiple target special effect processing controls. The special effect processing global table is then sent to each determined target special effect processing control, and the data flow direction can be known based on the special effect processing global table.

[0096] In this embodiment, the method of determining the global table of special effects processing of the target special effects can be that the special effects component objects constituting the target special effects can be obtained by analyzing the target special effects. When the special effects component objects are displayed, there may be a display hierarchy. According to the display hierarchy, it can be determined which special effects component object should be processed first and which special effects component object should be processed later, thereby obtaining the processing order of the special effects component objects, which is equivalent to the processing order of multiple target special effects processing controls, thereby forming a global table of special effects processing. Exemplarily, assuming that the target special effects include a scene placed in the rain and a cat head sticker effect, the order of processing the two can be determined according to the hierarchy of the scene in the rain and the cat head sticker, and a global table of special effects processing can be formed.

[0097] b5) performing the following steps through the target special effect processing control:

[0098] b51) When it is detected that the corresponding data buffer is filled with image frames to be processed, the image frames to be processed are extracted from the corresponding data buffer for special effect processing, and the intermediate special effect image frames formed after the special effect processing are backfilled into the corresponding data buffer. The image frames to be processed are image frames captured by the image acquisition control in the special effect processing engine, or are the previous intermediate special effect image frames formed after special effect processing by the previous special effect processing control.

[0099] In this embodiment, the special effect processing engine provides an image acquisition function, and this function is implemented by the image acquisition control in the special effect processing engine. The image acquisition control can be used to acquire image frames and use the acquired image frames as image frames to be processed. This situation refers to when the original acquired image frames are subjected to the first special effect processing, the image frames acquired by the image acquisition control are used as image frames to be processed. The image acquisition control can be informed through the processing scheduling control to which target special effect processing control the acquired image frames are to be flowed.

[0100] Considering that there are multiple target special effect processing controls associated with the target special effect, the image frames will be processed in sequence. This process can be understood as the special effect image frame formed after the previous special effect processing control performs special effect processing is recorded as the previous intermediate special effect image frame. After the previous intermediate special effect image frame is processed, the processed image frame will be given to the data buffer area corresponding to the target special effect processing control performing special effect processing at the current moment.

[0101] From the perspective of the target special effect processing control that is currently to perform special effect processing, in this embodiment, the target special effect processing control will detect whether the corresponding data buffer is filled with image frames to be processed. When it is detected that the corresponding data buffer is filled with image frames to be processed, the image frames to be processed can be pulled from the corresponding data buffer for special effect processing. Specifically, the target special effect processing control contains algorithms and functions required for special effect processing. After the image frames to be processed are stored in the data buffer, the target special effect processing control can extract the image frames to be processed from the data buffer, perform special effect processing on the image frames to be processed, and fill the intermediate special effect image frames formed after the special effect processing back into the corresponding data buffer.

[0102] This process can be considered as a process of performing special effects processing on the collected image frames to be processed in sequence based on multiple special effects processing controls. This process is a process of continuously pulling image frames and filling them back, and the pulling and filling processes are for different data buffers. That is, the special effects processing result of the previous special effects processing control is recorded as the previous intermediate special effects image frame, and the previous intermediate special effects image frame will be given to the next special effects processing control.

[0103] It should be clear that in order to identify whether the image frame stored in the data buffer is an image frame before or after processing by the target special effect processing control corresponding to the data buffer, a tag value can be set. It can be understood that the image frame to be processed is encapsulated in a package to be processed, and a tag value is set for it. Exemplarily, the tag value can be used to characterize that the image frame is an image frame to be processed.

[0104] Based on this step, the target special effect processing control can be used to pull the image frames to be processed in the data buffer area, perform special effect processing, and backfill the intermediate special effect image frames formed after the special effect processing.

[0105] b52) when it is determined according to the special effect data global table that there is a next special effect processing control, the intermediate special effect image frame is transferred as an image frame to be processed to a data buffer area corresponding to the next special effect processing control.

[0106] Among them, the special effect data global table includes the processing order of multiple target special effect processing controls, so that based on the current special effect processing control and the special effect data global table, it can be determined whether there is a next control to be processed. If there is no next control to be processed, the data in the data buffer area of ​​the current target special effect processing control is the image frame that will eventually be processed with the target special effect. If it is determined according to the special effect data global table that there is a next special effect processing control, the intermediate special effect image frame needs to be transferred as the image frame to be processed to the data buffer area corresponding to the next special effect processing control. In this way, the image frame to be processed can be pulled from the corresponding data buffer area by the next special effect processing control and special effect processing can be performed.

[0107] It is understandable that the special effect data global table determines the order in which each target special effect processing control performs special effect processing. The target special effect processing control obtains the image frame to be processed from its own corresponding data buffer area, performs special effect processing on the image frame to be processed, and transfers the intermediate special effect image frame after special effect processing to its own corresponding data buffer area. The intermediate special effect image frame is further transferred as the image frame to be processed to the data buffer area corresponding to the next special effect processing control, and the above steps are performed in sequence until it is determined according to the special effect data global table that the next special effect processing control no longer exists, and the above steps are no longer performed.

[0108] This fourth optional embodiment specifies the steps of performing special effects processing on the image frame to be processed and obtaining the intermediate special effects image frame when there are multiple target special effects processing controls associated with the target special effects. Based on a predetermined global table of special effects data, multiple target special effects processing controls can sequentially perform special effects processing on the image frame to be processed, and the image frame acquisition and special effects processing can be realized based on the special effects processing engine. Different from the prior art that requires the use of additional special effects processing controls to perform special effects on the collected data, this technical solution realizes the integrated realization of image frame acquisition and special effects processing.

[0109] On the basis of the fourth optional embodiment described above, as a fifth optional embodiment of this embodiment, the implementation of pulling the target special effect image frame from the data buffer area to display the target special effect image frame when the display condition is met can be optimized into the following steps:

[0110] a6) When it is determined through the target special effect processing control that there is no next special effect processing control according to the special effect data global table, and when it is detected that the corresponding data buffer area is backfilled with an intermediate special effect image frame after special effect processing, the intermediate special effect image frame is used as the target special effect image frame, and a special effect image frame transfer request is sent to the data transfer control in the special effect processing engine.

[0111] In this embodiment, it is necessary to determine whether two conditions are met through the target special effect processing control. One of the conditions is to determine that there is no next special effect processing control based on the special effect data global table. It can also be considered that the target special effect processing controls associated with the target special effect have completed the special effect processing on the image frame. The other condition is to detect that the corresponding data buffer area is backfilled with an intermediate special effect image frame after special effect processing. When these two conditions are met, it is considered that the intermediate special effect image frame is the image frame after the entire target special effect is applied, and the intermediate special effect image frame is used as the target special effect image frame. Similarly, when the target special effect image frame is backfilled into the corresponding data buffer area, the data transfer control is informed that the backfilling of the target special effect image frame to the corresponding data buffer area has been completed.

[0112] The special effect image frame transfer request can be specifically understood as a request to the data transfer control to inquire whether to pull the target special effect image frame from the data buffer area. Exemplarily, the special effect image frame transfer request can include the pull address information of the data buffer area, the inquiry request information, such as whether the target special effect image frame needs to be pulled from the data buffer area, etc. In this embodiment, there is no specific limitation on the special effect image frame transfer request.

[0113] b6) After receiving the special effect image frame transfer request through the data transfer control, it is determined that the display conditions are met, the target special effect image frame is pulled from the data buffer area and passed to the special effect display control in the special effect processing engine.

[0114] The special effect processing engine also includes a special effect display control. Specifically, after receiving a special effect image transfer request through the data transfer control, it is necessary to determine whether the display conditions are met. If the display conditions are met, the target special effect image frame is pulled from the data buffer and the target special effect image frame is transferred to the special effect display control.

[0115] c6) Displaying the target special effect image frame through the special effect display control.

[0116] Specifically, the special effect display control can display the target special effect image frame.

[0117] The above technical solution specifies the steps of how to pull the target special effect image frame from the data cache and display the target special effect image frame when there are multiple target processing controls associated with the target special effect. Only when it is determined that there is no next special effect processing control according to the global table of special effect data, and it is detected that the corresponding data cache is backfilled with an intermediate special effect image frame after special effect processing, will the intermediate special effect image frame be used as the target special effect image frame. Different from the prior art, the special effect image data after special effect enhancement processing by additional special effect processing controls cannot be directly obtained and displayed by the display control, but the data after special effect processing needs to be introduced into the additionally programmed display control for display; this technical solution realizes the integrated realization of special effect processing of image frames and display after special effect processing.

[0118] On the basis of the above embodiment, as the sixth optional embodiment of this embodiment, the method can be optimized to also include: providing display attribute information for displaying the target special effect image frame through the basic data support control in the special effect processing engine, so as to display the target special effect image according to the display attribute information.

[0119] In this embodiment, the special effect processing engine also includes a basic data support control, through which the display attribute information of the target special effect image frame display can be configured after the user gives some attribute configurations at the visualization level during the development process or the use process. Thus, the target special effect image can be displayed according to the display attribute information. Exemplarily, the display attribute information may include attribute information such as the resolution of the image frame display and the display size of the image frame, which is not specifically limited here.

[0120] The sixth optional embodiment described above adds a configuration function of display attribute information, and provides display attribute information for displaying a target special effect image frame through a basic data support control, so that the target special effect image is displayed according to the display attribute information.

[0121] Figure 2 The structure diagram of a special effect processing system provided by the embodiment of the present disclosure is shown in FIG. 1 , and the special effect processing system is used to execute the method described in the above embodiment. The embodiment of the present disclosure is applicable to the situation of centralized processing of image acquisition, special effect processing and display. Figure 2As shown, the system includes: an operation interaction layer 21 and a special effect processing engine 22, the operation interaction layer 21 is used to receive a special effect trigger operation of a target special effect; the special effect processing engine 22 is used to execute the following steps in response to the special effect trigger operation: pulling the image frame to be processed from the data cache area constructed relative to the target special effect for special effect processing, forming a target special effect image frame with the target special effect and backfilling it to the data cache area; pulling the target special effect image frame from the data cache area to display the target special effect image frame when the display conditions are met.

[0122] The application scenario of this embodiment can be understood as follows: an electronic device having a special effect processing method provided by this embodiment is provided with a special effect application function, and many special effects can be provided at the visualization level, and users can select special effects. In this embodiment, the visualization level is recorded as the operation interaction layer, and the operation performed by the user can be received through the operation interaction layer. There are many special effects provided on the operation interaction layer for users to choose from. In this embodiment, the special effect selected by the user is recorded as the target special effect, and the selection trigger operation of the target special effect is recorded as the special effect trigger operation.

[0123] In this embodiment, there is no specific restriction on the target special effect. The user can select the special effect that he wants to add according to his own needs. The user triggers the special effect by the target special effect that he wants to apply. The form of the special effect triggering operation is not specifically limited. For example, the special effect triggering operation can be performed by dragging or clicking. Correspondingly, this execution subject can receive the triggering operation of the target special effect through the operation interaction layer. This process can be understood as that after receiving the special effect triggering operation of the target special effect, the behavior of the special effect triggering operation can be analyzed to determine what kind of special effect the user has selected as the target special effect.

[0124] The special effects processing engine 22 is used to execute the following steps in response to the special effects triggering operation: pulling the image frame to be processed from the data cache area constructed relative to the target special effect for special effects processing, forming a target special effects image frame with the target special effects and backfilling it into the data cache area; pulling the target special effects image frame from the data cache area for display when the display conditions are met.

[0125] Different from the prior art that requires the use of additional special effects processing controls to add special effects to the collected data, in this embodiment, the special effects processing engine can be regarded as an underlying integrated architecture, which can be considered to include acquisition controls for image acquisition, display controls for image frame display, and processing controls for special effects processing, integrating the functions of image acquisition, special effects processing, and display of images after special effects processing.

[0126] The reason why the special effects processing engine can perform special effects processing on the image frames to be processed and display the target special effects image frames after special effects processing is because it is associated with a data cache area. The data can fall into the data cache area and can be retrieved when the next logic is executed. In this way, the integrated presentation of image frame acquisition, special effects processing, and target special effects image display after special effects processing is realized.

[0127] It should be noted that in this embodiment, a data cache area is pre-constructed. Through the constructed data cache area, the captured image frames can be filled into the data cache area. At the same time, the image frames can be pulled from the data cache area for special effects processing. The image frames after special effects processing can also be backfilled into the data cache area so that the image frames in the data cache area can be read and displayed by the upper layer.

[0128] It can be considered that the data buffer is a public area, and the captured image frames can be stored in the data buffer, and the image frames stored in the data buffer can be pulled out for special effect processing, and the image frames after special effect processing can be filled back into the data buffer. Furthermore, the image frames after special effect processing can be pulled out from the data buffer for final display.

[0129] This step can be considered as a response to the received special effect trigger operation. This step mainly realizes that the special effect processing engine can pull the image frame from the data buffer area constructed by the target special effect, and record the image frame to be processed in the data buffer area as the image frame to be processed. After the special effect processing engine pulls the image frame to be processed, it performs special effect processing on the image frame to be processed according to the target special effect, and records the image frame after special effect processing as the target special effect image frame with the target object. After obtaining the target special effect image frame, the target special effect image frame is backfilled into the data buffer area.

[0130] In this embodiment, before performing special effects processing on the image frame to be processed, the special effects processing engine also executes the step of constructing a data buffer relative to the target special effects. Specifically, after receiving a special effects trigger operation for a target special effect, it is possible to determine which types of objects the target special effect consists of through analysis, and each type of object needs to have a corresponding special effects processing control to process it. In this way, when it is determined which types of objects the target special effect contains, the special effects processing control corresponding to each object can be obtained from the pre-built special effects processing control set. Each special effects processing control can create a data buffer corresponding to itself, which can store image frames sent by the previous control, and can also fill the special effects image frames after the special effects processing control is processed by the special effects processing control back into the data buffer.

[0131] It should be clear that the target special effect may be composed of one or more objects. If the target special effect is composed of one object, only one data buffer needs to be built, the image frame to be processed will be stored in the data buffer, the image frame to be processed will be pulled from the data buffer, and the special effect processing will be performed on the image frame to be processed, and the processed target special effect image frame will be filled back into the data buffer.

[0132] When the target special effect is composed of multiple objects, each object corresponds to a special effect processing control. It is necessary to determine how many special effect processing controls are needed for special effect processing according to the multiple objects corresponding to the target special effect, as well as the execution order between the special effect processing controls and the execution direction. In this case, each special effect processing control corresponds to its own data buffer. From the perspective of a special effect processing control, after receiving the image frame processed by the previous special effect, the image frame processed by the previous special effect can be pulled from its own data buffer, and the image frame processed by the previous special effect can be processed by the special effect, and the processed image frame can be filled back into its own data buffer. The image frame processed by the special effect processing control backfilled by the special effect processing control is passed to the next special effect processing control. This process can be considered as a process of performing special effect processing on the collected image frames to be processed in turn based on multiple special effect processing controls. This process is a process of continuously pulling image frames and filling them back, and the pulling and filling processes are for different data buffers.

[0133] Among them, satisfying the display condition can be understood as the special effect image frames in the data cache area satisfying the requirements of the target special effect, or it can be considered that all the target special effect image frames have been backfilled in the data cache area. Whether the display condition is satisfied can be actively monitored or passively informed. When it is detected or monitored that the data cache area has stored the special effect image frames involved in the target special effect and has completed the target special effect processing, it is considered that the display condition is satisfied.

[0134] Continuing with the above description, when the display conditions are met, the target special effect image frame can be pulled from the data buffer area. The target special effect image frame can be considered as an image frame obtained by performing special effect processing of the collected image frame to be processed with the target special effect, and the target special effect image frame is displayed.

[0135] It is clear that the target special effect may be composed of one or more objects. If the target special effect is composed of only one object, after detecting that the backfill of the target special effect image frame to the corresponding data buffer is completed, the target special effect image frame is pulled from the data buffer to display the target special effect image frame when the display conditions are met. If the target special effect is composed of multiple objects, it is also necessary to determine whether the special effect processing of multiple special effect processing controls associated with all target special effects has been performed and whether there are backfilled image frames in the data buffer. Only when the above conditions are met, the target special effect image frame will be further pulled from the data buffer to display the target special effect image frame when the display conditions are met.

[0136] The above technical solution constructs a data cache area relative to the target special effects. The data cache area can be regarded as a common area for caching image frames, so that the special effects processing engine can store image frames in the data cache area, and can also pull image frames from the data cache area, thereby ensuring that the special effects processing engine can collect image frames to be processed, perform special effects processing on the collected image frames to be processed, and display the target special effects image frames after special effects processing. The reason why the special effects processing engine can perform special effects processing on the image frames to be processed and display the target special effects image frames after special effects processing is because it is associated with a data cache area, and the data can fall into the data cache area, and it can be obtained when the next logic is executed. In this way, the integrated presentation of image frame collection, special effects processing, and target special effects image display after special effects processing is realized. The above technical solution only needs to be directly based on the special effects processing engine to realize the functions of image acquisition, special effects processing on the acquired images, and displaying the target special effects image frames after special effects processing. It no longer needs to use additional special effects processing controls to add special effects to the collected data. It realizes the processing and display of special effects, simplifies the special effects processing process, thereby reducing the computing resources occupied by special effects processing and improving processing efficiency and usage performance.

[0137] Figure 3 Another structural diagram of a special effect processing system provided by an embodiment of the present disclosure is shown in FIG. Figure 3 As shown, further, the special effect processing engine 22 includes a processing scheduling control 221 and a special effect processing control set;

[0138] The processing scheduling control 221 is used to determine at least one target special effect processing control 222 associated with the target special effect from the special effect processing control set according to the special effect component object of the target special effect, wherein one target special effect processing control corresponds to the special effect processing of one special effect component object;

[0139] The target special effect processing control 222 is used to construct a data buffer area relative to the target special effect, wherein the data buffer area corresponds one-to-one to the target special effect processing control associated with the target special effect.

[0140] Exemplarily, the target special effect processing controls associated with the target special effect are determined to be target special effect processing control 1, target special effect processing control 2 and target special effect processing control 3. The target special effect processing control 1, target special effect processing control 2 and target special effect processing control 3 respectively correspond to a data cache area. The target special effect processing control 1, target special effect processing control 2 and target special effect processing control 3 and the corresponding data cache areas can be collectively referred to as target special effect processing.

[0141] Figure 4 Another structural diagram of a special effect processing system provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, further, the special effect processing engine 22 also includes an image acquisition control 223. When the number of target special effect processing controls associated with the target special effect is 1,

[0142] The image acquisition control 223 is used to fill the acquired image frame as the image to be processed into the data buffer area corresponding to the target special effect processing control;

[0143] The target special effect processing control 222 is used to extract the image frame to be processed from the data buffer area for special effect processing, and record the image frame after special effect processing as a target special effect image frame with the target special effect and fill it back into the data buffer area.

[0144] Figure 5 Another structural diagram of a special effect processing system provided by an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, further, the special effect processing engine 22 also includes a data transfer control 224 and a special effect display control 225.

[0145] The target special effect processing control 222 is used to send a special effect image frame transfer request to the data transfer control after detecting that the backfill of the target special effect image frame to the corresponding data buffer area is completed;

[0146] The data transfer control 224 is used to, after receiving the special effect image frame transfer request, determine that the display condition is met, pull the target special effect image frame from the data buffer area and transfer it to the special effect display control;

[0147] The special effect display control 225 is used to display the target special effect image frame.

[0148] Figure 6 Another structural diagram of a special effect processing system provided by an embodiment of the present disclosure is shown in FIG. Figure 6 As shown, further, when the number of target special effect processing controls associated with the target special effect is multiple,

[0149] The processing scheduling control 221 is used to determine the special effect processing global table of the target special effect and send it to multiple target special effect processing controls, and the special effect processing global table includes the processing order of multiple target special effect processing controls;

[0150] The target special effect processing control 221 is used to perform the following steps:

[0151] When it is detected that the corresponding data buffer is filled with an image frame to be processed, the image frame to be processed is extracted from the corresponding data buffer for special effect processing, and the intermediate special effect image frame formed after the special effect processing is backfilled into the corresponding data buffer, wherein the image frame to be processed is an image frame collected by an image collection control in the special effect processing engine, or is a previous intermediate special effect image frame formed after special effect processing by a previous special effect processing control;

[0152] When it is determined according to the special effect data global table that there is a next special effect processing control, the intermediate special effect image frame is transferred as an image frame to be processed to a data buffer area corresponding to the next special effect processing control.

[0153] For example, continue to refer to Figure 6 , target special effect processing control 1, target special effect processing control 2 and target special effect processing control 3, target special effect processing control 1, target special effect processing control 2 and target special effect processing control 3 correspond to a data buffer area respectively, assuming that the special effect processing order determined according to the special effect data global table is that target special effect processing control 1 takes precedence over target special effect processing control 2 takes precedence over target special effect processing control 3, then the image frame captured by the image acquisition control 223 is stored in the data buffer area of ​​the target special effect processing control 1 as the image frame to be processed; when the target special effect processing control 1 pulls the image frame to be processed for image processing, the generated intermediate special effect image frame is backfilled into the data buffer area of ​​the target special effect processing control 1, and the intermediate special effect image frame is passed as the image frame to be processed to the data buffer area corresponding to the target special effect processing control 2, and so on, until the target special effect processing control 3 completes the final special effect processing, and fills the processed image frame back into the data buffer area of ​​the target special effect processing control 3, which is equivalent to obtaining the target special effect image frame and can be given to the data transfer control 224.

[0154] Further, correspondingly, the target special effect processing control 222 is also used to determine that there is no next special effect processing control according to the special effect data global table, and when it is detected that the corresponding data buffer area is backfilled with an intermediate special effect image frame after special effect processing, use the intermediate special effect image frame as the target special effect image frame, and send a special effect image frame transfer request to the data transfer control in the special effect processing engine;

[0155] The data transfer control 224 is further used to, after receiving the special effect image frame transfer request, determine that the display condition is met, pull the target special effect image frame from the data buffer area and transfer it to the special effect display control in the special effect processing engine;

[0156] The special effect display control 225 is also used to display the target special effect image frame.

[0157] Figure 7 Another structural diagram of a special effect processing system provided by an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, further, the special effect processing engine also includes a basic data support control 226,

[0158] The basic data support control 226 is used to provide display attribute information for displaying the target special effect image frame, so as to display the target special effect image according to the display attribute information.

[0159] The special effects processing system provided by the embodiments of the present disclosure can execute the special effects processing method provided by any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

[0160] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0161] Figure 8 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 8 It shows an electronic device (eg, Figure 8 The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0162] like Figure 8As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 506. An edit / output (I / O) interface 505 is also connected to the bus 506.

[0163] Typically, the following devices may be connected to the I / O interface 506: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 509. The communication devices 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0164] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0165] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0166] The electronic device provided by the embodiment of the present disclosure and the special effects processing method provided by the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0167] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the special effect processing method provided by the above embodiments is implemented.

[0168] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0169] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0170] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0171] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0172] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: receives a special effect trigger operation of a target special effect through an operation interaction layer; responds to the special effect trigger operation through the following steps of a special effect processing engine: pulls an image frame to be processed from a data cache area constructed relative to the target special effect for special effect processing, forms a target special effect image frame with the target special effect and backfills it into the data cache area; pulls the target special effect image frame from the data cache area for display when a display condition is met.

[0173] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0174] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0175] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0176] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0177] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0178] According to one or more embodiments of the present disclosure, [Example 1] provides a special effects processing method, the method comprising: receiving a special effects trigger operation of a target special effect through an operation interaction layer; responding to the special effects trigger operation through the following steps of a special effects processing engine: pulling an image frame to be processed from a data cache area constructed relative to the target special effect for special effects processing, forming a target special effects image frame having the target special effect and backfilling it to the data cache area; pulling the target special effects image frame from the data cache area for display when a display condition is met.

[0179] According to one or more embodiments of the present disclosure, [Example 2] provides a special effect processing method, which can optimize the special effect processing engine including a processing scheduling control and a special effect processing control set; the method can be optimized to also include: through the processing scheduling control, according to the special effect component object of the target special effect, determining at least one target special effect processing control associated with the target special effect from the special effect processing control set, wherein one target special effect processing control corresponds to the special effect processing of one special effect component object; through the target special effect processing control, constructing a data cache area relative to the target special effect, wherein the data cache area corresponds one-to-one to the target special effect processing control associated with the target special effect.

[0180] According to one or more embodiments of the present disclosure, [Example Three] provides a special effects processing method. When the number of target special effects processing controls associated with the target special effect is 1, the method can optimize the pulling of the image frame to be processed from the data cache area constructed relative to the target special effect for special effects processing, forming a target special effects image frame with the target special effect and backfilling it into the data cache area, including: through the image acquisition control in the special effects processing engine, filling the acquired image frame as the image to be processed into the data cache area corresponding to the target special effects processing control; extracting the image frame to be processed from the data cache area through the target special effects processing control for special effects processing, and recording the image frame after special effects processing as a target special effects image frame with the target special effect and backfilling it into the data cache area.

[0181] According to one or more embodiments of the present disclosure, [Example Four] provides a special effects processing method, which can optimize the pulling of the target special effects image frame from the data cache area for display of the target special effects image frame when the display conditions are met, including: through the target special effects processing control, after detecting that the backfill of the target special effects image frame to the corresponding data cache area is completed, a special effects image frame transfer request is sent to the data transfer control in the special effects processing engine; through the data transfer control, after receiving the special effects image frame transfer request, it is determined that the display conditions are met, the target special effects image frame is pulled from the data cache area and passed to the special effects display control in the special effects processing engine; and the target special effects image frame is displayed through the special effects display control.

[0182] According to one or more embodiments of the present disclosure, [Example Five] provides a special effect processing method. When the number of target special effect processing controls associated with the target special effect is multiple, the method can optimize the pulling of the image frame to be processed from the data cache area constructed relative to the target special effect for special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data cache area, including: determining the special effect processing global table of the target special effect through the processing scheduling control and sending it to multiple target special effect processing controls, the special effect processing global table including the processing order of multiple target special effect processing controls; determining the special effect processing global table of the target special effect through the processing scheduling control; and sending it to multiple target special effect processing controls through the target The special effect processing control performs the following steps: when it is detected that the corresponding data buffer area is filled with image frames to be processed, the image frames to be processed are extracted from the corresponding data buffer area for special effect processing, and the intermediate special effect image frames formed after the special effect processing are backfilled into the corresponding data buffer area, wherein the image frames to be processed are image frames acquired by the image acquisition control in the special effect processing engine, or are the previous intermediate special effect image frames formed after special effect processing by the previous special effect processing control; when it is determined according to the special effect data global table that there is a next special effect processing control, the intermediate special effect image frame is passed as the image frame to be processed to the data buffer area corresponding to the next special effect processing control.

[0183] According to one or more embodiments of the present disclosure, [Example Six] provides a special effects processing method, which can optimize the display of the target special effects image frame by pulling the target special effects image frame from the data cache area when the display conditions are met, including: through the target special effects processing control, determining that there is no next special effects processing control according to the special effects data global table, and when it is detected that the corresponding data cache area is backfilled with an intermediate special effects image frame after special effects processing, using the intermediate special effects image frame as the target special effects image frame, and sending a special effects image frame transfer request to the data transfer control in the special effects processing engine; through the data transfer control, after receiving the special effects image frame transfer request, determining that the display conditions are met, pulling the target special effects image frame from the data cache area and passing it to the special effects display control in the special effects processing engine; and displaying the target special effects image frame through the special effects display control.

[0184] According to one or more embodiments of the present disclosure, [Example Seven] provides a special effects processing method, which can be optimized to also include: providing display attribute information for displaying the target special effects image frame through the basic data support control in the special effects processing engine, so as to display the target special effects image according to the display attribute information.

[0185] According to one or more embodiments of the present disclosure, [Example 8] provides a special effects processing system, which executes the special effects processing method as described in any one of Examples 1 to 7 above. The system includes: an operation interaction layer and a special effects processing engine; the operation interaction layer is used to receive a special effects trigger operation of a target special effect; the special effects processing engine is used to execute the following steps in response to the special effects trigger operation: pull the image frame to be processed from the data cache area constructed relative to the target special effect for special effects processing, form a target special effects image frame with the target special effect and backfill it to the data cache area; pull the target special effects image frame from the data cache area to display the target special effects image frame when the display condition is met.

[0186] According to one or more embodiments of the present disclosure, [Example 9] provides an electronic device, which includes one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the special effects processing method as described in any one of Examples 1 to 8 above.

[0187] According to one or more embodiments of the present disclosure, [Example 10] provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to execute the special effects processing method described in any one of Examples 1 to 8.

[0188] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0189] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0190] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A special effect processing method, characterized in that: include: Receive the special effect triggering operation of the target special effect through the operation interaction layer; The special effect processing engine responds to the special effect triggering operation through the following steps: Pulling the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area; When the display condition is met, the target special effect image frame is pulled from the data buffer area to display the target special effect image frame.

2. The method according to claim 1, characterized in that The special effect processing engine includes a processing scheduling control and a special effect processing control set; The method further comprises: By means of the processing scheduling control, according to the special effect component object of the target special effect, at least one target special effect processing control associated with the target special effect is determined from the special effect processing control set, wherein one target special effect processing control corresponds to the special effect processing of one special effect component object; A data buffer is constructed relative to the target special effect through the target special effect processing control, wherein the data buffer corresponds one-to-one to the target special effect processing control associated with the target special effect.

3. The method according to claim 2, characterized in that When the number of target special effect processing controls associated with the target special effect is 1, The step of pulling the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area includes: By using the image acquisition control in the special effect processing engine, the acquired image frames are filled into the data buffer area corresponding to the target special effect processing control as the image frames to be processed; The target special effect processing control extracts the image frame to be processed from the data buffer area for special effect processing, and the image frame after special effect processing is recorded as a target special effect image frame with the target special effect and is backfilled into the data buffer area.

4. The method according to claim 3, characterized in that Pulling the target special effect image frame from the data buffer area to display the target special effect image frame when the display condition is met includes: After detecting, through the target special effect processing control, that the backfilling of the target special effect image frame to the corresponding data buffer area is completed, a special effect image frame transfer request is sent to the data transfer control in the special effect processing engine; After receiving the special effect image frame transfer request through the data transfer control, it is determined that the display condition is met, the target special effect image frame is pulled from the data buffer area and transferred to the special effect display control in the special effect processing engine; The target special effect image frame is displayed through the special effect display control.

5. The method according to claim 2, characterized in that: When the number of target special effect processing controls associated with the target special effect is multiple, The step of pulling the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area includes: Determine the special effect processing global table of the target special effect through the processing scheduling control and send it to multiple target special effect processing controls, wherein the special effect processing global table includes the processing order of multiple target special effect processing controls; The following steps are performed by the target special effect processing control: When it is detected that the corresponding data buffer is filled with an image frame to be processed, the image frame to be processed is extracted from the corresponding data buffer for special effect processing, and the intermediate special effect image frame formed after the special effect processing is backfilled into the corresponding data buffer, wherein the image frame to be processed is an image frame collected by an image collection control in the special effect processing engine, or is a previous intermediate special effect image frame formed after special effect processing by a previous special effect processing control; When it is determined according to the special effect data global table that there is a next special effect processing control, the intermediate special effect image frame is transferred as an image frame to be processed to a data buffer area corresponding to the next special effect processing control.

6. The method according to claim 5, characterized in that Pulling the target special effect image frame from the data buffer area to display the target special effect image frame when the display condition is met includes: When it is determined through the target special effect processing control that there is no next special effect processing control according to the special effect data global table, and it is detected that the corresponding data buffer area is backfilled with an intermediate special effect image frame after special effect processing, the intermediate special effect image frame is used as the target special effect image frame, and a special effect image frame transfer request is sent to the data transfer control in the special effect processing engine; After receiving the special effect image frame transfer request through the data transfer control, it is determined that the display condition is met, the target special effect image frame is pulled from the data buffer area and transferred to the special effect display control in the special effect processing engine; The target special effect image frame is displayed through the special effect display control.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: The basic data support control in the special effect processing engine provides display attribute information for displaying the target special effect image frame, so that the target special effect image is displayed according to the display attribute information.

8. A special effects processing system, characterized in that: Used to execute the method according to any one of claims 1 to 7, comprising: operating an interaction layer and a special effects processing engine; The operation interaction layer is used to receive a special effect triggering operation of a target special effect; The special effect processing engine is used to execute the following steps in response to the special effect triggering operation: Pulling the image frame to be processed from the data buffer area constructed relative to the target special effect to perform special effect processing, forming a target special effect image frame with the target special effect and backfilling it into the data buffer area; When the display condition is met, the target special effect image frame is pulled from the data buffer area to display the target special effect image frame.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the special effect processing method as described in any one of claims 1-7.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the special effect processing method described in any one of 1-7 when executed by a computer processor.

Images