Dynamic effect processing method, dynamic effect processing device, electronic equipment and medium

Abstract

The invention provides a dynamic effect processing method, a dynamic effect processing device, electronic equipment and a computer readable storage medium, and belongs to the field of image processing. The method comprises the following steps: acquiring a to-be-processed image, and acquiring depth information of the to-be-processed image; the to-be-processed image comprises one or more preset objects; determining an image area of the preset object in the to-be-processed image; adjusting the image area according to the depth information to determine a dynamic effect area of the preset object in the to-be-processed image; and adding a dynamic effect to each dynamic effect region in the to-be-processed image according to the fuzzy parameter corresponding to each dynamic effect region to obtain a target image. The dynamic effect can be accurately and conveniently added to the to-be-processed image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a dynamic effect processing method, a dynamic effect processing device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In application scenarios such as games and animations, in order to provide users with a diverse and realistic visual experience, blur effects are often added to static objects to simulate dynamic effects in static images. For example, in order to simulate a real photography experience, racing games can provide players with a photo mode. In the photo mode, the tires of the captured vehicle images can present a static dynamic blur to show the true performance of the high-speed moving vehicle in the image. In the prior art, a mesh is usually set up to wrap the object to be processed (such as a tire), and a transparent material is assigned to the area where the mesh is located to offset and blend the surrounding pixels to achieve a dynamic blur effect. However, this method of setting up a mesh and calculating the object to be processed is relatively complicated, and when multiple meshes are set up, it consumes higher computing resources.

[0003] Therefore, how to conveniently and accurately add dynamic effects to objects in images in practical applications is a problem that needs to be solved urgently in the prior art.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a dynamic effect processing method, a dynamic effect processing device, an electronic device and a computer-readable storage medium, thereby overcoming the problem in the prior art that it is impossible to add dynamic effects to objects conveniently and efficiently, at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a dynamic effect processing method is provided, including: obtaining an image to be processed, and obtaining depth information of the image to be processed; the image to be processed includes one or more preset objects; determining an image area of ​​the preset object in the image to be processed; adjusting the image area according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed; determining a blur parameter corresponding to the dynamic effect area; adding a dynamic effect to the dynamic effect area in the image to be processed according to the blur parameter to obtain a target image.

[0008] According to one aspect of the present disclosure, a dynamic effect processing device is provided, including: a module for acquiring an image to be processed, used to acquire an image to be processed and acquire depth information of the image to be processed; the image to be processed includes one or more preset objects; a first area determination module, used to determine an image area of ​​the preset object in the image to be processed; a second area determination module, used to adjust the image area according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed; and a target image acquisition module, used to add dynamic effects to each of the dynamic effect areas in the image to be processed according to blur parameters corresponding to each of the dynamic effect areas, so as to obtain a target image.

[0009] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned dynamic effect processing methods by executing the executable instructions.

[0010] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned dynamic effect processing methods.

[0011] The exemplary embodiments of the present disclosure have the following beneficial effects:

[0012] Acquire an image to be processed and acquire depth information of the image to be processed; the image to be processed includes one or more preset objects; determine an image area of ​​the preset object in the image to be processed; adjust the image area according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed; add dynamic effects to each dynamic effect area in the image to be processed according to fuzzy parameters corresponding to each dynamic effect area, and obtain a target image. On the one hand, the present exemplary embodiment proposes a new dynamic effect processing method, which can realize adding dynamic blur effect to the preset object in the image to be processed by determining the dynamic effect area of ​​the preset object in the image to be processed and adding dynamic effect to the dynamic effect area in the image to be processed according to the blur parameter. Compared with the prior art that needs to set corresponding grids for different preset objects and calculate pixel offset and mixing based on different grids, in the present exemplary embodiment, the method of determining the dynamic effect area is simpler, making the dynamic effect processing flow of the image to be processed more convenient; on the other hand, the present exemplary embodiment first determines the image area of ​​the preset object in the image to be processed, and then adjusts the image area according to the depth information of the image to be processed, so as to accurately and effectively obtain the area where the dynamic effect actually needs to be added based on the depth information, thereby further ensuring the accuracy of adding dynamic effect; on the other hand, by using the present exemplary embodiment for dynamic effect processing, even if the image to be processed includes multiple preset objects, the corresponding dynamic effect area can be quickly and accurately determined, and the addition of blurred dynamic effect can be performed without consuming large computing resources, and the scope of application is relatively wide.

[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0015] Figure 1 A flow chart schematically illustrates a method for processing dynamic effects in this exemplary embodiment;

[0016] Figure 2 A schematic diagram schematically shows an image to be processed in this exemplary embodiment;

[0017] Figure 3 A schematic diagram schematically illustrates a second mask in this exemplary embodiment;

[0018] Figure 4 A schematic diagram schematically shows the effect of a third mask including a preset object, namely an edge, in this exemplary embodiment;

[0019] Figure 5 A schematic diagram schematically shows an effect diagram of a target image with a dynamic effect added in this exemplary embodiment;

[0020] Figure 6 A structural block diagram of a dynamic effect processing device in this exemplary embodiment is schematically shown;

[0021] Figure 7 An electronic device for implementing the above method in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION

[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0023] The exemplary embodiment of the present disclosure first provides a method for processing dynamic effects. The application scenario of the method of this embodiment may be: adding dynamic blur effects to static objects in the fields of animation and games.

[0024] The following is combined with Figure 1 This exemplary embodiment is further described as follows. Figure 1 As shown, the dynamic effect processing method may include the following steps S110 to S140:

[0025] Step S110, obtaining an image to be processed and obtaining depth information of the image to be processed; the image to be processed includes one or more preset objects.

[0026] Among them, the image to be processed refers to the image that needs to be processed with dynamic effects. It can be a video frame in the game video stream, or an original image used to make animations or develop games, etc. For example, in SLG (Simulation Game, strategy game) games, you can pause the game screen and use the screen to be displayed when paused as the image to be processed, add dynamic effects to it, and pause at any time in the game to obtain multiple images to be processed.

[0027] The image to be processed may include one or more preset objects, which are objects to which dynamic effects need to be added. For example, if the image to be processed includes a vehicle and a dynamic blur effect needs to be added to the vehicle, the vehicle can be used as a preset object; if the image to be processed includes a person and a horse and a dynamic blur effect needs to be added to the person when riding a horse, the person can be used as a preset object; if a dynamic blur effect needs to be added to the horse when running, the horse can be used as a preset object.

[0028] In this exemplary embodiment, depth information refers to parameter information that can be used to determine the image area of ​​a preset object in the image to be processed. It can be the actual depth value of each pixel in the image to be processed, and the actual depth value can reflect the distance between the pixel and the screen; it can also be a custom depth value predefined for each pixel in the image to be processed, for example, a custom depth value is configured for the area where the preset object is located, so that the preset object can be subsequently determined from the image to be processed based on the custom depth value.

[0029] The image to be processed can be directly obtained from the image database before the game, animation and other programs are run, or it can be obtained from the video stream while the program is running. After obtaining the image to be processed, depth information can be extracted from it. For example, the actual depth value of each pixel can be extracted from the image to be processed, or a custom depth value pre-defined for each pixel can be extracted from the cache.

[0030] Step S120, determining an image region of a preset object in the image to be processed.

[0031] The image area may be a mask area for indicating information such as the position and shape of the preset object in the image to be processed. For example, if the image to be processed includes person A, person B and other elements, and the preset objects are person A and person B, then the image area may be a mask area including person A, a mask image of person B, or a mask area including person A and person B, etc. In this exemplary embodiment, the image area of ​​the preset object in the image to be processed may be determined using depth information. For example, in some images, the actual depth value of the same preset object may be in the same depth range. By classifying the actual depth value, the preset object may be identified from the image to be processed, and the image area of ​​the preset object in the image to be processed may be further determined. Alternatively, a custom depth value may be configured in advance for the pixel points in the area where the preset object is located in the image to be processed. The custom depth value may be understood as a depth classification value, such as "00000000", "00000001", "00000010". After reading the custom depth value of each pixel point in the image to be processed, the preset object and the corresponding image area that need to be processed with dynamic effects may be determined according to the custom depth value.

[0032] Step S130: adjusting the image area according to the depth information to determine the dynamic effect area of ​​the preset object in the image to be processed.

[0033] In this exemplary embodiment, the image area can accurately reflect the information such as the position of the preset object in the image to be processed. Considering that this exemplary embodiment needs to add dynamic effects to the preset object, the area corresponding to the preset object with dynamic effects is usually not completely overlapped with its original image area. For example, when the preset object with dynamic blur has a blur effect, the edge will be extended or offset to a certain extent. Therefore, this exemplary embodiment can adjust the image area corresponding to the preset object, such as offsetting, extending or retracting the pixel area, so that the adjusted image area can have enough processing space for dynamic effects. The dynamic effect area refers to the area that actually needs to be processed with dynamic effects, such as the target mask after the initial determination of the character mask is offset. Specifically, when adjusting the image area, it can be adjusted according to the depth information. For example, the pixel point with a large actual depth value indicates that it is far away from the screen, and the dynamic effect may not be particularly obvious. A smaller adjustment can be applied to it. The pixel point with a small actual depth value indicates that it is close to the screen, and the dynamic effect may be more obvious. A larger adjustment can be applied to it.

[0034] Step S140 , adding dynamic effects to each dynamic effect area in the image to be processed according to the blur parameters corresponding to each dynamic effect area, to obtain a target image.

[0035] Among them, the blur parameter can be the blur dynamic effect intensity or the blur dynamic effect degree. The corresponding blur parameter can be determined based on the dynamic effect area. Different dynamic effect areas can correspond to different blur parameters. For example, the first dynamic effect area corresponds to the first blur parameter, and the second dynamic effect area corresponds to the second blur parameter. When the dynamic effect area of ​​character A is determined to be the first dynamic effect area, it can be determined that the first blur parameter can be used to add a blur dynamic effect to character A.

[0036] After determining the blur parameters corresponding to each dynamic area, a dynamic effect can be added to the dynamic effect area in the processed image according to the determined blur parameters. Specifically, the blur parameters can be used to calculate the pixel values ​​of the pixel points in the dynamic effect area in the processed image. For example, according to the blur intensity, Gaussian blur or radial blur is performed on the pixel values ​​of the pixel points to obtain a target image. The target image is an image with a dynamic blur effect after the dynamic effect is added to the preset object in the processed image.

[0037] In this exemplary embodiment, adding dynamic effects to the dynamic effect area may include a variety of methods, such as directly calculating the pixel values ​​of the pixel points corresponding to the dynamic effect area according to the blur parameters to obtain the target image, or calculating another blur effect mask according to the dynamic effect area and the blur parameters, and applying the blur effect mask to the image to be processed, such as fusing it with the image to be processed to obtain the target image, etc.

[0038] In an exemplary embodiment, the above step S110, obtaining the image to be processed, may include:

[0039] In response to a preset instruction, the original image to be displayed is used as the image to be processed;

[0040] After obtaining the target image, the method may further include:

[0041] The target image is displayed as a static image corresponding to the preset instruction.

[0042] This exemplary embodiment can be applied in multiple application fields such as games, animations, and the metaverse. For example, in a game scene, according to preset instructions, the original image to be displayed can be used as the image to be processed. For example, in SLG game applications, in response to a pause operation or entering a planning mode, the original game image to be displayed can be used as the image to be processed.

[0043] After obtaining the target image, the target image can be displayed as a static image corresponding to the preset instructions. For example, a static motion blur effect can be applied to the game scene. When the player performs a pause operation again or enters the planning mode, the target image with the dynamic effect added can be displayed. That is, the static image displayed at this time can have a motion blur effect.

[0044] Based on the above description, in this exemplary embodiment, an image to be processed is obtained, and depth information of the image to be processed is obtained; the image to be processed includes one or more preset objects; an image area of ​​the preset object in the image to be processed is determined; the image area is adjusted according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed; dynamic effects are added to each dynamic effect area in the image to be processed according to the blur parameters corresponding to each dynamic effect area, to obtain a target image. On the one hand, the present exemplary embodiment proposes a new dynamic effect processing method, which can realize adding dynamic blur effect to the preset object in the image to be processed by determining the dynamic effect area of ​​the preset object in the image to be processed and adding dynamic effect to the dynamic effect area in the image to be processed according to the blur parameter. Compared with the prior art that needs to set corresponding grids for different preset objects and calculate pixel offset and mixing based on different grids, in the present exemplary embodiment, the method of determining the dynamic effect area is simpler, making the dynamic effect processing flow of the image to be processed more convenient; on the other hand, the present exemplary embodiment first determines the image area of ​​the preset object in the image to be processed, and then adjusts the image area according to the depth information of the image to be processed, so as to accurately and effectively obtain the area where the dynamic effect actually needs to be added based on the depth information, thereby further ensuring the accuracy of adding dynamic effect; on the other hand, by using the present exemplary embodiment for dynamic effect processing, even if the image to be processed includes multiple preset objects, the corresponding dynamic effect area can be quickly and accurately determined, and the addition of blurred dynamic effect can be performed without consuming large computing resources, and the scope of application is relatively wide.

[0045] In an exemplary embodiment, the above-mentioned determining the image area of ​​the preset object in the image to be processed may include:

[0046] A first mask corresponding to the image to be processed is obtained; the first mask represents an image area of ​​a preset object in the image to be processed.

[0047] Among them, the mask can be understood as a mask used to block and display images, which can block part of the image content and display another part of the content. In this exemplary embodiment, the first mask can represent the image area of ​​the preset object in the image to be processed. For example, when the image to be processed is a game scene image, and the game scene includes game scene elements and character A in the game scene, the first mask can be an image area representing character A in the game scene image, wherein the area where character A is located can be displayed, for example, in a preset color or a preset form, and other areas can be "covered", such as not displaying or displaying gray, etc. In this exemplary embodiment, the first mask is a preliminary mask about the preset object, which can be understood as an accurate object mask without adding a dynamic blur effect to the preset object.

[0048] The first mask may be determined by image object recognition, semantic recognition, etc., or by extracting a custom depth value, a custom template value, or a custom classification value from the image to be processed. For example, after obtaining the image to be processed, the program may set different custom depth values ​​for the preset objects to be set with dynamic effects through the engine, and write them into the cache. For example, the program engine may be used to set the custom depth value for the preset objects to be set with dynamic effects. Figure 2 The pixel points in the pixel area where the preset object (person) to be added with dynamic effect in the image to be processed are set with a custom depth value. For example, the custom depth value can be set using binary 8-bit encoding. The custom depth value may include, "00000000 (0)", which means 00, adding no dynamic blur effect; "00000001 (1)", which means 01, adding a slow dynamic blur effect; "00000010 (2)", which means 10, adding a fast dynamic blur effect, etc. It should be noted that Figure 1 This is only used as an example. In specific applications, the category of the image to be processed, such as game images, animation images or other images, and the category and number of objects included in the image to be processed, such as 1 person, or 2 people and 1 horse, are not specifically limited in this disclosure. Through the setting of the above-mentioned custom depth value, custom depth values ​​corresponding to different categories of blurred dynamic effects can be set for different preset objects. In the actual image processing process, after obtaining the image to be processed, the custom depth value can be extracted from it, such as reading the custom depth value of the image pixel from the cache, and then the area where the different preset objects are located can be determined according to the custom depth value, that is, the first mask, and the blurred dynamic effects that need to be added, such as no special effects, slow blur effects, fast blur effects, etc. This exemplary embodiment sets multiple categories of custom depth values ​​and performs subsequent calculation processes based on the custom depth values. On the one hand, this exemplary embodiment can determine the first mask by configuring the custom depth value, which can avoid the need to make an independent network for the preset object in the prior art, and the problem of high computational overhead caused by repeated pixel calculation during calculation. On the other hand, this exemplary embodiment can distinguish the surface blur states of different preset objects through different categories of custom depth values, such as the expression of different speeds, which is more convenient and widely applicable in game applications.

[0049] In an exemplary embodiment, the above-mentioned adjusting the image area according to the depth information to determine the dynamic effect area of ​​the preset object in the image to be processed may include:

[0050] According to the depth information, offset sampling is performed for the first pixel point of the image to be processed in the first mask, and a second mask corresponding to the image to be processed is obtained according to the sampling result of the first pixel point; the second mask represents the dynamic effect area of ​​the preset object.

[0051] The first mask has a corresponding relationship with the image to be processed. For example, if the image to be processed has a pixel distribution of 10*10, the first mask also has a pixel distribution of 10*10. The first pixel point may be all the pixel points in the image to be processed. The second mask refers to the mask determined after offset sampling based on the first mask. The corresponding blur dynamic special effects can be added to the preset object through the second mask. The sampling can be the mask value of the position of the first pixel point after the offset is extracted on the first mask, such as the depth value of the first mask. The depth value can be a custom depth value. The custom depth value can also be understood as a custom template value, which can be used to indicate the category of the pixel point on the mask, such as whether it is the category of the preset object, which preset object category it is, or what kind of blur special effects need to be added, etc. The depth value can also be an actual depth value. The category is determined according to the range of the actual depth value, etc.

[0052] This exemplary embodiment can perform offset sampling for the first pixel point of the image to be processed in the first mask, and obtain a second mask corresponding to the image to be processed according to the sampling result of the first pixel point. The sampling method may include multiple methods. For example, this exemplary embodiment may use the Sobel algorithm to sample the custom depth values ​​of different types of objects, and perform edge pixel offset on the areas where different custom depth values ​​are located.

[0053] Before performing offset sampling, you can set the material to a preset material in the material details panel according to actual needs, such as setting it to a post-processing material, and then perform a specific offset sampling process.

[0054] The sampling can have a preset direction, such as graphically written in UEShader (a shader programming language in Unreal Engine), which is used to determine the second mask for adding dynamic effects at the end. The principle is to sample from the screen space in the two directions of (1, -1.5) and (-1, 1.5) according to the set pixel offset. Figure 3 A schematic diagram of a second mask determined after pixel offset sampling based on a first mask is shown.

[0055] In an exemplary embodiment, the depth information includes a depth value of a first pixel; and performing offset sampling for the first pixel of the image to be processed in the first mask according to the depth information may include:

[0056] Determine a pixel shift parameter of the first pixel according to the depth value of the first pixel;

[0057] The coordinates of the first pixel point are offset according to the pixel offset parameter, and the offset coordinates are sampled in the first mask to obtain a sampling result of the first pixel point.

[0058] Among them, the pixel offset parameter is only a parameter used to control the offset of the first pixel point, which may include the pixel offset amount, such as the number of pixels to offset, the offset direction, such as offset to the left or right, etc. According to the pixel offset parameter, the coordinates of the first pixel point can be offset, and the offset coordinates are sampled in the first mask to obtain the sampling result of the first pixel point, and the second mask corresponding to the image to be processed is obtained according to the sampling result. For example, the depth values ​​of the three first pixel points from left to right in the first mask are the first pixel point a, the depth value "02", the first pixel point b, the depth value "05", and the first pixel point c, the depth value "06". When the first pixel point a is offset, the pixel offset parameter is first determined to be offset to the right by 2 pixels, then the depth value "06" can be sampled at the first pixel point c, and "06" can be used as the depth value of the pixel point a' in the second mask corresponding to the first pixel point a. Through this offset processing, the depth value of the pixel point in the second mask can be obtained, thereby obtaining the second mask.

[0059] In an exemplary embodiment, determining the pixel shift parameter of the first pixel according to the depth value of the first pixel may include:

[0060] Determining a pixel shift degree of the first pixel point according to a depth value of the first pixel point; the pixel shift degree is negatively correlated with the depth value;

[0061] A pixel shift parameter of the first pixel is determined according to the basic shift parameter and the pixel shift degree of the first pixel.

[0062] This exemplary embodiment can determine the pixel offset degree of the first pixel point according to the depth value of the first pixel point. The larger the depth value of the first pixel point, the farther it is from the screen, the smaller the object is, the less obvious the dynamic effect to be added is, and the smaller the offset degree is. The smaller the depth value of the first pixel point, the closer it is to the screen, the larger the object is, the more obvious the dynamic effect to be added is, and the larger the offset degree is.

[0063] Then, the pixel offset parameters of the first pixel point can be determined according to the basic offset parameters and the pixel offset program of the first pixel point, wherein the basic offset parameters may include a basic offset amount, a preset offset direction, that is, a direction to offset, an offset amount, etc. The pixel offset parameters may include an actual offset amount, an actual offset direction, etc.

[0064] In this exemplary embodiment, the engine may include multiple nodes, through which the processing processes such as the collection, offset, and blurring of pixel depth values ​​can be realized. For example, the blurring setting of the custom depth value can be performed through "MF_PixelOperatorStencil", which also includes multiple nodes, such as "PxielOffset" (pixel offset sampling coordinate distance), "SingleDirection" (a 2D vector, indicating the direction of the desired pixel offset) and other parameter setting nodes. In the engine settings, by inputting the depth value of the first pixel in the first mask in the In (input) node, the pixel offset degree can be determined according to the rule of "near large and far small", and then the pixel offset parameter of the first pixel can be determined. Then, through the engine's own temporal mixing node "DitherTemproalAA", the pixel points can be sampled and mixed in frames to achieve a blurring effect.

[0065] In this exemplary embodiment, the "MF_PixelOperatorStencil" node can implement sampling of the depth value of the first pixel from the first mask, for example, by sampling a custom depth value in the manner of "offset direction*1*pixel offset value+pixel UV (two-dimensional texture coordinate) coordinates", and outputting the maximum value. In addition, this exemplary embodiment can also use the internal function "MF_StencilRemap" to sample the custom stencil value of the pixel according to the input UV.

[0066] After determining the second mask, the blur parameter corresponding to the second mask can be used to add a dynamic effect to the image area corresponding to the second mask in the processed image. For example, the pixel values ​​of the image area corresponding to the second mask can be sampled according to the blur parameter, and the average value can be taken to simulate the dynamic blur effect. According to actual needs, this exemplary embodiment can also set weights for different pixels according to the influence of different pixels on the blur effect, and further combine the pixel values ​​and corresponding weights of the pixels to perform weighted averaging to simulate the dynamic blur effect.

[0067] In an exemplary embodiment, the above dynamic effect processing method may further include:

[0068] When the preset object is blocked by other objects, the blocked area is determined and removed from the dynamic effect area; wherein the other objects are objects without adding dynamic effects.

[0069] In order to eliminate the problem of front-to-back overlap of the model, that is, when an object not affected by the blur effect blocks an object affected by the blur effect, the blocked area of ​​the object affected by the blur effect can be eliminated to prevent the occurrence of the problem of model penetration. For example, when the object closer to the screen does not need to add dynamic effects and no custom depth value is written, while the object farther away needs to add dynamic effects and write a custom depth value, this may cause errors in the blocked object. Therefore, considering this problem, this exemplary embodiment can use the "Block" node in the engine to eliminate the problem of front-to-back overlap of the model.

[0070] In an exemplary embodiment, the above step S140 may include:

[0071] Get the blur parameters corresponding to each dynamic effect area;

[0072] A third mask is obtained by performing blur processing on a second pixel point in the image to be processed according to the blur parameter; the second pixel point is a pixel point located in the dynamic effect area; and the third mask represents a blur value of the second pixel point;

[0073] The third mask is fused with the image to be processed to obtain a target image.

[0074] In this exemplary embodiment, corresponding blur parameters can be preset for different dynamic effect areas. For example, if the custom depth value of the current dynamic effect area is 1, the blur parameter corresponding to the custom depth value 1 can be used. If the custom depth value of the current dynamic effect area is 2, the blur parameter corresponding to the custom depth value 2 can be used. In this exemplary embodiment, the blur parameter corresponding to the dynamic effect area of ​​the preset object can also be determined according to the movement speed of the preset object. For example, if a person runs slowly and a vehicle runs fast, a blur parameter with a slower dynamic effect can be preset for the person, and a blur parameter with a faster dynamic effect can be preset for the vehicle. According to the preset object corresponding to the dynamic effect area, the corresponding blur parameter can be obtained.

[0075] The second pixel point is the pixel point in the image to be processed that needs to be processed with blurred dynamic effects. The points in the second mask correspond to the points in the image to be processed. After determining the second mask, that is, determining the area in the image to be processed that actually needs to add dynamic effects, a third mask with a dynamic blur value can be further provided. The third mask can be a mask image with a blur value corresponding to the image to be processed and the second mask, which includes the range in which the dynamic effects need to be added to the preset object, and also includes the effect value to which the dynamic effects need to be added. The third mask is fused with the image to be processed, for example, the third mask is mixed and calculated with the pixel points corresponding to the image to be processed, so as to obtain a target image with a dynamic blur effect.

[0076] As required, this exemplary embodiment may also set the transparency of the mixed dynamic blur effect for the third mask, and control the fusion effect of the third mask and the image to be processed by adjusting the transparency.

[0077] In an exemplary embodiment, fusing the third mask with the image to be processed to obtain a target image includes:

[0078] Determine a mixing weight corresponding to the second pixel point according to a difference between a first mask value of the second pixel point in the first mask and a second mask value in the second mask; the first mask represents an image area of ​​the preset object in the image to be processed; and the second mask represents a dynamic effect area of ​​the preset object in the image to be processed;

[0079] Based on the mixing weight of the second pixel, the third mask is fused with the image to be processed to obtain a target image.

[0080] The first mask value refers to the value in the first mask, such as the custom depth value in the first mask, and the second mask value refers to the value in the second mask, such as the custom depth value in the second mask. After the third mask is determined, the third mask can be fused with the image to be processed, that is, the fuzzy value in the third mask is applied to the corresponding area of ​​the image to be processed, so as to add a dynamic effect to the area where the preset object is located in the image to be processed, for example, the fuzzy value in the third mask is mixed with the corresponding pixel point in the image to be processed, and the mixed calculation can be to fuse the third mask with the image to be processed according to the mixed weight.

[0081] Considering that in actual applications, the pixel points in the area where the preset object is located can be configured with a custom depth value to determine the first mask, but the edge and surrounding of the preset object may not be set with a custom depth value, so that after the second mask is generated by pixel offset based on the first mask, the difference between the first mask value and the second mask value in some areas is large, and the difference between the first mask value and the second mask value in some areas is small. This exemplary embodiment can determine the mixing weight corresponding to the second pixel point according to the degree of difference between the first mask value and the second mask value in the area, so as to use different mixing weights for different areas to perform mixing calculations, so as to ensure the effectiveness of adding dynamic effects and achieve the effect of clear main body and blurred edges of the preset object. Specifically, this exemplary embodiment can determine the mixing weight of the second pixel in the dynamic effect area according to the difference between the first mask value and the second mask value of the second pixel. For example, when the first mask is sampled out through the custom depth value as mentioned above, the custom depth value of the preset object itself is fixed. The mixing weight of the second pixel is determined according to the difference between the first mask value and the second mask value of the second pixel, and based on the mixing weight, targeted mixing processing is performed on areas with different degrees of mask value differences. After the mixing calculation, the image values ​​of the pixels at the edge and surrounding of the preset object that do not have custom depth values ​​will be weakened, that is, the difference in the main area is small and the difference at the edge is large, resulting in a clear main body and blurred edges. Figure 4 A schematic diagram of the effect of a third mask including a preset object and an edge is shown.

[0082] In an exemplary embodiment, after determining the dynamic effect area of ​​the preset object in the image to be processed, the dynamic effect processing method further includes:

[0083] According to the depth information, the dynamic effect areas of the multiple preset objects are sorted according to the depth values. If there are overlapping areas in the dynamic effect areas of the multiple preset objects, it is determined according to the sorting result that the overlapping area belongs to the dynamic effect area with the smallest depth value.

[0084] In this exemplary embodiment, in order to ensure the accuracy of determining the dynamic effect area and prevent errors in Z sorting, the first mask or the second mask can be depth sampled, compared and sorted. When it is determined that there are overlapping areas in the dynamic effect areas of multiple preset objects, it can be determined based on the sorting results that the overlapping area belongs to the dynamic effect area with the smallest depth value.

[0085] In another exemplary embodiment, when calculating pixel blending, an average value can be used to simulate the dynamic blur effect. The specific blending calculation can also be implemented by a processing engine, which includes a node for pixel blending effects, wherein viewportUV is a custom node in HLSL (High Level Shader Language), and the following process can be completed by writing relevant code: Calculate the multiple of the screen space for converting texture coordinates to screen coordinates; define a texture index for specifying the texture to be sampled in the SceneTextureLookup function; define an array for storing sample offsets, which determines the range and intensity of the blur; calculate the direction vector from the center of the screen (0.5, 0.5) to the current pixel; initialize the accumulated result to a zero vector, and accumulate the color values ​​of all samples; traverse all sample offsets; calculate the position of the current sample, and multiply the offset by the blur distance (BlurDist) and the direction vector; limit the sample position to the screen range (0.0 to 1.0); sample the scene texture at the specified position and accumulate the result to sum; calculate the average value of all samples to obtain the blurred color value.

[0086] Among them, RadialGradientExponential can be a node that comes with the engine. It can create a radial gradient process based on the input UV coordinates, and apply an exponential density function of a specified density to affect the gradient effect, thereby achieving a radial blur effect from the center to the outside.

[0087] This exemplary embodiment can achieve a blur effect for a specific custom depth value by mixing the screen pixel color and the blurred screen pixel color according to the custom depth value of the pixel point, and distinguish the effects of different custom depth values ​​based on this method. In addition, this exemplary embodiment can also eliminate the effects of non-customized custom depth value areas through other nodes in the engine to achieve the correct calculation effect. Figure 5 A schematic diagram showing the effect of adding a dynamic effect to a target image is shown.

[0088] The exemplary embodiment of the present disclosure also provides a dynamic effect processing device. Figure 6The device 600 may include a module 610 for acquiring an image to be processed, for acquiring the image to be processed and the depth information of the image to be processed; the image to be processed includes one or more preset objects; a first area determination module 620, for determining the image area of ​​the preset object in the image to be processed; a second area determination module 630, for adjusting the image area according to the depth information to determine the dynamic effect area of ​​the preset object in the image to be processed; and a target image acquisition module 640, for adding dynamic effects to each dynamic effect area in the image to be processed according to the fuzzy parameters corresponding to each dynamic effect area, to obtain a target image.

[0089] In an exemplary embodiment, the first area determination module includes: a first mask acquisition unit, used to acquire a first mask corresponding to the image to be processed; the first mask represents the image area of ​​the preset object in the image to be processed.

[0090] In an exemplary embodiment, the second area determination module includes: a pixel sampling unit, which is used to offset sample the first pixel of the image to be processed in the first mask according to the depth information, and obtain a second mask corresponding to the image to be processed according to the sampling result of the first pixel; the second mask represents the dynamic effect area of ​​the preset object.

[0091] In an exemplary embodiment, the depth information includes a depth value of a first pixel; a pixel sampling unit includes: an offset parameter determination subunit, used to determine a pixel offset parameter of the first pixel according to the depth value of the first pixel; an offset sampling subunit, used to offset the coordinates of the first pixel according to the pixel offset parameter, and sample the offset coordinates in a first mask to obtain a sampling result of the first pixel.

[0092] In an exemplary embodiment, the offset parameter determination subunit includes: an offset degree determination subunit, which is used to determine the pixel offset degree of the first pixel point according to the depth value of the first pixel point; the pixel offset degree is negatively correlated with the depth value; and an offset parameter determination subunit, which is used to determine the pixel offset parameter of the first pixel point according to the basic offset parameter and the pixel offset degree of the first pixel point.

[0093] In an exemplary embodiment, a target image acquisition module includes: a blur parameter acquisition unit, used to obtain blur parameters corresponding to each dynamic effect area; a third mask determination unit, used to blur a second pixel point in the image to be processed according to the blur parameters to obtain a third mask; the second pixel point is a pixel point located in the dynamic effect area; the third mask represents the blur value of the second pixel point; and a target image acquisition unit, used to merge the third mask with the image to be processed to obtain a target image.

[0094] In an exemplary embodiment, a target image obtaining unit includes: a mixing weight determination subunit, used to determine the mixing weight corresponding to the second pixel point based on the difference between the first mask value of the second pixel point in the first mask and the second mask value in the second mask; the first mask represents the image area of ​​the preset object in the image to be processed; the second mask represents the dynamic effect area of ​​the preset object in the image to be processed; and a target image obtaining subunit, used to fuse the third mask with the image to be processed based on the mixing weight of the second pixel point to obtain the target image.

[0095] In an exemplary embodiment, after determining the dynamic effect area of ​​the preset object in the image to be processed, the dynamic effect processing device also includes: an area determination unit, which is used to sort the dynamic effect areas of multiple preset objects according to depth values ​​based on depth information; if there are overlapping areas in the dynamic effect areas of multiple preset objects, it is determined according to the sorting result that the overlapping area belongs to the dynamic effect area with the smallest depth value.

[0096] In an exemplary embodiment, the module for acquiring an image to be processed includes: an instruction response unit, which is used to respond to a preset instruction and use the original image to be displayed as the image to be processed; after obtaining the target image, the dynamic effect processing device also includes: a display unit, which is used to display the target image as a static image corresponding to the preset instruction.

[0097] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0098] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0099] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the dynamic effect processing method described above is implemented.

[0100] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0101] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0102] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0103] Computer programs can be carried or transmitted through electrical, magnetic, optical, electromagnetic, infrared and other signals. Electronic devices can convert signals carrying computer programs into digital signals to run computer programs. When a computer program runs on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device can execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned dynamic effect processing method, which includes the following steps: obtaining an image to be processed, and obtaining depth information of the image to be processed; the image to be processed includes one or more preset objects; determining the image area of ​​the preset object in the image to be processed; adjusting the image area according to the depth information to determine the dynamic effect area of ​​the preset object in the image to be processed; adding dynamic effects to each dynamic effect area in the image to be processed according to the blur parameters corresponding to each dynamic effect area, and obtaining a target image.

[0104] The above method steps are executed by a computer program. On the one hand, this exemplary embodiment proposes a new dynamic effect processing method. By determining the dynamic effect area of ​​the preset object in the image to be processed and adding the dynamic effect to the dynamic effect area in the image to be processed according to the blur parameter, the dynamic blur effect can be added to the preset object in the image to be processed. Compared with the prior art that needs to set corresponding grids for different preset objects and calculate pixel offset and mixing based on different grids, in this exemplary embodiment, the method for determining the dynamic effect area is simpler, making the dynamic effect processing flow of the image to be processed more convenient; on the other hand, this exemplary embodiment determines the image area of ​​the preset object in the image to be processed first, and then adjusts the image area according to the depth information of the image to be processed. It can accurately and effectively obtain the area where the dynamic effect actually needs to be added based on the depth information, further ensuring the accuracy of adding the dynamic effect; on the other hand, by using this exemplary embodiment for dynamic effect processing, even if the image to be processed includes multiple preset objects, the corresponding dynamic effect area can be quickly and accurately determined, and the addition of blurred dynamic effect can be performed without consuming large computing resources, and the scope of application is relatively wide.

[0105] The exemplary embodiment of the present disclosure also provides an electronic device. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the method in the exemplary embodiment by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0106] Reference below Figure 7 , the electronic device is exemplarily described in the form of a general-purpose computing device. It should be understood that Figure 7 The electronic device 700 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0107] like Figure 7 As shown, the electronic device 700 may include: a processor 710 , a memory 720 , a bus 730 , an I / O (input / output) interface 740 , a network adapter 750 , and a display 760 .

[0108] The memory 720 may include a volatile memory, such as a RAM 721, a cache unit 722, and may also include a non-volatile memory, such as a ROM 723. The memory 720 may also include one or more program modules 724, such program modules 724 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 724 may include each module in the above-mentioned device.

[0109] The processor 710 may include one or more processing units. For example, the processor 710 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0110] The processor 710 may be used to execute executable instructions stored in the memory 720, such as executing the above-mentioned dynamic effect processing method.

[0111] The bus 730 is used to realize the connection between different components of the electronic device 700, and may include a data bus, an address bus, and a control bus.

[0112] The electronic device 700 can communicate with one or more external devices 800 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 740 .

[0113] The electronic device 700 can communicate with one or more networks through the network adapter 750. For example, the network adapter 750 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 750 can communicate with other modules of the electronic device 700 through the bus 730.

[0114] The electronic device 700 can display a graphical user interface through the display 760, such as displaying a game editing scene.

[0115] although Figure 7 Not shown, other hardware and / or software modules may also be provided in the electronic device 700, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID system, a tape drive, and a data backup storage system.

[0116] As can be seen from the above, the technical solution of the present disclosure can be implemented as a method, an apparatus, a system, a computer program product, a storage medium, an electronic device, etc. Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, such as being respectively referred to as a "circuit", "module" or "system".

[0117] It should be understood that the present disclosure is not limited to the specific method steps or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Those skilled in the art will easily think of other embodiments based on the specific embodiments provided by the present disclosure. Therefore, the specific embodiments provided by the present disclosure are only exemplary, and the scope and spirit of the present disclosure are indicated by the claims, and any variations, uses or adaptive changes of the present disclosure should be covered, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure.

[0118] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0119] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A dynamic effect processing method, characterized in that: include: Acquire an image to be processed, and acquire depth information of the image to be processed; The image to be processed includes one or more preset objects; Determine an image area of ​​the preset object in the image to be processed; Adjusting the image area according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed; Dynamic effects are added to each of the dynamic effect areas in the image to be processed according to blur parameters corresponding to each of the dynamic effect areas to obtain a target image.

2. The method according to claim 1, characterized in that The determining the image area of ​​the preset object in the image to be processed includes: A first mask corresponding to the image to be processed is obtained; the first mask represents an image area of ​​the preset object in the image to be processed.

3. The method according to claim 2, characterized in that The step of adjusting the image area according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed includes: According to the depth information, offset sampling is performed for the first pixel point of the image to be processed in the first mask, and a second mask corresponding to the image to be processed is obtained according to the sampling result of the first pixel point; the second mask represents the dynamic effect area of ​​the preset object in the image to be processed.

4. The method according to claim 3, characterized in that The depth information includes a depth value of the first pixel; and performing offset sampling for the first pixel of the image to be processed in the first mask according to the depth information includes: determining a pixel offset parameter of the first pixel according to the depth value of the first pixel; The coordinates of the first pixel point are offset according to the pixel offset parameter, and the offset coordinates are sampled in the first mask to obtain a sampling result of the first pixel point.

5. The method according to claim 4, characterized in that The determining a pixel offset parameter of the first pixel point according to the depth value of the first pixel point includes: Determining a pixel shift degree of the first pixel point according to the depth value of the first pixel point; the pixel shift degree is negatively correlated with the depth value; A pixel shift parameter of the first pixel point is determined according to a basic shift parameter and a pixel shift degree of the first pixel point.

6. The method according to claim 1, characterized in that The adding a dynamic effect to each of the dynamic effect regions in the image to be processed according to the blur parameters corresponding to each of the dynamic effect regions to obtain a target image includes: Obtaining blur parameters corresponding to each of the dynamic effect areas; A second pixel point in the image to be processed is blurred according to the blur parameter to obtain a third mask; the second pixel point is a pixel point located in the dynamic effect area; and the third mask represents a blur value of the second pixel point; The third mask is fused with the image to be processed to obtain the target image.

7. The method according to claim 6, characterized in that The step of fusing the third mask with the image to be processed to obtain the target image includes: Determine a mixing weight corresponding to the second pixel point according to a difference between a first mask value of the second pixel point in the first mask and a second mask value in the second mask; the first mask represents an image area of ​​the preset object in the image to be processed; and the second mask represents a dynamic effect area of ​​the preset object in the image to be processed; Based on the mixing weight of the second pixel point, the third mask is fused with the image to be processed to obtain the target image.

8. The method according to claim 1, characterized in that After determining the dynamic effect area of ​​the preset object in the image to be processed, the method further includes: According to the depth information, the dynamic effect areas of the plurality of preset objects are sorted according to the depth values. If there are overlapping areas in the dynamic effect areas of the plurality of preset objects, it is determined according to the sorting result that the overlapping area belongs to the dynamic effect area with the smallest depth value.

9. The method according to claim 1, characterized in that: The step of obtaining the image to be processed comprises: In response to a preset instruction, the original image to be displayed is used as the image to be processed; After obtaining the target image, the method further includes: The target image is displayed as a static image corresponding to the preset instruction.

10. A dynamic effect processing device, characterized in that: include: The module for acquiring an image to be processed is used to acquire the image to be processed and obtain the depth information of the image to be processed; The image to be processed includes one or more preset objects; A first region determination module, used to determine an image region of the preset object in the image to be processed; A second area determination module, used for adjusting the image area according to the depth information to determine a dynamic effect area of ​​the preset object in the image to be processed; The target image acquisition module is used to add dynamic effects to each of the dynamic effect areas in the image to be processed according to the fuzzy parameters corresponding to each of the dynamic effect areas to obtain a target image.

11. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the dynamic effect processing method described in any one of claims 1-9 by executing the executable instructions.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the dynamic effect processing method described in any one of claims 1 to 9 is implemented.

Images