Method, apparatus, device and storage medium for image rendering

Abstract

According to an embodiment of the present disclosure, a method, an apparatus, a device, and a storage medium for image rendering are provided. The method described herein includes: dividing a captured image into multiple regions, the multiple regions corresponding to multiple augmented reality categories, and the multiple augmented reality categories respectively having corresponding rendering priorities; and rendering the multiple regions based on the rendering priorities, so that in the process of rendering a first augmented reality effect for a first region among the multiple regions, rendering a second region in the multiple regions is avoided, wherein the first augmented reality category associated with the first region is different from the second augmented reality category associated with the second region. In this way, multiple augmented reality special effects can be rendered for multiple regions of the image in sequence based on the rendering priorities, and occlusion or overlap between different augmented reality special effects can be avoided.

Description

Technical Field

[0001] Example embodiments of the present disclosure relate generally to the field of computers, and in particular, to methods, apparatuses, devices, and computer-readable storage media for image rendering. Background Art

[0002] Augmented reality (AR) is a technology that combines the real world with virtual content, and has been widely used in many fields such as artificial intelligence, robotics, education, medical care, and film and television production. With the help of AR technology, virtual content can be superimposed on real content. For example, AR scenes can be made with reference to scenes in the real world, so that users who watch AR scenes can get an immersive feeling. In order to provide users with a rich visual experience, it may be necessary to present more than one AR special effect for different scenes. In some cases, human objects can also be presented as part of the scene, and even interact with one or more AR special effects in the scene. Summary of the invention

[0003] In a first aspect of the present disclosure, an image rendering method is provided. The method includes: dividing a captured image into a plurality of regions, the plurality of regions corresponding to a plurality of augmented reality (AR) categories, the plurality of AR categories respectively having corresponding rendering priorities; and rendering the plurality of regions based on the rendering priorities, so that in the process of rendering a first AR effect for a first region among the plurality of regions, rendering of a second region among the plurality of regions is avoided, wherein a first AR category associated with the first region is different from a second AR category associated with the second region.

[0004] In a second aspect of the present disclosure, a device for image rendering is provided. The device includes: an image segmentation module configured to segment a captured image into multiple regions, the multiple regions corresponding to multiple augmented reality AR categories, and the multiple AR categories respectively have corresponding rendering priorities; and an image rendering module configured to render the multiple regions based on the rendering priorities, so that in the process of rendering a first AR effect for a first region among the multiple regions, rendering a second region among the multiple regions is avoided, wherein a first AR category associated with the first region is different from a second AR category associated with the second region, and a rendering priority of the second AR category is different from a rendering priority of the first AR category.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the electronic device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the medium, and when the program is executed by a processor, the method of the first aspect is implemented.

[0007] It should be understood that the contents described in this summary of the invention are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0010] Figure 2 A flowchart of an image rendering process according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram of a user interface for image rendering according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic structural block diagram of an apparatus for image rendering according to some embodiments of the present disclosure is shown; and

[0013] Figure 5 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0016] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0017] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0018] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0019] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information is sent to the user in a manner such as a pop-up window, in which the prompt information can be presented in text form. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0021] AR technology has been widely used in many fields. As an exemplary application, in video shooting and production scenes, users can overlay AR effects on images to enrich visual effects. For example, users can use AR technology to add virtual objects to the presented image, or present human objects in a virtual scene, or even interact with one or more AR effects in the scene. In some cases, the image captured by the user is dynamic rather than static. For example, the image can change with the direction and position of the user's shooting device. In this case, it may be desirable for the AR effect presented to change accordingly. In addition, it may be desirable to present more than one AR effect in the same image.

[0022] In traditional image rendering technology, AR effects are post-processed, which may cause part of the AR effect to be covered or blocked by another part of the AR effect. In order to make multiple AR effects coexist in the same image, a single AR effect can be rendered in the original image separately, and then multiple intermediate images can be superimposed together to obtain the final image. This method will generate a large amount of redundant data, and there may be a situation where multiple AR effects overlap in the final image. If there are human objects in the image, further processing is required for the occlusion relationship between the human objects and the AR special effects, which increases the rendering complexity. Therefore, it is expected to improve the image rendering technology.

[0023] According to an embodiment of the present disclosure, a template test is used to render special effects for an area in an image. After the area rendering is completed, a post-processing method is used to render foreground special effects such as virtual objects and human objects. In this way, multiple AR special effects can be rendered in the same image without generating redundant data, and occlusion and overlap between multiple AR special effects can be avoided.

[0024] The embodiments of the present disclosure will be described below in conjunction with exemplary application scenarios. However, it should be understood that the provided image rendering solution is applicable to various scenarios involving AR technology, including but not limited to artificial intelligence, medical applications, map drawing, film and television animation, and the like.

[0025] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. The example environment 100 may include a terminal device 110 and an editing device 120. It should be understood that in some embodiments, the terminal device 110 and the editing device 120 may be as follows: Figure 1 The terminal device 110 and the editing device 120 are shown as being implemented as independent devices. However, depending on the device capabilities, in other embodiments, the terminal device 110 and the editing device 120 may be integrated as a single electronic device. Therefore, the embodiments of the present disclosure are not limited in this respect.

[0026] The terminal device 110 can capture and present images. In an embodiment of the present disclosure, the image may include a static image and a dynamic image, for example, a video image. An application 112 for image capture and production may be installed on the terminal device 110. The application 112 may provide virtual special effects for image rendering. The virtual special effects may be, for example, effects produced based on AR technology, which are also referred to as AR effects hereinafter. In an embodiment of the present disclosure, AR effects include, but are not limited to, background effects, foreground effects, and the like.

[0027] Accordingly, the terminal device 110 can present an AR effect on the captured image through the application 112. Figure 1As shown, the terminal device 110 captures an original image 102 in which there is a human object 104. The terminal device 110 can add background effects 131 to 133 and 141 to 145 and a foreground effect 106 to the image 102 through the application 112, and in turn obtain an image 122 for rendering.

[0028] The terminal device 110 can communicate with the editing device 120. In some embodiments, the editing device 120 can be used to render an image. Additionally or alternatively, the editing device 120 can edit virtual special effects used to render the image. In such an embodiment, the terminal device 110 can obtain the virtual special effects from the editing device 120 and add them to the rendered image through the application 112.

[0029] Alternatively, in some other embodiments, the terminal device 110 may send the original image 102 to the editing device 120 for rendering virtual special effects. Accordingly, the terminal device 110 obtains the rendered image 122 from the editing device 120 for presentation.

[0030] The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of interface for the user (such as "wearable" circuits, etc.). The editing device 120 is a computing system / server of various types that can provide computing power, including but not limited to mainframes, edge computing nodes, electronic devices in cloud environments, and the like.

[0031] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0032] In the description below, the embodiments will be described with reference to example states of several user interfaces (UIs). It should be understood that these UIs and interactions are merely illustrative, and various interface designs and interaction methods may actually exist. In addition, the controls included in these UIs may be implemented with any currently known or future developed UI elements and technologies. In addition, the types, forms, modes of operation, layouts in the UIs, arrangements, etc. of these controls are all schematic, and are not intended to limit the scope of the present disclosure in any form.

[0033] Figure 21 shows a flowchart of an image rendering process 200 according to some embodiments of the present disclosure. The process 200 may be implemented at the terminal device 110 or the editing device 120 or at any suitable device. Figure 1 The editing device 120 is combined with Figure 3 The process 200 is described. However, it should be understood that in other embodiments, the process 200 may be implemented by the terminal device 110 or by an electronic device integrated with the terminal device 110 or the editing device 120. The embodiments of the present disclosure are not limited in this respect.

[0034] As mentioned above, AR effects may include background effects, foreground effects, etc., depending on the type of AR effect. Figure 1 For example, the background effects 131 to 133 and 141 to 145 are presented in the background area of ​​the image, and different background effects can be rendered for different background areas, while placement effects such as box 106 are presented in the foreground area. In general, the background effects will not block or cover the foreground effects. In addition, since there is a human object 104 in the image, the blocking relationship between the human object 104 and the foreground effects needs to be considered.

[0035] In order to render multiple AR effects in the same image and avoid different AR effects overlapping each other or human objects being unnecessarily blocked by AR effects, in process 200, segmentation-based rendering can be performed under template testing, and then post-processing is performed on foreground effects and human objects.

[0036] In an embodiment of the present disclosure, the term "template buffer" refers to a data buffering technology in three-dimensional drawing, in which a byte-length value can be assigned to each pixel in units of pixels, which is also referred to as the value of the template buffer below. The template buffer can be used to limit the area of ​​rendering. The terms "template test" and "post-processing" refer to different stages in the rendering pipeline. In the template test stage, the fragment of the picture is set to an unsigned integer value, which is also referred to as a reference value. Then, the reference value of the fragment is compared with the value of the template buffer. If the comparison passes, the color value of the corresponding fragment is updated. If the comparison fails, the color value of the fragment is not updated. Post-processing refers to processing such as ambient occlusion, depth of field, motion blur, etc. for the image after all rendering is completed.

[0037] In block 210 , the editing device 120 divides the captured image 102 into a plurality of regions corresponding to a plurality of AR categories, each of which has a corresponding rendering priority.

[0038] In some embodiments, the editing device 120 may segment the plurality of regions based on image semantics. Figure 1In the example of , by recognizing the semantics of the image 102, the sky portion can be segmented into a first region 130, and the ground portion can be segmented into a second region 140, wherein the first region 130 and the second region 140 correspond to different AR categories, i.e., a first AR category and a second AR category. Different AR categories can have corresponding AR effects. For example, the first AR category has first AR effects 131 to 133 for rendering the sky, and the second AR category has second AR effects 141 to 145 for rendering the ground.

[0039] In box 220, the editing device 120 renders multiple areas based on the rendering priority, so that in the process of rendering the first AR effects 131 to 133 for the first area 130 among the multiple areas, rendering the second area 140 among the multiple areas is avoided, wherein the first AR category associated with the first area 130 is different from the second AR category associated with the second area.

[0040] In some embodiments, the editing device 120 may use a template test to render multiple regions of the image 102. In this way, segmented rendering of the image may be achieved, wherein the AR effect corresponding to each AR category is rendered only in the respective segmented region and cannot be rendered in other regions.

[0041] In some embodiments, the editing device 120 may create a corresponding AR node for each AR category, and each AR node may be associated with at least one model object. The model object may be rendered within the corresponding segmented area, but cannot be rendered outside the segmented area.

[0042] Figure 3 A schematic diagram of a user interface 300 for image rendering according to some embodiments of the present disclosure is shown. The user interface 300 may be an editing interface on the editing device 120, and includes a visual authoring area 310 for presenting a template buffer corresponding to the image 102, an area 312 for presenting an image preview, an area 320 for presenting special effect information, an area 330 for presenting parameter information of special effects, and an area 340 for presenting model rendering information. Figure 3 As shown, for the first AR category and the second AR category, the editing device 120 creates two AR nodes, wherein the first AR node is “AR segmentation class node: sky” and the second AR node is “AR segmentation class node: ground”.

[0043] In some embodiments, the editing device 120 may designate a stencil buffer corresponding to the first region 130 as a first reference value, wherein the first reference value is used to indicate a first AR category and is different from reference values ​​designated for other regions other than the first region in the image 102. The editing device 120 may render the first AR effects 131 to 133 for the first region 130 using a first renderer associated with the first AR category, and the first renderer renders the first AR effects 131 to 133 only for regions having the first reference value.

[0044] like Figure 3 As shown in the area 310 in FIG. 1 , the editing device 120 may designate the template buffer corresponding to the first area 130 as a first reference value (e.g., 1), while other areas are designated as different reference values. The first renderer may render the first AR effects 131 to 133 based on the first reference value, while avoiding rendering the second reference value.

[0045] Taking the first area 130 as an example, an exemplary implementation of the first renderer is given below. In this implementation, the mesh component (Mesh) can adopt a plane, and the depth test and depth writing are turned off, so that the first renderer is only used to fill the template without affecting other objects in the image 102. After turning on the template test, the operation after the template test passes can be set to fill the corresponding area according to the image segmentation, for example, set to "replace". The rule for passing the template test can be set to "always", that is, the template test is always passed. In addition, the reference value of the template test corresponding to the first area 130 is specified as a reference value that is different from other segmented areas. Additionally, since there is no need to write to the color buffer, the colorWriteMask of the mixed mode can be set to 0. It should be understood that the above settings are only examples, and in practice, any other appropriate settings can be adopted. Therefore, the embodiments of the present disclosure are not limited in this respect.

[0046] In some embodiments, the first AR category may be associated with at least one model object. In response to one of the at least one model object being located in the first area 130, the editing device 120 may render the model object using a first AR effect. Figure 3 In the example of , the two AR nodes have models 11 to 1N and models 22 to 2N respectively. Accordingly, the models 11 to 1N are only rendered in the first area 130 but cannot be rendered in the second area 140 .

[0047] Additionally or alternatively, in some embodiments, different AR nodes may employ corresponding shaders.

[0048] In some embodiments, the editing device 120 can recursively traverse all model objects with renderers of the first AR node. As an exemplary implementation, after the template test is turned on, the reference value of the template test of the corresponding model object is set to the reference value of the current AR node, that is, the first reference value. In addition, the operation after the template test passes can be set to "keep", and the rule for the template test to pass can be set to not render the unsegmented part of the image, for example, it can be set to "equal". Of course, in practice, any other appropriate settings can also be adopted as needed.

[0049] In some embodiments, a corresponding AR node may be set for all segmented regions of the image 102 in a manner similar to the first region 130. Figure 3 In the example shown, the editing device 120 may specify the template buffer corresponding to the second area 140 as a second reference value (i.e., 0). The second reference value is used to indicate the second AR category, and is different from the reference values ​​specified for other areas in the image 102 except the second area 140, for example, the reference value 0 of the second area 140 is different from the reference value 1 of the first area 130. Since the rendering priority of the first AR category is higher than the rendering priority of the second AR category, according to an embodiment of the present disclosure, the editing device 120 may use the second renderer associated with the second AR category to render the second AR effect for the second area 140 on the image in which the first area has been rendered, and the second renderer only renders the second AR effect for the area with the second reference value. In this way, by rendering the segmented areas of the image in sequence according to the rendering priority, redundant data can be avoided, storage space can be saved, and rendering efficiency can be improved.

[0050] like Figure 3 As shown, the user interface 300 also includes controls 321 and 322 for selecting or adding AR nodes, respectively. It should be understood that the parameters and settings, interface layout and controls shown in the interface 300 are merely exemplary, and in actual applications, any appropriate interface may be used.

[0051] Therefore, according to the embodiments of the present disclosure, in the process of making rendering special effects or rendering images, by creating new AR nodes, specific AR effects can be added to the corresponding areas of the image. Moreover, the newly created AR effects will not affect the existing AR effects. Compared with the traditional post-processing method, the image rendering effect is enhanced and the special effects production process is simplified.

[0052] In some embodiments, for multiple AR categories, the rendering priority can be determined based on the scene tree of the image 102. The scene tree can be determined, for example, based on a predetermined rule, or alternatively, predefined by a user. For example, the rendering priority can be in the order of sky, building, ceiling, ground, human object, virtual object object. By determining the rendering order of different AR segmentation class nodes by rendering priority, it is possible to avoid the special effects rendering of one area from unnecessarily covering or blocking the special effects rendering of another area.

[0053] After completing the segmentation class rendering, the editing device 120 may render a foreground effect and / or a human object for the image 102. In some embodiments, after rendering the plurality of regions 130 and 140, the editing device 120 may render at least one foreground effect in the image 102 based on post-processing. As an example, the foreground effect 106 may correspond to a third AR category, for example, Figure 3 The “AR placement class node” shown in FIG. The AR placement class node may have at least one model 31 to 3N. In other words, in an embodiment of the present disclosure, a template test may be used for a segmentation class node, while a conventional rendering process may be used for other nodes such as foreground effects.

[0054] In some embodiments, after rendering the foreground effect, the editing device 120 may render the person object 104 in the image 102 based on the predetermined presentation relationship. As an exemplary implementation, the editing device 120 may cut out the person object 104 and perform post-processing on it. In some embodiments, the person object 104 may correspond to Figure 3 The “AR segmentation class node: person” shown in FIG. 5 , however, the AR segmentation class node for the person object does not use the template test.

[0055] As an exemplary implementation, the renderer of the AR segmentation class node for the character object 104 may adopt the following settings: the mesh component (Mesh) may adopt a plane, and the material may be set as post-processing for cutting out the character object 104. In addition, the currently rendered texture (e.g., FBO) and the input texture may be blended according to the character object segmentation mask to retain part of the character object 104. It should be understood that the above settings are merely examples, and in practice, any other appropriate settings may be adopted. Therefore, the embodiments of the present disclosure are not limited in this respect.

[0056] In some embodiments, the predetermined presentation relationship may include at least one of the following: the character object occludes at least a portion of the at least one foreground effect 104, at least one foreground effect occludes at least a portion of the character object 104, the character object 104 is between multiple foreground effects, etc. For example, the character object 104 may be presented before the foreground effect 106, be at least partially occluded by the foreground effect 106, or be between multiple foreground effects as required.

[0057] In some embodiments, the editing device 120 may use different cameras to track close objects and distant objects. As an exemplary embodiment, for distant objects, such as the sky, buildings, etc., the editing device 120 may use a first camera to track the rotation of the object, and the first camera may use, for example, a sensor of an electronic device. In this case, the tracking of distant objects may not require the use of an algorithm. For close objects, such as the ground, ceiling, placement-type virtual objects, etc., the editing device 120 may use a second camera to track the rotation and position of the object, and the second camera may be controlled using a technology based on simultaneous localization and mapping (SLAM).

[0058] According to an embodiment of the present disclosure, segmentation-type rendering is performed for the segmented areas of the image under a template test, and post-processing is used for the foreground effects and human objects. In this way, multiple AR effects can be rendered in the same image according to different AR categories, while avoiding overlap between AR effects and unnecessary occlusion of human objects by AR effects. Therefore, the production process of AR special effects can be simplified and the efficiency of image rendering can be improved.

[0059] Figure 4 A schematic structural block diagram of an apparatus 400 for image rendering according to some embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or included in a terminal device 110 or an editing device 120. Each module / component in the apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0060] As shown, the device 400 includes an image segmentation module 410, which is configured to segment a captured image into multiple regions, the multiple regions correspond to multiple augmented reality AR categories, and the multiple AR categories have corresponding rendering priorities. The device 400 also includes an image rendering module 420, which is configured to render the multiple regions based on the rendering priorities, so that in the process of rendering a first AR effect for a first region among the multiple regions, rendering a second region among the multiple regions is avoided, wherein the first AR category associated with the first region is different from the second AR category associated with the second region.

[0061] In some embodiments, the image rendering module 420 includes: a first template buffer module, configured to designate a template buffer corresponding to the first area as a first reference value, the first reference value being used to indicate a first AR category and being different from reference values ​​designated for other areas in the image except the first area; and a first area rendering module, configured to render a first AR effect for the first area using a first renderer associated with the first AR category, the first renderer rendering the first AR effect only for an area having the first reference value.

[0062] In some embodiments, the image rendering module 420 includes: a second template buffer module, configured to designate a template buffer corresponding to the second area as a second reference value, the second reference value being used to indicate a second AR category and being different from reference values ​​designated for other areas in the image except the second area; and a second area rendering module, configured to render a second AR effect for the second area on a search image in which the first area has been rendered using a second renderer associated with the second AR category, the second renderer rendering the second AR effect only for an area having the second reference value, wherein the rendering priority of the first AR category is higher than the rendering priority of the second AR category.

[0063] In some embodiments, multiple regions of an image are rendered in a stencil test of a rendering pipeline.

[0064] In some embodiments, the first AR category is associated with at least one model object, and wherein the image rendering module 420 includes: a model object rendering module configured to render the model object with a first AR effect in response to one of the at least one model objects being located in the first area.

[0065] In some embodiments, rendering priority is determined based on a scene tree of the image.

[0066] In some embodiments, the apparatus 400 further includes: a foreground rendering module configured to render at least one foreground effect in the image based on post-processing after rendering the plurality of regions.

[0067] In some embodiments, the device 400 also includes: a character object rendering module, configured to render a character object in the image after rendering the foreground effect based on a predetermined presentation relationship, wherein the predetermined presentation relationship includes at least one of the following: the character object occludes at least a portion of at least one foreground effect, at least one foreground effect occludes at least a portion of the character object, and the character object is between multiple foreground effects.

[0068] Figure 5 A block diagram of an electronic device 500 is shown in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 5The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to implement Figure 1 terminal device 110 or editing device 120.

[0069] like Figure 5 As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0070] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the electronic device 500.

[0071] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0072] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0073] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0074] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0075] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0076] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0077] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0078] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0079] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. An image rendering method, comprising: Segmenting the captured image into a plurality of regions, the plurality of regions corresponding to a plurality of augmented reality effect categories, the plurality of augmented reality effect categories respectively having corresponding rendering priorities; as well as performing augmented reality effect rendering for the plurality of regions based on the rendering priorities, so that in a process of rendering a first augmented reality effect for a first region among the plurality of regions, rendering of a second region among the plurality of regions is avoided, wherein a first augmented reality effect category associated with the first region is different from a second augmented reality effect category associated with the second region; The performing of rendering of augmented reality effects for the multiple regions based on the rendering priority includes: designating a stencil buffer corresponding to the first region as a first reference value, the first reference value being used to indicate the first augmented reality effect category and being different from reference values ​​designated for other regions in the image except the first region; Rendering the first augmented reality effect for the first area using a first renderer associated with the first augmented reality effect category, the first renderer rendering the first augmented reality effect only for areas having the first reference value; and rendering the second augmented reality effect for the second area on the image in which the first area has been rendered using a second renderer associated with the second augmented reality effect category, the second renderer rendering the second augmented reality effect only for areas having a second reference value, The rendering priority of the first augmented reality effect category is higher than the rendering priority of the second augmented reality effect category.

2. The method according to claim 1, wherein performing augmented reality effect rendering based on the rendering priority comprises: The template buffer corresponding to the second area is designated as the second reference value, where the second reference value is used to indicate the second augmented reality effect category and is different from reference values ​​designated for other areas in the image except the second area.

3. The method of claim 1 , wherein the first augmented reality effect category is associated with at least one model object, and wherein rendering the plurality of regions comprises: In response to one of the at least one model object being located in the first area, the model object is rendered using the first augmented reality effect. The method of claim 1 , wherein the rendering priority is determined based on a scene tree of the image. The method of claim 1 , wherein the plurality of regions of the image are rendered in a stencil test of a rendering pipeline.

6. The method according to claim 1, further comprising: After rendering the plurality of regions, at least one foreground effect is rendered in the image based on post-processing.

7. The method according to claim 6, further comprising: After rendering the foreground effect, rendering a human object in the image based on a predetermined presentation relationship, The predetermined presentation relationship includes at least one of the following: The human object occludes at least a portion of the at least one foreground effect, The at least one foreground effect occludes at least a portion of the human object, The human object is between a plurality of foreground effects.

8. A device for image rendering, comprising: an image segmentation module configured to segment the captured image into a plurality of regions, the plurality of regions corresponding to a plurality of augmented reality effect categories, the plurality of augmented reality effect categories respectively having corresponding rendering priorities; as well as an image rendering module configured to perform augmented reality effect rendering for the multiple regions based on the rendering priority, so that in a process of rendering a first augmented reality effect for a first region among the multiple regions, rendering a second region among the multiple regions is avoided, wherein a first augmented reality effect category associated with the first region is different from a second augmented reality effect category associated with the second region; The image rendering module comprises: a first template buffer module configured to designate a template buffer corresponding to the first region as a first reference value, wherein the first reference value is used to indicate the first augmented reality effect category and is different from reference values ​​designated for other regions in the image except the first region; A first area rendering module is configured to render the first augmented reality effect for the first area using a first renderer associated with the first augmented reality effect category, wherein the first renderer renders the first augmented reality effect only for an area having the first reference value; and A second area rendering module is configured to use a second renderer associated with the second augmented reality effect category to render the second augmented reality effect for the second area on the image in which the first area has been rendered, the second renderer only rendering the second augmented reality effect for an area having a second reference value, wherein the rendering priority of the first augmented reality effect category is higher than the rendering priority of the second augmented reality effect category.

9. The device according to claim 8, wherein the image rendering module comprises: A second template buffer module is configured to designate the template buffer corresponding to the second area as the second reference value, wherein the second reference value is used to indicate the second augmented reality effect category and is different from the reference value designated for other areas in the image except the second area.

10. The apparatus of claim 8, wherein the first augmented reality effect category is associated with at least one model object, and wherein the image rendering module comprises: The model object rendering module is configured to render the model object using the first augmented reality effect in response to one of the at least one model object being located in the first area.

11. An electronic device comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 7 when executed by the at least one processing unit.

12. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images