Image processing method, device, terminal and storage medium

Abstract

The present disclosure provides an image processing method and device, a terminal and a storage medium. The image processing method comprises: acquiring a composite image; acquiring a background image of a region with a depth identifier; determining whether the composite image is a depth image; when it is determined that the composite image is a depth image, acquiring pixels of the composite image corresponding to the region with the depth identifier to obtain a first image; and synthesizing the background image and the first image to obtain a target image. The composite image of the present disclosure obtains scene depth, realizes occlusion removal in a three-dimensional scene in the process of superimposing and synthesizing with the background image, has a front and back occlusion relationship with other objects in the scene, and also retains the advantage of higher clarity of the composite image.

Description

Technical Field

[0001] This disclosure relates to the field of information technology, and in particular to image processing methods and apparatus, terminals and storage media. Background Technology

[0002] Unreal Engine is a leading virtual reality (VR) content development engine in the industry, enabling developers to produce high-quality VR content. Current mainstream mobile VR devices all utilize customized Android systems. Thanks to Android's rendering framework, in addition to the usual stereoscopic binocular layers, VR applications can achieve multi-layer overlay effects through compositors. This allows images rendered in multiple processes to be superimposed and displayed in the same frame.

[0003] When multiple overlay layers or composite images are superimposed on a stereo image layer containing the scene or a background image, the composite image fails to establish a depth relationship with the scene within the background image. In some applications, the composite image and background image are not simply superimposed; that is, the composite image does not completely cover or occlude the background image. Instead, occlusion culling with objects in the scene is required to achieve a depth-based blending. However, because the composite image and background image are not in the same process, the composite image cannot possess scene depth. Summary of the Invention

[0004] To address the existing problems, this disclosure provides an image processing method, apparatus, terminal, and storage medium.

[0005] The following technical solution is adopted in this disclosure.

[0006] This disclosure provides an image processing method, which includes: acquiring a composite image; acquiring a background image of a region with depth markings; determining whether the composite image is a depth image; when the composite image is determined to be a depth image, acquiring pixels of the composite image corresponding to the region of the depth markings to obtain a first image; and compositing the background image and the first image to obtain a target image.

[0007] Another embodiment of this disclosure provides an image processing apparatus, the image processing apparatus comprising: a composite image acquisition module configured to acquire a composite image; a background image acquisition module configured to acquire a background image of a region having a depth identifier; a depth image determination module configured to determine whether the composite image is a depth image; a pixel acquisition module configured to, when the composite image is determined to be a depth image, acquire pixels of the composite image corresponding to the region of the depth identifier to obtain a first image; and an image synthesis module configured to synthesize the background image and the first image to obtain a target image.

[0008] In some embodiments, this disclosure provides a terminal, including: at least one memory and at least one processor; wherein the memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the above-described image processing method.

[0009] In some embodiments, this disclosure provides a storage medium for storing program code for executing the image processing method described above.

[0010] The synthesized image disclosed herein obtains scene depth, and achieves occlusion culling in the 3D scene during the superposition and synthesis process with the background image. It also has the advantage of having a front-to-back occlusion relationship with other objects in the scene, and retains the higher clarity of the synthesized image. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0012] Figure 1 This is a schematic diagram of the existing image overlay display process.

[0013] Figure 2 This is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0014] Figure 3 and Figure 4 This is a schematic diagram illustrating the process of image overlay display according to an embodiment of the present disclosure.

[0015] Figure 5 This is a partial module of an image processing apparatus according to another embodiment of this disclosure.

[0016] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0017] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0018] It should be understood that the various steps described in the method embodiments of this disclosure can be performed in sequence and / or in parallel. Furthermore, method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0019] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0021] It should be noted that the use of the word "a" in this disclosure is illustrative rather than restrictive, and those skilled in the art should understand that it should be understood as "one or more" unless otherwise expressly indicated in the context.

[0022] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0023] Figure 1This diagram illustrates a schematic flowchart of the overlay display of an existing composite image (Overlayer) and a background image (e.g., a stereoscopic binocular image Eyelayer). The background image rendering object is obtained in Unreal Engine, rendered in the compositor, and then composited with one or more composite images, which are then overlaid and displayed. Because the composite image and the background image are in different processes, the composite image lacks scene depth, affecting the overlay display effect. For example, if the background image is an office scene with a blackboard, and the composite image is a replacement image for the corresponding area of ​​the blackboard in the background image to improve the clarity of the content displayed on the blackboard, but a presenter in the scene obscures part of the blackboard, simply overlaying the composite image and the background image would cause the composite image to obscure part of the presenter's body in the background image, affecting the overlay display effect.

[0024] Figure 2 A flowchart of an image processing method according to embodiments of the present disclosure is provided. The image processing method of the present disclosure may include step S101, obtaining a composite image. In some embodiments, the composite image is an image used to improve the clarity of content in a portion of a background image, or an image used to composite with a background image for overlay display.

[0025] In some embodiments, the method of this disclosure may further include step S102, acquiring a background image of a region with depth identification. In some embodiments, such as in a virtual display scene, the background image may include a binocular image. In some embodiments, the region with depth identification is typically a region that has undergone depth detection and occlusion culling. For example, taking the example of the narrator occluding part of the blackboard mentioned above, the region with depth identification could be the area of ​​the blackboard that is not occluded by the narrator. In this case, the blackboard has undergone occlusion culling, eliminating the area occluded by the narrator.

[0026] In some embodiments, the method of this disclosure may further include step S103, determining whether the synthesized image is a depth image. In some embodiments, the synthesized image is required to have scene depth only when it is a depth image.

[0027] In some embodiments, the method of this disclosure may further include step S104, whereby, when it is determined that the synthesized image is a depth image, pixels corresponding to the depth marker region in the synthesized image are obtained to obtain a first image. In some embodiments, by obtaining pixels corresponding to the depth marker region in the synthesized image, rather than all pixels in the synthesized image, occlusion will not occur after the obtained first image is synthesized with the background image, thereby achieving scene depth.

[0028] In some embodiments, the method disclosed herein may further include step S105, which involves compositing the background image and the first image to obtain a target image. For example, taking the example of the narrator obscuring part of the blackboard mentioned above, by obtaining the pixels corresponding to the depth marker area in the composite image, i.e., the pixels of the area of ​​the blackboard not obscured by the narrator, the composite image will not obscure the narrator after compositing the background image and the first image. That is, the first image sampled from the composite image will be perfectly superimposed on the background image, generating scene depth and improving the display effect.

[0029] In some embodiments, when it is determined that the synthesized image is not a depth image, all pixels of the synthesized image are acquired, and the background image and the synthesized image are combined to obtain the target image. Therefore, when it is determined that the synthesized image is not a depth image, the synthesized image is directly overlaid on the background image.

[0030] In some embodiments, obtaining a composite image includes: obtaining an interface for creating a composite image in a compositor; and calling the interface to create the composite image. In some embodiments, an interface for creating a composite image in a compositor is provided to Unreal Engine through a runtime library, and Unreal Engine can then create the composite image by calling the interface.

[0031] In some embodiments, obtaining a background image of a region with depth marking includes: obtaining the identity identifier (ID) of the composite image and the memory address of the rendering object of the composite image; creating a proxy object of the composite image in the scene of the background image, associating the proxy object with the corresponding composite image through the identity identifier, and copying the rendering object in the proxy object into the composite image; generating a corresponding polygon patch after enabling depth marking based on the size settings and spatial position of the proxy object in the scene; and adding depth markings to the polygon patches to obtain a background image of a region with depth markings. In some embodiments, a proxy object of the composite image in the scene of the background image is created in Unreal Engine. During creation, the proxy object is associated with the corresponding composite image in the compositor through the ID, the rendering object in the proxy object is copied into the composite image, and a corresponding polygon patch is dynamically generated after enabling depth marking based on the size settings and spatial position of the proxy object in the scene. In some embodiments, after generating the polygon patches, depth markings are added to the polygon patches to obtain a background image of a region with depth markings.

[0032] In some embodiments, adding a depth identifier to a polygonal facet includes: drawing the polygonal facet as green; or setting the alpha channel of the polygonal facet to 0. In some embodiments, when the polygonal facet is drawn as green using a shader as a blended depth identifier, after depth detection and occlusion culling rendering steps, the undisturbed portion of the polygonal facet will be green in the final rendered background image. In some embodiments, when the alpha channel of the polygonal facet is set to 0 using a shader as a blended depth identifier, after depth detection and occlusion culling rendering steps, the alpha channel of the undisturbed portion of the polygonal facet will be 0, and the occluded portion will be 1 in the final rendered background image.

[0033] In some embodiments, determining whether a synthesized image is a depth image includes: determining whether the synthesized image has depth identification enabled; if depth identification is enabled, the synthesized image is a depth image. This makes it easier to determine whether a synthesized image is a depth image.

[0034] In some embodiments, obtaining the pixels corresponding to the depth-identified region of the composite image includes: obtaining the screen space coordinates corresponding to the composite image; and obtaining the pixels corresponding to the depth-identified region of the composite image based on the layer and screen space coordinates of the composite image. In some embodiments, since the created composite image is in three-dimensional space, the two-dimensional screen space to be replaced by the composite image needs to be known before compositing, so the screen space coordinates corresponding to the composite image need to be obtained. In some embodiments, multiple composite images can be superimposed and displayed for the same background image, so the layer position of the composite image is also considered during pixel sampling. In some embodiments, the screen space coordinates of the composite image proxy are calculated in the compositor based on the three-dimensional space coordinates in the submitted rendering parameters, and each composite image is sampled and composited according to the layer and screen space coordinates of the composite image. During the sampling process, for composite images with depth identification enabled, only pixels in the background image whose corresponding screen space coordinates are green or whose alpha channel is 0 will be sampled and covered. The composited image is output to the Virtual Display and finally displayed on the screen by, for example, SurfaceFlinger in Android systems.

[0035] In some embodiments, the composite image includes one or more composite images. For example... Figure 1 and Figure 4 As shown, multiple composite images can be combined with a background image.

[0036] The following uses the synthetic image Overlayer and the stereo image (Eyelayer) as examples, combined with... Figure 3 and Figure 4To better understand this disclosure, the following description only briefly outlines aspects that differ from existing image overlays. In Unreal Engine, a proxy object for the composite image is created in the scene. The compositor interface is called to create the composite image, and the identity of the composite image and the memory address of the rendering object are obtained when the newly created composite image is acquired. During creation, the proxy object is associated with the corresponding composite image in the compositor using an ID. The rendering object from the proxy object is copied into the composite image. Simultaneously, based on the size settings of the proxy object and its spatial position in the scene, depth marking is enabled, and corresponding polygonal faces are dynamically generated. For example, the polygonal faces are drawn green using a shader as depth markings for the overlay. After depth detection and occlusion culling rendering steps, the unoccluded portions of the polygonal faces will be green in the final rendered background image. In the compositor, the screen space coordinates of the composite image proxy are calculated based on the 3D space coordinates in the submitted rendering parameters. Then, each composite image is sampled and composited according to the layer of the composite image and the screen space coordinates. During the sampling process, for composite images with depth marking enabled, only pixels in the background image that are green in screen space coordinates or have an alpha channel of 0 will be sampled and covered. The composited image is then output to the virtual display and finally displayed on the screen by SurfaceFlinger, such as on Android.

[0037] This disclosure obtains the spatial position of the composite image in the scene through a composite image proxy object, dynamically generates polygon patches of the corresponding type to proxy the depth information of the composite image, participates in depth detection and occlusion culling in the engine rendering pipeline, and renders the polygon patches as green or sets the alpha channel to 0 to convey the occlusion culling relationship between the composite image and other objects in the scene, thereby achieving a deep fusion of the composite image and the background image.

[0038] Embodiments of this disclosure also provide an image processing apparatus 400. The image processing apparatus 400 includes a composite image acquisition module 401, a background image acquisition module 402, a depth image determination module 403, a pixel acquisition module 404, and an image synthesis module 405. In some embodiments, the composite image acquisition module 401 is configured to acquire a composite image. In some embodiments, the background image acquisition module 402 is configured to acquire a background image of a region with depth markers. In some embodiments, the depth image determination module 403 is configured to determine whether the composite image is a depth image. In some embodiments, the pixel acquisition module 404 is configured to acquire pixels of the composite image corresponding to the region of the depth markers when it is determined that the composite image is a depth image, thereby obtaining a first image. In some embodiments, the image synthesis module 405 is configured to synthesize the background image and the first image to obtain a target image.

[0039] It should be understood that the description of the image processing method also applies to the image processing apparatus 400 described herein, but for simplicity, it will not be described in detail here.

[0040] In some embodiments, the pixel acquisition module is further configured to: acquire all pixels of the composite image when it is determined that the composite image is not a depth image; the image compositing module is further configured to composite the background image and the composite image to obtain a target image. In some embodiments, acquiring the composite image includes: acquiring an interface for creating the composite image in the compositor; and calling the interface to create the composite image. In some embodiments, acquiring the background image of a region with a depth identifier includes: acquiring the identity identifier of the composite image and the memory address of the rendering object of the composite image; creating a proxy object of the composite image in the scene of the background image, associating the proxy object with the corresponding composite image through the identity identifier, and copying the rendering object in the proxy object into the composite image; generating a corresponding polygon patch after enabling the depth identifier according to the size setting of the proxy object and its spatial position in the scene; and adding the depth identifier to the polygon patch to obtain the background image of the region with the depth identifier. In some embodiments, adding the depth identifier to the polygon patch includes: drawing the polygon patch in green; or setting the α channel of the polygon patch to 0. In some embodiments, determining whether the synthesized image is a depth image includes: determining whether the synthesized image has depth labeling enabled; if the synthesized image has depth labeling enabled, it is a depth image. In some embodiments, obtaining the pixels in the synthesized image corresponding to the region of the depth labeling includes: obtaining the screen space coordinates corresponding to the synthesized image; obtaining the pixels in the synthesized image corresponding to the region of the depth labeling based on the layer of the synthesized image and the screen space coordinates. In some embodiments, the synthesized image includes one or more synthesized images.

[0041] Furthermore, this disclosure also provides a terminal, comprising: at least one memory and at least one processor; wherein the memory is used to store program code, and the processor is used to call the program code stored in the memory to execute the above-described image processing method.

[0042] In addition, this disclosure also provides a computer storage medium storing program code for executing the above-described image processing method.

[0043] The image processing method and apparatus of this disclosure have been described above based on embodiments and application examples. Furthermore, this disclosure also provides a terminal and a storage medium, which will be described below.

[0044] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 500 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0045] like Figure 6 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0046] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0047] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0048] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0049] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0050] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0051] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0052] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0053] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0054] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0055] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0056] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0057] According to one or more embodiments of this disclosure, an image processing method is provided, the image processing method comprising: acquiring a composite image; acquiring a background image of a region having a depth identifier; determining whether the composite image is a depth image; when the composite image is determined to be a depth image, acquiring pixels of the composite image corresponding to the region of the depth identifier to obtain a first image; and compositing the background image and the first image to obtain a target image.

[0058] According to one or more embodiments of this disclosure, when it is determined that the synthesized image is not a depth image, all pixels of the synthesized image are obtained, and the background image and the synthesized image are combined to obtain a target image.

[0059] According to one or more embodiments of this disclosure, obtaining a composite image includes: obtaining an interface for creating the composite image in a compositor; and calling the interface to create the composite image.

[0060] According to one or more embodiments of this disclosure, obtaining a background image of a region with depth identifiers includes: obtaining an identity identifier of the composite image and the memory address of the rendering object of the composite image; creating a proxy object of the composite image in the scene of the background image, associating the proxy object with the corresponding composite image through the identity identifier, and copying the rendering object in the proxy object into the composite image; generating a corresponding polygon patch after enabling depth identifiers according to the size setting of the proxy object and its spatial position in the scene; and adding the depth identifier to the polygon patch to obtain a background image of the region with the depth identifiers.

[0061] According to one or more embodiments of this disclosure, adding the depth identifier to the polygonal patch includes: drawing the polygonal patch in green; or setting the α channel of the polygonal patch to 0.

[0062] According to one or more embodiments of this disclosure, determining whether the synthesized image is a depth image includes: determining whether the synthesized image has depth identification enabled, wherein the synthesized image is a depth image when depth identification is enabled.

[0063] According to one or more embodiments of this disclosure, obtaining the pixels of the synthesized image corresponding to the region of the depth identifier includes: obtaining the screen space coordinates corresponding to the synthesized image; and obtaining the pixels of the synthesized image corresponding to the region of the depth identifier based on the layer of the synthesized image and the screen space coordinates.

[0064] According to one or more embodiments of this disclosure, the composite image includes one or more composite images.

[0065] According to one or more embodiments of this disclosure, an image processing apparatus is also provided, the image processing apparatus comprising: a composite image acquisition module configured to acquire a composite image; a background image acquisition module configured to acquire a background image of a region having a depth identifier; a depth image determination module configured to determine whether the composite image is a depth image; a pixel acquisition module configured to, when the composite image is determined to be a depth image, acquire pixels of the composite image corresponding to the region of the depth identifier to obtain a first image; and an image synthesis module configured to synthesize the background image and the first image to obtain a target image.

[0066] According to one or more embodiments of the present disclosure, a terminal is provided, comprising: at least one memory and at least one processor; wherein the at least one memory is used to store program code, and the at least one processor is used to invoke the program code stored in the at least one memory to execute the method described in any one of the above descriptions.

[0067] According to one or more embodiments of the present disclosure, a storage medium is provided for storing program code for performing the methods described above.

[0068] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0069] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0070] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image processing method, characterized in that, The image processing method includes: Obtain the composite image; Obtain the background image of the region with depth markings; Determine whether the synthesized image is a depth image; When it is determined that the synthesized image is a depth image, the pixels corresponding to the region of the depth identifier in the synthesized image are obtained to obtain a first image; The background image and the first image are combined to obtain the target image; Obtaining the background image of the region with depth identification includes: Obtain the identity identifier of the synthesized image and the memory address of the rendering object of the synthesized image; Create a proxy object of the composite image in the scene of the background image, associate the proxy object with the corresponding composite image through the identity identifier, and copy the rendering object in the proxy object into the composite image; Based on the size settings of the proxy object and its spatial position in the scene, a corresponding polygonal patch is generated after depth marking is enabled; The depth marker is added to the polygonal patch to obtain a background image of the region with the depth marker.

2. The image processing method according to claim 1, characterized in that, When it is determined that the synthesized image is not a depth image, all pixels of the synthesized image are obtained, and the background image and the synthesized image are combined to obtain the target image.

3. The image processing method according to claim 1, characterized in that, Obtaining the composite image includes: Obtain the interface for creating the composite image in the compositor; The synthesized image is created by calling the interface.

4. The image processing method according to claim 1, characterized in that, Adding the depth identifier to the polygonal patch includes: Draw the polygonal patch in green; or Set the α channel of the polygonal patch to 0.

5. The image processing method according to claim 1, characterized in that, Determining whether the synthesized image is a depth image includes: determining whether the synthesized image has depth identification enabled; if the synthesized image has depth identification enabled, it is a depth image.

6. The image processing method according to claim 1, characterized in that, Obtaining the pixels in the synthesized image corresponding to the region identified by the depth marker includes: Obtain the screen space coordinates corresponding to the synthesized image; The pixels corresponding to the depth identifier region of the synthesized image are obtained based on the layer of the synthesized image and the screen space coordinates.

7. The image processing method according to claim 1, characterized in that, The composite image includes one or more composite images.

8. An image processing apparatus, characterized in that, The image processing device includes: The composite image acquisition module is configured to acquire composite images. The background image acquisition module is configured to acquire the background image of a region with depth markings. A depth image determination module is configured to determine whether the synthesized image is a depth image; The pixel acquisition module is configured to, when it is determined that the synthesized image is a depth image, acquire the pixels of the synthesized image corresponding to the region of the depth identifier, and obtain a first image; The image synthesis module is configured to synthesize the background image and the first image to obtain the target image; Obtaining the background image of the region with depth identification includes: Obtain the identity identifier of the synthesized image and the memory address of the rendering object of the synthesized image; Create a proxy object of the composite image in the scene of the background image, associate the proxy object with the corresponding composite image through the identity identifier, and copy the rendering object in the proxy object into the composite image; Based on the size settings of the proxy object and its spatial position in the scene, a corresponding polygonal patch is generated after depth marking is enabled; The depth marker is added to the polygonal patch to obtain a background image of the region with the depth marker.

9. A terminal, comprising: At least one memory and at least one processor; The at least one memory is used to store program code, and the at least one processor is used to call the program code stored in the at least one memory to execute the image processing method according to any one of claims 1 to 7.

10. A storage medium for storing program code for performing the image processing method according to any one of claims 1 to 7.

Images