Image synthesis method, device and image processing equipment

Abstract

The present application provides an image synthesis method, device and image processing equipment, which obtains virtual scene data including at least one virtual element with transparency and renders the at least one virtual element to obtain at least one virtual scene image with transparency information. After obtaining a real scene image of a real scene, the virtual scene image with transparency information and the real scene image are merged to display the virtual element in the real scene to obtain a synthesized image. Since the virtual scene image has transparency information when performing image synthesis, the original transparency effect of the virtual element is retained. After completing the image synthesis, the synthesized image is converted into a preset format, thereby avoiding the problem of loss of transparency information caused by performing image synthesis after converting the format of the virtual scene image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image synthesis method, device and image processing equipment. Background Art

[0002] Image processing technology based on the combination of virtual and real is a technology that fuses virtually generated images with truly photographed images, thereby making virtual elements appear in real scenes or making truly photographed elements appear in virtual scenes.

[0003] In some specific video broadcasting processes, there are specific requirements for image synthesis. For example, in the broadcasting process, the real image is in YUV format without transparency information, and the synthesized image is also required to be in YUV format. Therefore, it is required that the virtual elements must be produced based on the scene with a green screen background, and then rendered and converted into a virtual element image in YUV format with a green screen as the background, and then the virtual element image is merged with the real scene image through the green screen cutout synthesis method through the director device. However, the YUV format does not carry transparency information. The virtual element image that originally had transparency will lose the transparency information after being converted to the YUV format, which will cause the virtual elements in the synthesized image to no longer have transparency. Summary of the invention

[0004] In order to overcome the above-mentioned deficiencies in the prior art, the purpose of the present application is to provide an image synthesis method, the method comprising:

[0005] Acquiring data of a virtual scene, wherein the virtual scene includes at least one virtual element having transparency;

[0006] Rendering the at least one virtual element to obtain at least one virtual scene image with transparency information;

[0007] Get real scene images of real scenes;

[0008] fusing the virtual scene image and the real scene image so that the virtual elements are displayed in the real scene to obtain a composite image;

[0009] The composite image is output to convert the composite image into a preset format.

[0010] In a possible implementation, the step of rendering the at least one virtual element to obtain at least one virtual scene image with transparency information includes:

[0011] According to the types of the virtual elements, different types of virtual elements are rendered respectively to obtain virtual scene images with different transparency information.

[0012] In a possible implementation, the step of rendering different types of virtual elements respectively according to the types of the virtual elements to obtain virtual scene images with different transparency information includes:

[0013] Rendering the virtual element having the transparent area as a first virtual scene image with a first transparency;

[0014] Rendering the virtual element having the non-completely transparent area as a second virtual scene image with a second transparency;

[0015] Rendering the opaque virtual elements and the preset green screen background into a third virtual scene image of a third transparency;

[0016] The step of fusing the virtual scene image with the real scene image comprises:

[0017] According to the third transparency, a third virtual scene image after removing the green screen background is merged with the real scene image to obtain an intermediate image;

[0018] According to the first transparency and the second transparency, the first virtual scene image and the second virtual scene image are fused with the intermediate image to obtain the composite image.

[0019] In a possible implementation, the virtual scene image is in RGBA format;

[0020] The step of rendering the virtual element having the transparent area into a first virtual scene image with a first transparency comprises:

[0021] Rendering the virtual element having the transparent area into a first virtual scene image with an overall transparency value of 0;

[0022] The step of rendering the virtual element having the non-completely transparent area into a second virtual scene image of a second transparency comprises:

[0023] Rendering the virtual element having the non-completely transparent area into a second virtual scene image with an overall transparency value greater than 0 and less than 255;

[0024] The step of rendering the opaque virtual elements and the preset green screen background into a third virtual scene image of a third transparency includes:

[0025] Rendering the opaque virtual elements and the preset green screen background into a third virtual scene image with an overall transparency value of 255;

[0026] The step of converting the composite image into a preset format comprises:

[0027] The composite image is converted to YUV format.

[0028] In a possible implementation, before the step of fusing the third virtual scene image after removing the green screen background with the real scene image, the method further includes:

[0029] The green screen background in the third virtual scene image is removed, and the local transparency value of the position corresponding to the green screen background is set to 0.

[0030] In a possible implementation, the step of acquiring data of a virtual scene, wherein the virtual scene includes at least one virtual element having transparency, includes:

[0031] The virtual scene is generated by the UE4 engine, and the virtual elements in the virtual scene have material attribute tags;

[0032] The step of rendering different types of virtual elements respectively according to the types of the virtual elements to obtain virtual scene images with different transparency information comprises:

[0033] Traversing each virtual element in the virtual scene, and for each virtual element, determining the type of the virtual element according to a material attribute tag of the virtual element;

[0034] Different types of virtual elements are rendered using different rendering targets RenderTarget in the UE4 engine to obtain the virtual scene image with different transparency information.

[0035] In a possible implementation, the step of rendering different types of virtual elements respectively according to the types of the virtual elements to obtain virtual scene images with different transparency information includes:

[0036] Different types of virtual elements in the virtual scene are rendered respectively based on a set viewing angle to obtain virtual scene images with different transparency information and the same image size.

[0037] Another object of the present application is to provide an image synthesis device, the image synthesis device comprising:

[0038] A first acquisition module, used to acquire data of a virtual scene, wherein the virtual scene includes at least one virtual element with transparency;

[0039] An image rendering module, used for rendering the at least one virtual element to obtain at least one virtual scene image with transparency information;

[0040] A second acquisition module is used to acquire a real scene image of a real scene;

[0041] An image synthesis module, used for fusing the virtual scene image with the real scene image, so that the virtual elements are displayed in the real scene, and obtaining a synthesized image;

[0042] The image output module is used to output the composite image to convert the composite image into a preset format.

[0043] Another object of the present application is to provide an image processing device, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the image synthesis method provided in the present application is implemented.

[0044] Another object of the present application is to provide a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by one or more processors, the image synthesis method provided by the present application is implemented.

[0045] Compared with the prior art, this application has the following beneficial effects:

[0046] The image synthesis method, device and image processing device provided in the embodiments of the present application render virtual elements into virtual scene images with transparency information, fuse the virtual scene images with real scene images, and then convert the synthesized images into a preset format, so that the virtual elements in the synthesized images have transparency.

[0047] Furthermore, in the solution provided in the embodiment of the present application, virtual elements can be rendered into virtual scene images of different transparency according to the type of virtual elements, and then merged with the real scene image. In this way, different types of virtual elements in the virtual scene are decoupled, which facilitates subsequent image synthesis actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A schematic diagram of a prior art image synthesis solution;

[0050] Figure 2 A schematic diagram of a usage scenario of an image processing device provided in an embodiment of the present application;

[0051] Figure 3A schematic diagram of the steps of the image synthesis method provided in the embodiment of the present application;

[0052] Figure 4 A schematic diagram of an image synthesis process provided in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of an image processing device provided in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the functional modules of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0057] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0058] In the description of the present application, it should be noted that the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0059] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0060] Please refer to Figure 1In the existing broadcasting and television studio system, the real scene images collected by the broadcasting and television image acquisition device 920 are usually in YUV format, and the director device 940 used for image synthesis in the broadcasting and television studio system can only perform green screen background cutout synthesis actions based on the YUV format. Therefore, in the existing virtual-real combination scheme based on this scene, virtual elements are usually generated in a virtual scene with a green screen background by a virtual scene generation device 910, and then the entire virtual scene is rendered into a virtual scene image in RGBA format, and then the virtual scene image is converted into YUV format by a device similar to a capture card 930, and then output to the director device 940 in the form of a digital serial interface. The director device 940 then fuses the virtual scene image with the captured real scene image by means of green screen background cutout, and finally forms a synthetic image in YUV format.

[0061] However, after the virtual scene image is converted into the YUV format, the original transparency information will be lost, resulting in that in the subsequent image synthesis, the image of the virtual element can only be simply superimposed on the real scene image and cannot show its transparency effect.

[0062] In view of this, this embodiment provides a solution suitable for synthesizing video images combining virtuality and reality in the above-mentioned broadcast scene and retaining the transparency effect of virtual elements. The various steps of the solution are described in detail below.

[0063] Please refer to Figure 2 , is a schematic diagram of a use scenario of an image processing device 100 provided in this embodiment, and the image processing device 100 can communicate with an image acquisition device 200 to obtain a real scene image acquired by the image acquisition device 200. For example, the image processing device 100 can communicate with an image acquisition device 200 of a broadcasting system to acquire a real scene image in a YUV format.

[0064] In addition, the image processing device 100 can generate a virtual scene image according to the virtual elements in the virtual scene. In one example, the image processing device 100 can be configured with a virtual scene generation model for generating a virtual scene, for example, the 4th generation Unreal Engine (Unreal Engine 4, UE4). A virtual scene with virtual elements can be generated by the virtual scene generation model. In another example, the image processing device 100 can communicate with other devices to obtain data of virtual elements generated by other devices.

[0065] The image processing device 100 can also communicate with the image format conversion device 300. The image processing device 100 can fuse the virtual scene image and the real scene image into a composite image and send it to the image format conversion device 300, so as to convert the composite image into a preset format. For example, the image format conversion device 300 can be an acquisition card, and the image processing device 100 can send the composite image to the acquisition card. The acquisition card can convert the composite image into a YUV format and then send it to the director device 400 for subsequent video secondary processing or video output.

[0066] Please refer to Figure 3 , Figure 3 A method that can be applied to Figure 2 The figure is a schematic flow chart of the steps of the image synthesis method of the image processing device 100.

[0067] Step S110, obtaining data of a virtual scene, wherein the virtual scene includes at least one virtual element with transparency.

[0068] In this example, the virtual scene may be generated by a virtual scene generation model configured in the image processing device 100, or the image processing device 100 may obtain a generated virtual scene from other devices. The virtual scene may include at least one virtual element with transparency, and the virtual element may include transparent or semi-transparent virtual items or virtual special effects, etc.

[0069] Specifically, in this embodiment, the data of the virtual scene may be periodically acquired at a certain frame rate to keep the acquired data of the virtual scene roughly consistent with the changes of the virtual scene.

[0070] Step S120: Render the at least one virtual element to obtain at least one virtual scene image with transparency information.

[0071] In this embodiment, the virtual element may be rendered as a picture in a format with transparency information, for example, may be rendered as an image in RGBA format.

[0072] Specifically, in this embodiment, the virtual elements may be rendered periodically at a certain frame rate, so that the obtained video data composed of virtual scene images can reflect the dynamics or change trends of the virtual elements in the virtual scene.

[0073] Step S130, acquiring a real scene image of the real scene.

[0074] In this embodiment, the real scene image may be a real image captured by the image acquisition device 200 , for example, the real scene image is a video image in a YUV format captured by the image acquisition device 200 of a radio and television station.

[0075] Step S140: fusing the virtual scene image and the real scene image so that the virtual elements are displayed in the real scene to obtain a composite image.

[0076] Step S150: outputting the composite image to convert the composite image into a preset format.

[0077] In this embodiment, the image processing device 100 may perform an image synthesis action. Since the virtual scene image obtained by the image processing device 100 has transparency information, the original transparency effect of the virtual element may be retained when the image processing device 100 performs image synthesis. Then, after the image synthesis is completed, the synthesized image is converted into a preset format, thereby avoiding the problem of transparency information loss caused by performing image synthesis after format conversion of the virtual scene image.

[0078] In a possible implementation, the virtual scene may include multiple types of virtual elements, such as virtual elements with transparent areas, virtual elements with incompletely transparent areas, and opaque virtual elements. For example, the virtual character image in the virtual scene is an opaque virtual element, the cloud and fog special effect is a virtual element with an incompletely transparent area, and the glass water cup is a virtual element with a completely transparent area.

[0079] In step S120, different types of virtual elements may be rendered respectively according to the types of the virtual elements to obtain a plurality of virtual scene images with different transparency information.

[0080] Exemplarily, in step S110, the virtual scene may be generated by a UE4 engine, wherein the virtual elements in the virtual scene have material attribute tags.

[0081] In step S120, each virtual element in the virtual scene can be traversed, and for each virtual element, the type of the virtual element can be determined according to the material attribute tag of the virtual element. Then, different types of virtual elements are rendered using different rendering targets RenderTarget in the UE4 engine to obtain virtual scene images with different transparency information.

[0082] For details, please refer to Figure 4In this embodiment, a virtual scene with a green screen background can be generated by the UE4 engine. In step S120, according to the material attribute tag, the virtual element with a transparent area can be rendered as a first virtual scene image with a first transparency through RenderTarget1, the virtual element with an incompletely transparent area can be rendered as a second virtual scene image with a second transparency through RenderTarget2, and the opaque virtual element and the preset green screen background can be rendered as a third virtual scene image with a third transparency through RenderTarget3. In step S140, the real scene image, the first virtual scene image, the second virtual scene image and the third virtual scene image are fused to obtain the composite image.

[0083] Taking the virtual scene image in RGBA format as an example, in step S120, the virtual elements with transparent areas can be rendered into a first virtual scene image with an overall transparency value of 0, the virtual elements with non-completely transparent areas can be rendered into a second virtual scene image with an overall transparency value greater than 0 and less than 255, and the opaque virtual elements and the preset green screen background can be rendered into a third virtual scene image with an overall transparency value of 255.

[0084] In some possible implementations, different types of virtual elements in the virtual scene may be rendered based on a set viewing angle in step S120 to obtain virtual scene images with different transparency information and the same image size. In this way, in the image synthesis process of step S140, images of the same size may be directly synthesized without the need for additional image position alignment operations.

[0085] Exemplarily, in this embodiment, the first virtual scene image is FrameData1, the second virtual scene image is FrameData2, the third virtual scene image is FrameData3, the real scene image is FrameData4, the transparency of the first virtual scene image is alpha1, the transparency of the second virtual scene image is alpha2, and the transparency of the third virtual scene image is alpha3.

[0086] In the image synthesis process of step S140, the green screen background in the third virtual scene image can be removed first, for example, the third virtual scene image is subjected to color key removal (Keying) processing to remove the green screen background in the third virtual scene image. Then, the local transparency value of the position corresponding to the green screen background is set to 0, and the processed third virtual scene image FrameData′3 and the transparency alpha′3 of the processed third virtual scene image are obtained. Then, the image fusion is performed in the following manner:

[0087] FrameResult1=(1-alpha′3)×FrameData4+alpha′3×FrameData′3

[0088] FrameResult2=(1-alpha1)×FrameResult1+alpha1×FrameData1

[0089] FrameResult3=(1-alpha2)×FrameResult2+alpha2×FrameData2

[0090] FrameResult1 and FrameResult2 are intermediate images in the processing process, and FrameResult3 is the synthesized image finally obtained.

[0091] In some possible implementations, since the frame rate of the virtual scene image rendered in step S120 may be different from the frame rate of the real scene image obtained in step S130, the frame rates of the real scene image and the virtual scene image may be adjusted before step S140 to make the frame rates of the real scene image and the virtual scene image consistent.

[0092] For example, in this embodiment, two parallel processes may be configured in the image processing device 100. Process A may be used to execute step S110 and step S120. In process A, a virtual scene generation engine may be run, and each virtual element in the virtual scene may be traversed, and a corresponding RenderTarget may be created or selected according to the type of each virtual element for image rendering to obtain a virtual scene image in RGBA format. In addition, process A also has the ability of cross-process data communication.

[0093] Process B can receive virtual scene images rendered by different RenderTargets across processes in the form of OBS layers, and can be used to receive real scene images from other devices. For example, process B can receive RGBA image data frames output by three different types of RenderTargets in process A in real time through pipelines, shared memory, sockets, etc., while process B uses the existing hardware input layer to receive image data frames in YUV format captured by the image acquisition device 200 of the acquisition card. Then, frame alignment and image synthesis actions are performed in process B.

[0094] Based on the same inventive concept, this embodiment also provides an image processing device 100, which is an electronic device with certain image processing and communication capabilities, such as a server, a personal computer, a workstation, etc. Figure 5, Figure 5 is a block diagram of the image processing device 100. The image processing device 100 comprises an image synthesis device 110, a machine-readable storage medium 120 and a processor 130.

[0095] The machine-readable storage medium 120 and the processor 130 are electrically connected directly or indirectly to achieve data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The image synthesis device 110 includes at least one software function module that can be stored in the machine-readable storage medium 120 in the form of software or firmware or fixed in the operating system (OS) of the image processing device 100. The processor 130 is used to execute the executable modules stored in the machine-readable storage medium 120, such as the software function modules and computer programs included in the image synthesis device 110.

[0096] The machine-readable storage medium 120 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The machine-readable storage medium 120 is used to store a program, and the processor 130 executes the program after receiving an execution instruction.

[0097] The processor 130 may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 130 may be a general-purpose processor 130, including a central processing unit 130 (CPU), a network processor 130 (NP), etc.; it may also be a digital signal processor 130 (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor 130 may be a microprocessor 130 or the processor 130 may also be any conventional processor 130, etc.

[0098] Please refer to Figure 6 This embodiment further provides an image synthesis device 110, which includes at least one functional module that can be stored in a machine-readable storage medium 120 in the form of software. From a functional perspective, the image synthesis device 110 may include a first acquisition module 111, an image rendering module 112, a second acquisition module 113, an image synthesis module 114, and an image output module 115.

[0099] The first acquisition module 111 is used to acquire data of a virtual scene, where the virtual scene includes at least one virtual element with transparency.

[0100] In this embodiment, the first acquisition module 111 can be used to execute Figure 3 As shown in step S110, for a detailed description of the first acquisition module 111, please refer to the description of step S110.

[0101] The image rendering module 112 is used to render the at least one virtual element to obtain at least one virtual scene image with transparency information.

[0102] In this embodiment, the image rendering module 112 can be used to perform Figure 3 As shown in step S120, for a detailed description of the image rendering module 112, please refer to the description of step S120.

[0103] The second acquisition module 113 is used to acquire a real scene image of a real scene.

[0104] In this embodiment, the second acquisition module 113 can be used to execute Figure 3 As shown in step S130, for a detailed description of the second acquisition module 113, please refer to the description of step S130.

[0105] The image synthesis module 114 is used to fuse the virtual scene image and the real scene image so that the virtual elements are displayed in the real scene to obtain a synthesized image.

[0106] In this embodiment, the image synthesis module 114 can be used to perform Figure 3 As shown in step S140 , for a detailed description of the image synthesis module 114 , please refer to the description of step S140 .

[0107] The image output module 115 is used to output the composite image to convert the composite image into a preset format.

[0108] In this embodiment, the image output module 115 can be used to perform Figure 3As shown in step S150, for a detailed description of the image output module 115, please refer to the description of step S150.

[0109] In summary, the image synthesis method, device and image processing device provided by the embodiments of the present application render virtual elements into virtual scene images with transparency information, merge the virtual scene images with real scene images, and then convert the synthesized images into a preset format, so that the virtual elements in the synthesized images have transparency.

[0110] Furthermore, in the solution provided in the embodiment of the present application, virtual elements can be rendered into virtual scene images of different transparency according to the type of virtual elements, and then merged with the real scene image. In this way, different types of virtual elements in the virtual scene are decoupled, which facilitates subsequent image synthesis actions.

[0111] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0112] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0113] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0114] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0115] The above are only various implementations of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image synthesis method, characterized in that: The method comprises: Acquiring data of a virtual scene, wherein the virtual scene includes at least one virtual element having transparency; Rendering the at least one virtual element to obtain at least one virtual scene image with transparency information; Get real scene images of real scenes; fusing the virtual scene image and the real scene image so that the virtual elements are displayed in the real scene to obtain a composite image; Outputting the composite image to convert the composite image into a preset format; The step of rendering the at least one virtual element to obtain at least one virtual scene image with transparency information comprises: Rendering the virtual element having the transparent area as a first virtual scene image with a first transparency; Rendering the virtual element having the non-completely transparent area as a second virtual scene image with a second transparency; Rendering the opaque virtual elements and the preset green screen background into a third virtual scene image of a third transparency; The step of fusing the virtual scene image with the real scene image comprises: According to the third transparency, a third virtual scene image after removing the green screen background is merged with the real scene image to obtain an intermediate image; According to the first transparency and the second transparency, the first virtual scene image and the second virtual scene image are fused with the intermediate image to obtain the composite image.

2. The method according to claim 1, characterized in that The virtual scene image is in RGBA format; The step of rendering the virtual element having the transparent area into a first virtual scene image with a first transparency comprises: Rendering the virtual element having the transparent area into a first virtual scene image with an overall transparency value of 0; The step of rendering the virtual element having the non-completely transparent area into a second virtual scene image of a second transparency comprises: Rendering the virtual element having the non-completely transparent area into a second virtual scene image with an overall transparency value greater than 0 and less than 255; The step of rendering the opaque virtual elements and the preset green screen background into a third virtual scene image of a third transparency includes: Rendering the opaque virtual elements and the preset green screen background into a third virtual scene image with an overall transparency value of 255; The step of converting the composite image into a preset format comprises: The composite image is converted to YUV format.

3. The method according to claim 2, characterized in that Before the step of fusing the third virtual scene image after removing the green screen background with the real scene image, the method further includes: The green screen background in the third virtual scene image is removed, and the local transparency value of the position corresponding to the green screen background is set to 0.

4. The method according to claim 1, characterized in that The step of acquiring data of a virtual scene, wherein the virtual scene includes at least one virtual element having transparency, comprises: The virtual scene is generated by the UE4 engine, and the virtual elements in the virtual scene have material attribute tags; The step of rendering different types of virtual elements respectively according to the types of the virtual elements to obtain virtual scene images with different transparency information comprises: Traversing each virtual element in the virtual scene, and for each virtual element, determining the type of the virtual element according to a material attribute label of the virtual element; Different types of virtual elements are rendered using different rendering targets RenderTarget in the UE4 engine to obtain the virtual scene image with different transparency information.

5. The method according to claim 1, characterized in that The step of rendering different types of virtual elements respectively according to the types of the virtual elements to obtain virtual scene images with different transparency information comprises: Different types of virtual elements in the virtual scene are rendered respectively based on a set viewing angle to obtain virtual scene images with different transparency information and the same image size.

6. An image synthesis device, characterized in that: The image synthesis device comprises: A first acquisition module, used to acquire data of a virtual scene, wherein the virtual scene includes at least one virtual element with transparency; An image rendering module, used for rendering the at least one virtual element to obtain at least one virtual scene image with transparency information; A second acquisition module is used to acquire a real scene image of a real scene; An image synthesis module, used for fusing the virtual scene image with the real scene image, so that the virtual elements are displayed in the real scene, and obtaining a synthesized image; An image output module, used for outputting the composite image to convert the composite image into a preset format; The image rendering module is further used to render the virtual elements with transparent areas into a first virtual scene image with a first transparency; render the virtual elements with non-completely transparent areas into a second virtual scene image with a second transparency; render the opaque virtual elements and the preset green screen background into a third virtual scene image with a third transparency; The image synthesis module is also used to fuse the third virtual scene image after removing the green screen background with the real scene image according to the third transparency to obtain an intermediate image; and to fuse the first virtual scene image and the second virtual scene image with the intermediate image according to the first transparency and the second transparency to obtain the synthesized image.

7. An image processing device, characterized in that: The method comprises a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are executed by one or more processors, the method according to any one of claims 1 to 5 is implemented.

Images