Cutting method, device, system and image fusion method based on virtual shooting

Abstract

The present invention discloses a cutout method, device, system and image fusion method based on virtual shooting. The cutout method includes obtaining a first image and a second image obtained by shooting a target at the same time, wherein the first image is an image shot by an ordinary camera or a video camera, and the second image is an image shot by a depth camera; using the second image, separating the foreground and background in the first image, and performing image registration on the separated foreground and background; edge processing is performed on the second image to obtain multiple edge lines, from which the target edge line corresponding to the cutout target is determined; the target edge line is mapped to the first image for display as the edge of the cutout target; after super-resolution processing is performed on the first image, the transparency of the edge of the cutout target in the first image is adjusted, and the cutout target is extracted therefrom. The method, device and system provided by the present invention can improve the efficiency and effect of cutout, and make the image fusion more realistic and natural.

Description

Technical Field

[0001] The present invention relates to the technical field of cutout and image fusion, and in particular to a cutout method, device, system and image fusion method based on virtual shooting. Background Art

[0002] In the prior art, many scenes are shot with the help of green screens, with people or objects shot with the green screen as the background. The shot images are processed in the computer, the green background is removed, and other scene images are replaced as the background. Finally, various images containing people or objects and scene images are displayed on the screen, such as the scenes hosted by the host in the filming of movies, etc. However, in the scenes shot with the green screen, it is necessary to build a green screen and a special environment (to avoid the sunlight or other background colors affecting the picture display). And after shooting with the green screen, it is necessary to manually use the cutout software to cut out the image, so the user watching on the display terminal cannot see the cutout or image fusion effect in real time.

[0003] At present, with the widespread use of LED, LED display screens have also been built into three-sided LED stage display screens to replace the green screen usage scenarios mentioned above. For example, in the built LED stage display screen, the host can stand in the center of the screen to host, and the screen displays the corresponding picture content as the background, and finally shoot through the camera (also known as virtual shooting).

[0004] However, when using LED screens to build LED stage screens for virtual shooting, if the foreground image (such as the host) needs to be cut out and placed in another scene image, or the background image (such as the image displayed on the LED screen) needs to be cut out and merged with other foreground images, due to algorithm and real-time problems, the cutout effect is not good, and the image obtained after the cutout is merged with other images, which will cause the display to be unnatural and unreal. For users who watch through the display terminal, there is a problem of poor experience, so those skilled in the art are in urgent need of finding a new technical solution to solve the above problems. Summary of the invention

[0005] In view of the above problems, the present invention provides a cutout method, device, system and image fusion method based on virtual shooting.

[0006] The virtual shooting-based cutout method provided by the present invention comprises obtaining a first image and a second image obtained by shooting a target at the same time, wherein the first image is an image shot by an ordinary camera or a video camera, and the second image is an image shot by a depth camera;

[0007] Using the second image, separating the foreground and background in the first image, performing image registration on the separated foreground and background, and adjusting the positions of the foreground and background in the first image;

[0008] Selecting a foreground or background in the first image as a cutout target according to a cutout requirement;

[0009] Performing edge processing on the second image to obtain a plurality of edge lines, from which a target edge line corresponding to the cutout target is determined;

[0010] Mapping the target edge line to the first image for display as the edge of the cutout target;

[0011] After super-resolution processing is performed on the first image, the transparency of the edge of the cutout object in the first image is adjusted, and the cutout object is extracted therefrom.

[0012] Further, using the second image to separate the foreground and background in the first image includes:

[0013] Depth information is extracted from the second image, and binarization is performed on the first image according to the depth information to separate the foreground and background in the first image.

[0014] Furthermore, determining the target edge line corresponding to the cutout target includes:

[0015] The edge line closest to the cutout target is taken as the target edge line corresponding to the cutout target.

[0016] Furthermore, adjusting the transparency of the edge of the cutout object in the first image includes:

[0017] If the cutout target is the foreground, adjust the transparency of the edge of the cutout target to the range of [0.5, 1];

[0018] If the cutout target is the background, adjust the transparency of the edge of the cutout target to the range of [0, 0.5).

[0019] Furthermore, the edge of the cutout target is composed of a plurality of edge pixels, and adjusting the transparency of the edge of the cutout target in the first image further includes:

[0020] If the cutout target is the foreground, the greater the distance between the edge pixel and the cutout target, the smaller the transparency of the edge pixel is set;

[0021] If the cutout target is the background, at the edge of the cutout target, the greater the distance between the edge pixel and the cutout target, the greater the transparency of the edge pixel is set.

[0022] The present invention also provides an image fusion method, the method comprising:

[0023] According to the above-mentioned cutout method based on virtual shooting, a cutout target is extracted;

[0024] The cutout target is fused with the image to be fused, and the fused image is sent to the display terminal for display.

[0025] The present invention also provides a cutout device based on virtual shooting, the device includes an image acquisition module, a foreground and background separation module, an edge processing module, a mapping module, and an extraction module, wherein:

[0026] An image acquisition module, connected to the foreground and background separation module, is used to acquire a first image and a second image obtained by photographing the target at the same time, wherein the first image is an image taken by an ordinary camera or a video camera, and the second image is an image taken by a depth camera;

[0027] A foreground and background separation module is connected to the edge processing module and is used to separate the foreground and background in the first image using the second image, and to perform image registration on the separated foreground and background, and adjust the positions of the foreground and background in the first image;

[0028] The edge processing module is connected to the mapping module and is used to select the foreground or the background in the first image as the cutout target according to the cutout requirement; perform edge processing on the second image to obtain multiple edge lines, and determine the target edge line corresponding to the cutout target;

[0029] A mapping module, connected to the extraction module, for mapping the target edge line to the first image for display as the edge of the cutout target;

[0030] The extraction module is used to adjust the transparency of the edge of the cutout target in the first image after super-resolution processing is performed on the first image, and to extract the cutout target therefrom.

[0031] Furthermore, the foreground and background separation module uses the second image to separate the foreground and background in the first image, including: extracting depth information from the second image, and binarizing the first image according to the depth information to separate the foreground and background in the first image.

[0032] Furthermore, the edge processing module determines the target edge line corresponding to the cutout target, including: taking the edge line closest to the cutout target as the target edge line corresponding to the cutout target.

[0033] The present invention also provides a cutout system based on virtual shooting, the system comprising a server, a depth camera and a common camera or a video camera;

[0034] An ordinary camera or a video camera is used to shoot the target at the same time as the depth camera to obtain a first image;

[0035] A depth camera is used to shoot a target at the same time as a normal camera or a video camera to obtain a second image;

[0036] The server is connected to the depth camera and the ordinary camera or the video camera, and is used to extract the cutout target from the first image according to the cutout method based on virtual shooting.

[0037] The virtual shooting-based cutout method, device, system and image fusion method provided by the present invention have at least the following beneficial effects:

[0038] (1) The cutout method, device, system and image fusion method based on virtual shooting provided by the present invention realize automatic cutout and automatic image fusion without manual processing, thereby improving the efficiency of cutout and image fusion. After automatic cutout and automatic image fusion, the images can be directly displayed in real time on the display terminal. In addition, the cutout and image fusion in the present invention are based on cutout and image fusion after virtual shooting. Compared with cutout and image fusion based on green screen shooting, there is no need to build a green screen or set up a special environment.

[0039] (2) The virtual shooting-based cutout method, device, and system provided by the present invention utilizes a second image captured by a depth camera to extract depth information from the second image, separates the foreground and background in the first image captured by an ordinary camera or a video camera, and adjusts the positions of the separated foreground and background in the first image through image registration, extracts the target edge line of the cutout target from the second image, maps it to the first image, and then adjusts the transparency of the edge of the cutout target by super-resolution processing of the first image, and finally obtains an accurate cutout target. The transparency of the edge of the cutout target is adjusted to improve the cutout effect, so that the obtained cutout target can be more realistically and naturally integrated with other images in the future.

[0040] (3) The image fusion method provided by the present invention uses the cutout method provided by the present invention to obtain an accurate cutout target with adjusted edge transparency, and fuses the cutout target with the image to be fused, thereby improving the display effect of the fused image and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 is a flow chart of a cutout method based on virtual shooting in one embodiment of the present invention;

[0043] Figure 2 A flow chart of an edge transparency adjustment method in one embodiment of the present invention;

[0044] Figure 3 This is a flow chart of an image fusion method in one embodiment of the present invention;

[0045] Figure 4 Schematic diagram of the structure of a cutout device based on virtual shooting in one embodiment of the present invention;

[0046] Figure 5 Schematic diagram of a cutout system based on virtual shooting in one embodiment of the present invention

[0047] 401 - image acquisition module, 402 - foreground and background separation module, 403 - edge processing module, 404 - mapping module, 405 - extraction module, 501 - server, 502 - depth camera, 503 - ordinary camera or video camera, 504 - display terminal. DETAILED DESCRIPTION

[0048] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0049] In one embodiment of the present invention, Figure 1 As shown, a cutout method based on virtual shooting is disclosed. The execution subject of the method is a server, and specifically includes the following steps:

[0050] Step S101: Acquire a first image and a second image obtained by photographing a target at the same time.

[0051] Among them, the targets include the LED stage display built by the LED display screen and the objects being filmed.

[0052] The first image is an image taken by an ordinary camera or a video camera, and the second image is an image taken by a depth camera. A depth camera refers to a camera that can measure the distance (depth) from an object (the object here is the target mentioned in the present invention) to the camera. The second image obtained after the depth camera is taken is a depth image. A depth image is also called a range image, which refers to an image that uses the distance (depth) from the depth camera to each point in the photographed target as a pixel value.

[0053] Step S102: using the second image, separating the foreground and background in the first image, performing image registration on the separated foreground and background, and adjusting the positions of the foreground and background in the first image.

[0054] Specifically, in this step, using the second image to separate the foreground and background in the first image includes: extracting depth information from the second image, and binarizing the first image according to the depth information to separate the foreground and background in the first image.

[0055] Among them, the depth information refers to the pixel value of each pixel in the second image. Since in the second image, the pixel value of each pixel represents: the distance from the point in the shooting target corresponding to the pixel to the depth camera, the distance from each point in the shooting target to the depth camera can be obtained by extracting the depth information. By combining the depth information with the first image, a binarization process can be performed according to the depth information corresponding to each pixel in the first image to distinguish the foreground and background in the first image, specifically, to distinguish the position of the pixel corresponding to the foreground and the position of the pixel corresponding to the background.

[0056] In this step, after extracting the depth information from the second image, the object in the first image and the LED stage display screen built by the LED display screen can be distinguished according to the depth information, wherein the object (such as a person) is the foreground, and the LED stage display screen built by the LED display screen is the background. After distinguishing the object in the first image and the LED stage display screen built by the LED display screen, the pixel values ​​of the pixels corresponding to the object and the LED stage display screen built by the LED display screen in the first image can be binarized. Binarization (English: Thresholding) is one of the simplest methods for image segmentation. Binarization can convert a grayscale image into a binary image, and set the grayscale value (pixel value) of the pixel point on the image to 0 or 255, that is, the entire image presents an obvious visual effect of only black and white. Therefore, in this step, the pixels corresponding to the foreground in the first image (foreground pixels) and the pixels corresponding to the background (background pixels) are binarized, so as to separate the foreground and background in the first image.

[0057] More specifically, since the first image and the second image are images obtained by different cameras shooting the target at the same time, the position of the camera shooting the first image and the camera shooting the second image at the same time cannot be the same, that is, there is a position difference between the ordinary camera or video camera shooting the first image and the depth camera shooting the second image, so when shooting the target, the content of the captured image will be different. Therefore, in this step, the first image is binarized according to the depth information, and after separating the foreground and background in the first image, the second image needs to be used for image registration (making the points corresponding to the same position of the target in the first image and the second image correspond one by one, so as to achieve the purpose of information fusion, so that the depth information is more accurately fused with the pixel points in the first image), and then the positions of the foreground and background in the first image are adjusted, so as to identify the accurate positions of the foreground and background.

[0058] The second image and the first image are used for image registration, and an existing multi-angle image registration method can be used. Multi-angle image registration refers to the registration of images with different field of view angles obtained from different observation points (corresponding to the first image and the second image in this application), so as to reconstruct the image depth or shape.

[0059] Furthermore, when using the second image and the first image for multi-angle image registration, it can be implemented based on a registration method based on the feature information of the image to be registered, and feature information such as points, lines, and edges in the image to be registered can be extracted without the need for other auxiliary information. While reducing the amount of calculation and improving efficiency, it can have a certain degree of robustness to changes in image grayscale.

[0060] Furthermore, according to the different selected feature information, the feature-based image registration methods are divided into three categories: (a) Matching based on feature points. Generally, the selected feature points are pixels that have some singularity relative to their fields. Feature points are often easy to extract, but the information contained in feature points is relatively small, and can only reflect their position coordinate information in the image. Therefore, finding matching feature points in two images is the key. (b) Matching based on feature regions. Find some obvious regional information in the image as feature regions. After finding the feature regions in practical applications, the centroid points of the regions are used for registration. (c) Matching based on feature edges. The most obvious feature in the image is the edge, and the edge feature is also one of the easiest features to extract. Therefore, the edge matching method is more robust and has a wide range of applications.

[0061] Step S103: selecting the foreground or the background in the first image as a cutout target according to the cutout requirement.

[0062] Step S104: performing edge processing on the second image to obtain a plurality of edge lines, from which a target edge line corresponding to the cutout target is determined.

[0063] Specifically, in this step, the sobel operator (edge ​​detection operator) can be used to perform edge processing to extract multiple edge lines from the second image. More specifically, the second image is represented at the pixel level, and after edge processing, multiple edge lines (specifically, position coordinates corresponding to multiple edge lines) are obtained from the second image. Since each edge line is composed of multiple pixel points, the multiple edge lines obtained in this step are the position coordinates of each pixel point in each edge line.

[0064] Among them, the Sobel operator is an important processing method in the field of computer vision, mainly used to obtain the first-order gradient of digital images. Its common application and physical meaning is edge detection. The Sobel operator is to detect the edge by taking the weighted difference of the grayscale values ​​of the four areas of the top, bottom, left and right of each pixel in the image and reaching the extreme value at the edge.

[0065] Furthermore, in this step, after the edge line is obtained, determining the target edge line corresponding to the cutout target includes: taking the edge line closest to the cutout target as the target edge line corresponding to the cutout target.

[0066] Step S105: Mapping the target edge line into the first image for display as the edge of the cutout target.

[0067] Since the CCD specifications of ordinary cameras (or video cameras) and depth cameras may be inconsistent, the resolution of the captured images will also be different. In general, the pixels of the first image captured by an ordinary camera are significantly higher than those of the second image captured by a depth camera. Assuming that the width and resolution of the first image are several times that of the second image, that is, multiple pixels in the first image must correspond to one pixel in the second image (such as the number of rows of pixels in the first image is 5 times the number of rows of pixels in the second image, then a row of pixels in the second image is equivalent to 5 rows of pixels in the first image). Therefore, in this step, the target edge line is mapped to the first image for display. It is necessary to determine the pixel corresponding to the target edge line in the first image based on the mapping relationship between the second image and the pixel points in the first image after the position coordinates of the target edge line in the second image are known, so that the target edge line is mapped to the first image for display as the edge of the cutout target.

[0068] In another implementation, an ordinary camera (or camera) with the same CCD specifications can be used to shoot with a depth camera to ensure that the resolution of the first image and the second image obtained are consistent, so that each pixel in the second image obtained corresponds to a pixel in the first image, thereby improving the matching efficiency.

[0069] Step S106: After super-resolution processing is performed on the first image, the transparency of the edge of the cutout object in the first image is adjusted, and the cutout object is extracted therefrom.

[0070] Specifically, adjusting the transparency of the edge of the cutout target refers to adjusting the transparency of the pixel points corresponding to the edge (ie, edge pixels), which is equivalent to adjusting the clarity of the image.

[0071] Furthermore, in one implementation of this embodiment, Figure 2 As shown, adjusting the transparency of the edge of the cutout target in the first image includes the following steps:

[0072] Step S201: Determine whether the cutout target is the foreground, if so, execute step S202, otherwise, execute step S203.

[0073] Step S202: Adjust the transparency of the edge of the cutout target to within the range of [0.5, 1].

[0074] Step S203: Adjust the transparency of the edge of the cutout target to be within the range of [0, 0.5).

[0075] Furthermore, the edge of the cutout target is composed of multiple edge pixels, and adjusting the transparency of the edge of the cutout target in the first image also includes: if the cutout target is the foreground, at the edge of the cutout target, the greater the distance between the edge pixels and the cutout target, the smaller the transparency of the edge pixels is set; if the cutout target is the background, at the edge of the cutout target, the greater the distance between the edge pixels and the cutout target, the greater the transparency of the edge pixels is set.

[0076] After the foreground and background are separated in the first image, there is an obvious line segment separation between the foreground and the background, and the false situation of the picture is serious, which will have a certain impact on the fusion of the subsequent images. In the prior art, the transparency of the edge between the foreground and the background is processed with 0 and 1, but this transparency processing makes the boundary between the foreground and the background obvious, and there will also be obvious line segment separation, and the transition between the foreground and the background is abrupt. Therefore, in this step, the first image is super-resolution processed to improve the resolution of the first image, so that the transparency of the edge of the cutout target can be adjusted to make the transparency within the range of 0-1. Compared with adjusting the edge transparency to 0 or 1, adjusting the transparency to the range of 0-1 can make the edge boundary between the processed foreground and the background softer, thereby making the obtained cutout target and other images to be fused more naturally.

[0077] The cutout method based on virtual shooting provided in the present embodiment uses a second image taken by a depth camera to extract depth information from the second image, separates the foreground and background in the first image taken by an ordinary camera or a video camera, and adjusts the positions of the separated foreground and background in the first image through image registration, extracts the target edge line of the cutout target from the second image, maps it to the first image, and then adjusts the transparency of the edge of the cutout target by super-resolution processing of the first image to finally obtain the cutout target, so that the obtained cutout target can be better integrated with other images, making the final fused image real and natural, improving the display effect of the fused image, improving the user experience effect, and improving the real-time cutout efficiency (the server can be used for real-time automatic cutout, without the need for manual cutout through software for display).

[0078] In another embodiment of the present invention, the present invention also provides an image fusion method, such as Figure 3 As shown, the method comprises the following steps:

[0079] Step S301: extracting a cutout target according to the cutout method based on virtual shooting;

[0080] Step S302: Fusing the cutout target with the image to be fused, and sending the fused image to a display terminal for display.

[0081] Specifically, it should be understood that when the cutout target is the foreground, and the image to be fused is the scene image that needs to be displayed, the scene image that needs to be displayed is obtained, the processed foreground is fused into the scene image, and the combined image is sent to the user's display terminal for display; wherein, the scene image is an image of a pure background, and the color difference between the scene image and the foreground image needs to be kept as small as possible; the scene image can be derived from the scene image that has been set in advance, and the images can be arranged and displayed in the software display interface, and each scene corresponds to an image to be fused. In addition, when the cutout target is the background, the foreground image can be extracted from the image captured by the green screen as the image to be fused and fused with the cutout target.

[0082] More specifically, in the present embodiment, the cutout target is fused with the pre-extracted image to be fused by adding the cutout target to the image to be fused for fusion through pixel-by-pixel scanning. In the process of application rendering, the processed cutout target is added to the image to be fused for fusion through pixel-by-pixel scanning, and pixels other than the cutout target (for example, if the cutout target is the foreground, the pixels other than the cutout target refer to the background in the first image; if the cutout target is the background, the pixels other than the cutout target refer to the foreground in the first image) are not added, which can reduce data processing and improve the processing performance of the software.

[0083] The present invention also provides a cutout device based on virtual shooting, such as Figure 4 As shown, the device includes an image acquisition module 401, a foreground and background separation module 402, an edge processing module 403, a mapping module 404, and an extraction module 405, wherein:

[0084] The image acquisition module 401 is connected to the foreground and background separation module 402, and is used to acquire a first image and a second image obtained by photographing the target at the same time, wherein the first image is an image taken by an ordinary camera or a video camera, and the second image is an image taken by a depth camera;

[0085] The foreground and background separation module 402 is connected to the edge processing module 403 and is used to separate the foreground and background in the first image using the second image, and to perform image registration on the separated foreground and background, and adjust the positions of the foreground and background in the first image;

[0086] The edge processing module 403 is connected to the mapping module 404 and is used to select the foreground or the background in the first image as the cutout target according to the cutout requirement; perform edge processing on the second image to obtain multiple edge lines, and determine the target edge line corresponding to the cutout target;

[0087] A mapping module 404, connected to the extraction module 405, is used to map the target edge line to the first image for display as the edge of the cutout target;

[0088] The extraction module 405 is used to adjust the transparency of the edge of the cutout object in the first image after performing super-resolution processing on the first image, and extract the cutout object therefrom.

[0089] Furthermore, in another embodiment of the present invention, the foreground and background separation module 402 uses the second image to separate the foreground and background in the first image, including: extracting depth information from the second image, and binarizing the first image according to the depth information to separate the foreground and background in the first image.

[0090] Furthermore, in yet another embodiment of the present invention, the edge processing module 403 determines the target edge line corresponding to the cutout target, including: taking the edge line closest to the cutout target as the target edge line corresponding to the cutout target.

[0091] The present invention also provides a cutout system based on virtual shooting, such as Figure 5 As shown, the system includes a server 501, a depth camera 502, and a common camera or video camera 503;

[0092] A common camera or a video camera 503, used to shoot a target at the same time as the depth camera to obtain a first image;

[0093] The depth camera 502 is used to shoot the target at the same time as the ordinary camera or the video camera to obtain a second image;

[0094] The server 501 is connected to the depth camera and the ordinary camera or the video camera, and is used to extract the cutout target from the first image according to the cutout method based on virtual shooting.

[0095] Further, such as Figure 5 As shown, the system further includes a display terminal 504, which is connected to the server 501 and is used to display the cutout target in real time or to display the fused image in real time after the server fuses the cutout target with the image to be fused.

[0096] The cutout method, device, system and image fusion method based on virtual shooting provided by the present invention realize automatic cutout and automatic fusion of images without manual processing, thereby improving the efficiency of cutout and image fusion. At the same time, the cutout method, device and system based on virtual shooting provided by the present invention extract depth information from the second image by using the second image taken by the depth camera, separate the foreground and background in the first image taken by the ordinary camera or the video camera, and adjust the positions of the separated foreground and background in the first image through image registration, extract the target edge line of the cutout target from the second image, map it to the first image, and then adjust the transparency of the edge of the cutout target by super-resolution processing of the first image, finally obtain an accurate cutout target, and the transparency of the edge of the cutout target is adjusted, so as to improve the cutout effect, so that the obtained cutout target can be more realistically and naturally fused with other images in the future. In addition, the system also includes a display terminal. Since the automatic cutout and automatic image fusion are realized in the present invention without manual participation, the cutout target or the fused image can be displayed in the display terminal in real time during the cutout or image fusion process, so that the user can observe the cutout effect or image fusion effect in real time. In addition, the image fusion method provided by the present invention uses the above-mentioned cutout method to obtain an accurate cutout target with adjusted edge transparency, and fuses the cutout target with the image to be fused, thereby improving the display effect of the fused image and improving the user experience.

[0097] The terms and expressions used in the specification of the present invention are for illustration only and are not meant to constitute limitations. The "first" and "second" used in the claims and the specification of the present invention herein are only for the purpose of convenience of distinction, have no special meaning, and are not intended to limit the present invention. It should be understood by those skilled in the art that various changes may be made to the details in the above-mentioned embodiments without departing from the basic principles of the disclosed embodiments. Therefore, the scope of the present invention is determined only by the claims. In the claims, unless otherwise specified, all terms should be understood in the broadest and most reasonable sense.

Claims

1. A cutout method based on virtual shooting, characterized in that: The method comprises: Acquire a first image and a second image obtained by photographing the target at the same time, wherein the first image is an image photographed by a common camera or a video camera, and the second image is an image photographed by a depth camera; Using the second image, separating the foreground and background in the first image, performing image registration on the separated foreground and background, and adjusting the positions of the foreground and background in the first image; Selecting a foreground or a background in the first image as a cutout target according to a cutout requirement; Performing edge processing on the second image to obtain a plurality of edge lines, from which a target edge line corresponding to a cutout target is determined; Mapping the target edge line to the first image for display as the edge of the cutout target; After performing super-resolution processing on the first image, adjusting the transparency of the edge of the cutout target in the first image, and extracting the cutout target therefrom; The using the second image to separate the foreground and background in the first image comprises: Depth information is extracted from the second image, and binarization is performed on the first image according to the depth information to separate the foreground and background in the first image.

2. The virtual shooting-based cutout method according to claim 1, characterized in that: Determining the target edge line corresponding to the cutout target includes: The edge line closest to the cutout target is used as the target edge line corresponding to the cutout target.

3. The virtual shooting-based cutout method according to claim 1, characterized in that: The step of adjusting the transparency of the edge of the cutout target in the first image comprises: If the cutout target is the foreground, adjust the transparency of the edge of the cutout target to the range of [0.5, 1]; If the cutout target is the background, the transparency of the edge of the cutout target is adjusted to be within the range of [0, 0.5).

4. The method for cutting out an image based on virtual shooting according to claim 3, characterized in that: The edge of the cutout target is composed of a plurality of edge pixels, and the adjusting the transparency of the edge of the cutout target in the first image further includes: If the cutout target is the foreground, in the edge of the cutout target, the greater the distance between the edge pixel and the cutout target, the smaller the transparency of the edge pixel is set; If the cutout target is the background, at the edge of the cutout target, the greater the distance between the edge pixel and the cutout target, the greater the transparency of the edge pixel is set.

5. An image fusion method, characterized in that: The method comprises: According to the cutout method based on virtual shooting according to any one of claims 1 to 4, extracting a cutout target; The cutout target is fused with the image to be fused, and the fused image is sent to a display terminal for display.

6. A cutout device based on virtual shooting, characterized in that: The device comprises an image acquisition module, a foreground and background separation module, an edge processing module, a mapping module, and an extraction module, wherein: The image acquisition module is connected to the foreground and background separation module, and is used to acquire a first image and a second image obtained by photographing the target at the same time, wherein the first image is an image taken by an ordinary camera or a video camera, and the second image is an image taken by a depth camera; The foreground and background separation module is connected to the edge processing module and is used to separate the foreground and background in the first image by using the second image, and to perform image registration on the separated foreground and background, and adjust the positions of the foreground and background in the first image; The edge processing module is connected to the mapping module and is used to select the foreground or the background in the first image as the cutout target according to the cutout requirement; perform edge processing on the second image to obtain multiple edge lines, and determine the target edge line corresponding to the cutout target; The mapping module is connected to the extraction module and is used to map the target edge line to the first image for display as the edge of the cutout target; The extraction module is used to adjust the transparency of the edge of the cutout target in the first image after super-resolution processing is performed on the first image, and extract the cutout target therefrom; The foreground and background separation module uses the second image to separate the foreground and background in the first image, including: extracting depth information from the second image, and binarizing the first image according to the depth information to separate the foreground and background in the first image.

7. The virtual shooting-based cutout device according to claim 6, characterized in that: The edge processing module determines the target edge line corresponding to the cutout target, including: taking the edge line closest to the cutout target as the target edge line corresponding to the cutout target.

8. A cutout system based on virtual shooting, characterized in that: The system includes a server, a depth camera and a common camera or a video camera; The ordinary camera or video camera is used to photograph the target at the same time as the depth camera to obtain a first image; The depth camera is used to photograph the target at the same time as the ordinary camera or the video camera to obtain a second image; The server is connected to the depth camera and the ordinary camera or video camera, and is used to extract a cutout target from the first image according to the cutout method based on virtual shooting according to any one of claims 1 to 4.

Images