Method for generating new view angle image and electronic equipment

Abstract

The invention provides a method for generating a new view angle image and electronic equipment, and the method comprises the steps: carrying out the multi-plane image (MPI) representation after the image is divided into blocks, and mapping the obtained MPI representation image of each image block into a coordinate system of the new view angle image based on homography transformation, and synthesizing a new view image based on the depth of the image block (MPI characterizing the depth of the image). According to the scheme, while the number of layers and the size of the MPI are effectively reduced, generation of a new view angle image is realized by means of simple coordinate transformation and depth-based synthesis, so that the efficiency of three-dimensional mirror operation is effectively improved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional camera movement technology, and more particularly to a method and electronic device for generating images from new perspectives. Background Technology

[0002] Three-dimensional (3D) camera movement refers to the continuous generation of images from new perspectives from a given scene, creating a dynamic effect where the camera moves relative to the objects in the image. In the field of 3D camera movement, the speed of movement depends on the rate of acquiring 3D representations. Common methods for acquiring 3D representations include layered depth image (LDI) representation and multiplane image (MPI) representation. LDI representation is computationally complex and has a long latency, resulting in slow 3D camera movement speeds and poor performance. In contrast, MPI representation uses multiple layers of MPI images to represent each object in the scene, significantly reducing computational load and latency compared to LDI. However, the computational complexity of MPI representation is still relatively high, which leads to higher requirements for the computing power of electronic devices. It is more suitable for electronic devices with relatively strong computing power, such as computer mainframes, servers, and cloud devices. For electronic devices with relatively low computing and storage capabilities, such as mobile phones and tablets, the MPI representation method still results in excessive latency due to the large amount of computation, unsatisfactory 3D camera movement effects, and even an inability to handle such complex calculations.

[0003] Therefore, how to improve the generation rate of new perspective images in order to enhance the 3D camera movement speed is an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a method and electronic device for generating new perspective images, which can improve the generation rate of new perspective images, thereby increasing the 3D camera movement speed.

[0005] In a first aspect, a method for generating a new perspective image is provided. The method includes: performing an overlap block operation on the image to be processed to obtain a first image block set, the first image block set including multiple first image blocks, the boundary of each first image block not exceeding the boundary of the image to be processed; performing multi-plane image (MPI) representation on the multiple first image blocks respectively to obtain multiple first MPI representation images, each first image block corresponding to one first MPI representation image; performing homography transformation on the multiple first MPI representation images to obtain multiple second MPI representation images of the new perspective image; and fusing the multiple second MPI representation images into the new perspective image according to the depth of the multiple first MPI representation images.

[0006] In this application's technical solution, the main approach involves segmenting the image into blocks and then performing MPI representation. The resulting MPI representation image of each image block is then mapped to the coordinate system of the new perspective image based on homography transformation. Finally, the new perspective image is synthesized based on the depth of the image blocks (the depth of the MPI representation image). This solution effectively reduces the number and size of MPI layers while generating new perspective images through simple coordinate transformations and depth-based synthesis, thereby significantly improving the efficiency of 3D camera movement.

[0007] In conjunction with the first aspect, in some implementations of the first aspect, an overlap block operation is performed on the image to be processed to obtain a first set of image blocks. This includes: starting from the first vertex of the image to be processed, using the two edges containing the first vertex as starting edges, and sequentially overlapping and framing the first image blocks with boxes of the same size into the image to be processed; if the box exceeds the end edge of the image to be processed, the first image block is then framing and framing the first image block into the image to be processed starting from the intersection of the box and the end edge, with the end edge being the other two edges of the image to be processed besides the two starting edges. In this implementation, overlapping and framing the image blocks with boxes of fixed size is simple and convenient. For boxes that would exceed the boundary of the image to be processed without adjustment, their relative positions are adjusted to ensure that all image blocks are of the same size and that the content within each image block is the original content of the image to be processed.

[0008] In one example, the overlap of adjacent first image patches is all within a preset overlap range. In this example, controlling the overlap within the preset range allows for a relatively good balance between the quality and efficiency of MPI representation, avoiding both excessive and insufficient representation. This preset overlap range can be determined based on the size of the first image patch, or it can be determined by setting different overlap levels for a certain number of test image patches and then basing the preset overlap range on the MPI representation results of these test image patches.

[0009] In conjunction with the first aspect, in certain implementations of the first aspect, multiple second MPI representation images are fused into a new viewpoint image according to the depth of the multiple first MPI representation images. This includes: sorting the multiple second MPI representation images in order from near to far or from far to near according to the depth of the multiple first MPI representation images; and fusing the sorted multiple second MPI representation images into a new viewpoint image. In this implementation, the second MPI representation images are sorted according to depth, thereby determining the adjacency relationship between different MPI representation images, thus making the fusion process more efficient.

[0010] In conjunction with the first aspect, in certain implementations of the first aspect, homography transformation is performed on multiple first MPI representation images to obtain multiple second MPI representation images of a new viewpoint image. This includes: determining the coordinates and optical centers of the first vertices of multiple second image blocks corresponding to the multiple first image blocks in the new viewpoint image based on the coordinates and optical centers of the first vertices of the multiple first image blocks; and performing homography transformation and sampling on the multiple first MPI representation images based on the coordinates and optical centers of the first vertices of the multiple second image blocks to obtain multiple second MPI representation images. In this implementation, the updated positions in the new image (new viewpoint image) are determined based on the positions of the image blocks in the original image, thereby determining the relative positions of the first MPI representation images in the new image coordinate system.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: padding multiple first MPI representation images to obtain multiple third MPI representation images; and performing homography transformation on the multiple third MPI representation images to obtain multiple second MPI representation images of the new viewpoint image. In this implementation, padding the first representation images makes the transitions between them more natural when all representation images are subsequently fused, effectively eliminating edge lines.

[0012] In one example, the padding size used is greater than 0 and less than the maximum padding size, which indicates the size corresponding to overfilling. This effectively avoids edge line problems caused by homography transformation sampling while preventing the introduction of other artificially added elements. It should be understood that padding is used to smooth the boundaries between MPI representation images corresponding to different image blocks, facilitating the fusion of multiple MPI representation images and preventing a sense of discontinuity or edge lines at the boundaries when merging different MPI representation images. It also avoids overfilling that could lead to incorrect or distorted image content. However, it should be understood that the padding requirement in this application's scheme stems from new problems introduced by the steps included in this application's scheme, primarily caused by homography transformation and sampling when obtaining the second MPI representation image.

[0013] In one example, multiple first MPI representation images are filled to obtain multiple third MPI representation images. This includes: filling the area to be filled by copying the color values ​​of a first region in each first MPI representation image to the area to be filled. The first region is a region whose distance from the boundary of the first MPI representation image is less than or equal to a preset distance threshold, which is determined based on the size of the area to be filled; and filling the opacity value of the area to be filled with zero. In this example, filling by color copying makes the transition more natural and the effect better.

[0014] In a second aspect, an apparatus for generating images from new perspectives is provided, the apparatus comprising units consisting of software and / or hardware for performing any of the methods of the first aspect.

[0015] Thirdly, an electronic device is provided, comprising: one or more processors, and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to enable the electronic device to implement any of the methods of the first aspect.

[0016] Fourthly, a chip system is provided, the chip system being applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to enable the electronic device to implement any of the methods of the first aspect.

[0017] Optionally, the chip system also includes a memory electrically connected to the processor.

[0018] Optionally, the chip system may also include a communication interface.

[0019] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium including instructions that, when executed on an electronic device, enable the electronic device to implement any of the methods of the first aspect.

[0020] In a sixth aspect, a computer program product is provided, comprising a computer program that, when executed by an electronic device, can implement any of the methods of the first aspect. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of a three-dimensional camera movement scene applicable to the embodiments of this application.

[0022] Figure 2 This is a comparison chart of the MPI characterization results of the traditional MPI characterization method and the MPI characterization results of the present application.

[0023] Figure 3 This is a schematic diagram of two overlapping block methods.

[0024] Figure 4 This is a schematic flowchart illustrating a method for generating a new perspective image according to an embodiment of this application.

[0025] Figure 5 This is a comparison diagram of new perspective images generated based on different fill sizes according to embodiments of this application.

[0026] Figure 6 This is a schematic flowchart illustrating another method for generating images from a new perspective according to an embodiment of this application.

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0028] Figure 8 This is a schematic diagram of the system structure of an electronic device according to an embodiment of this application.

[0029] Figure 9 This is a schematic diagram of the hardware structure of another electronic device according to an embodiment of this application. Detailed Implementation

[0030] The embodiments of this application will now be described in conjunction with the accompanying drawings.

[0031] Figure 1 This is a schematic diagram of a three-dimensional camera movement scene applicable to an embodiment of this application. For example... Figure 1 Image (a) shows the original image, also known as the initial headshot or original image. So-called 3D camera movement is based on this original image, continuously generating new perspective images. The sequence of the original image and these new perspective images, played sequentially at a certain frame rate, generates a dynamic video. Figure 1 To generate Figure 1 Take the 3D camera movement video of the image shown in (a) as an example.

[0032] Figure 1 The original image shown in (a) includes a foreground and a background. The foreground includes a standing person (an example of a target object, referred to as target A for ease of description), while the background includes trees and a road. 3D camera movement gives the user the effect of a camera constantly moving relative to target A. This could be achieved by the camera zooming in, zooming out, or rotating around target A. Figure 1 Image (b) shows the generated new perspective image of frame i, where i is a positive integer. From this new perspective image, it can be seen that the relative position and size of target A in the original image remain unchanged in the new perspective image of frame i. However, the road in the background appears increasingly distant, and new elements such as trees B appear in the distance, presenting a visual effect of the scenery continuously extending forward. It should be understood that i can be 1 or greater than 1, meaning... Figure 1 There may be other new perspective images between (a) and (b). Figure 1Image (c) shows the generated (i+j)th frame of the new perspective image, where j is a positive integer. From this (i+j)th frame of the new perspective image, it can be seen that the relative position and size of target A in the i-th frame of the new perspective image remain unchanged in the (i+j)th frame of the new perspective image. However, the road in the background is more distant, and the content displayed in the background continues to increase, such as the appearance of new trees C. It should be understood that j can be 1 or greater than 1, meaning... Figure 1 Other new perspective images may exist between (b) and (c). From Figure 1 As can be seen in (a)-(c), during the 3D camera movement, the background will continuously extend into the distance, the road will appear to be getting longer and longer, and the number of trees on both sides of the road will increase.

[0033] In order to achieve Figure 1 The 3D camera movement effect shown has been achieved using LDI (Limited Distributed Image) representation, but this method is computationally intensive and inefficient. Therefore, MPI (Mean Differential Image) representation-based methods were developed. In MPI-based methods, multiple layers of MPI representation data are generated for each object, with one MPI layer corresponding to each depth. Each MPI layer includes a color (RGB) image and an opacity (alpha, A) image; hence, MPI-represented images are often considered RGBA images. However, this traditional MPI-based method still suffers from high computational cost and long computation time. This is because each depth represents the entire original image as an MPI-represented image at that depth. In reality, the depth of each object in an image is finite, almost never fully covering the entire depth range of the original image. This inevitably results in a large amount of redundant information in the final MPI-represented image.

[0034] To address the aforementioned issues, this application proposes a novel scheme for generating images from new perspectives. This primarily involves overlapping and segmenting the original image into blocks, then performing MPI representation on each block, thereby effectively reducing the number of MPI layers and the size of each MPI layer. Because an image block is significantly smaller than the entire image, the depth range it covers is necessarily much smaller than the depth range of the entire image, and it may even fall within a single MPI layer. Therefore, the number of MPI layers corresponding to each object in the entire image is inevitably reduced. Furthermore, since each image block has a small area, the size of its resulting MPI representation image is also necessarily much smaller than the size of the MPI representation image of the entire image.

[0035] Figure 2 This is a comparison chart of the MPI characterization results using the traditional MPI characterization method and the MPI characterization results using the method described in this application. Figure 2In this context, we assume that the original image contains several shapes, including circles, squares, stars, and rhombuses, which represent different objects located at different depths of field in the image.

[0036] like Figure 2 As shown in (a), when using the traditional MPI representation scheme, an MPI representation image of the original image is obtained at each depth. However, the MPI representation images at different layers may contain only one object, or even none at all. This results in a relatively large number and size of final MPI representation images, with a significant amount of redundant information. This example uses layers A1 to A5, but it should be understood that the actual number of layers depends on the depth range covered by the original image and the depth range of each element in the image.

[0037] like Figure 2 As shown in (b), when using the MPI representation scheme provided in this application, the original image is first divided into image blocks, and then MPI representation is performed on each image block. Since the depth range of the image blocks is small, each image block corresponds to one MPI layer, and the size of the image block is also small. Therefore, the size of the MPI representation image on the corresponding MPI layer is also relatively small. Here, layers A1-A5 are taken as an example, but it should be understood that other numbers are possible in practice. In addition, in practice, it is possible that the number of layers obtained by the MPI representation using the traditional scheme is greater than the number of layers obtained by the MPI representation of the scheme in this application, which will not be elaborated further.

[0038] from Figure 2 As can be seen from the large triangle in (b), the MPI representation is not performed on an object-by-object basis, nor on the entire image basis, but on an image patch basis. Therefore, in the original image, the large triangle is divided into 4 parts. After calculating the MPI representation of these 4 parts, 4 MPI representation images are obtained. However, these 4 MPI representation images all correspond to the same depth, that is, on the same layer (taking layer A3 as an example). But it can be seen that the MPI representation image on this layer is 4 patch MPI representation images, rather than the complete MPI representation image corresponding to the entire original image.

[0039] Image segmentation effectively reduces the number and size of MPI layers of objects in the original image, eliminating invalid and redundant information, thereby reducing processing complexity and computational load, improving the efficiency of generating new perspective images, and increasing 3D camera movement speed. However, it should be understood that because this application's scheme adds segmentation operations, it is necessary to consider how to fuse these segmented MPI representation images into a new perspective image. Therefore, the subsequent processing will differ from traditional MPI representation schemes; otherwise, the generation of the new perspective image cannot be completed. Details are provided below.

[0040] Figure 3 This is a schematic diagram of two overlapping block methods. For example... Figure 3 The image shown in (a) is the original image (also known as the original image, the image to be processed, the initial portrait, or the reference image). When performing overlap segmentation on it, a box of a specific size can be used to segment the image according to a specific degree of overlap. Figure 3 The size shown in box A is used as an example.

[0041] like Figure 3 As shown in (b), each small square can frame an image block. When capturing an image block, the top-left corner of the original image is used as the starting point, and the two sides containing that corner (the top and left sides) are used as the starting edges. Following the size of the square shown in square A, multiple image blocks are sequentially drawn into the image. When approaching the right and bottom edges (the ending edges), the square may extend beyond the boundaries of the original image, such as... Figure 3 Image patches numbered {7, 14, 28, 35, 42} and 36-41 are shown in (b). {7, 14, 28, 35, 42} are image patches that extend beyond the right boundary of the original image, and 36-42 are image patches that extend beyond the bottom boundary of the original image. In this case, the boundary values ​​of the original image are used to fill the excess portion, thus making... Figure 3 The size of (b) is greater than Figure 3 In the image (a), the main issue is that the right and bottom sides show areas filled with boundary values, resulting in a stretched appearance. This stretching causes distortion at the right and bottom boundaries of the new perspective image, creating a stretched look. Figure 3 In diagram (b), we see an example of uniform overlap, where all adjacent image patches have the same degree of overlap. To illustrate the size of each image patch, Figure 3 In (b), the boundary of image block number 3 is highlighted in bold. It overlaps with image blocks 2, 4, 9, 10 and 11.

[0042] It should be understood that Figure 3 Taking the top left corner as the starting point as an example, but in reality, any vertex of the image can be the starting point, the two edges where the starting point is located are the starting edges, and the remaining two edges are the ending edges, without any restrictions.

[0043] To avoid stretching, this application modifies the way the image block is framed when capturing the part near the end edge. When the frame intersects with the end edge, the image block is framed out into the image from that intersection point.

[0044] like Figure 3As shown in (c), each small square can enclose an image block. When selecting an image block, the top-left corner of the original image is used as the starting point, and the two sides containing that corner (the top and left sides) are used as the starting edges. Multiple image blocks are sequentially enclosed inwards from the image. When approaching the right and bottom edges (the ending edges), the square may exceed the image boundary. The process remains the same. Figure 3 The box numbered as shown in (b) is used in this case. Using the right and bottom boundaries as new starting edges, and the intersection of the box and the new starting edges as the starting point, a new image block is drawn inwards. It can be seen that... Figure 3 Image patches numbered {7, 14, 28, 35, 42} and 36-41, shown in (b), are stretched. Figure 3 In section (c), there is no longer any stretch fill, that is, as... Figure 3 Image blocks numbered {7, 14, 28, 35, 42} and 36-41 shown in (c) do not exhibit stretching. Figure 3 (c) is an example of non-uniform overlap, which shows that... Figure 3 The dimensions of (c) and Figure 3 The dimensions of (a) are equal. To illustrate the so-called non-uniform overlap, Figure 3 In (c), the two image blocks 37 and 42 are thickened, and it can be clearly seen that the degree of overlap between them and the adjacent image blocks has changed, but their boundaries do not exceed the boundaries of the original image.

[0045] Figure 4 This is a schematic flowchart illustrating a method for generating a new perspective image according to an embodiment of this application. The following is a description of... Figure 4 The steps shown will be explained.

[0046] S401. Perform overlapping block operation on the image to be processed to obtain the first image block set.

[0047] The first image block set includes multiple first image blocks, and the boundary of each first image block does not exceed the boundary of the image to be processed.

[0048] It should be understood that the number of first image patches depends on the size of the segmentation unit used for segmentation and the size of the image to be processed. It should also be understood that the size of the image patches is generally relatively small, thus allowing each image patch to cover a smaller depth range, thereby ensuring that each image patch corresponds to only one MPI layer. The size of the image to be processed is generally relatively large, but even for smaller images and / or images with relatively small depth ranges, the scheme of this application can still improve the rate of MPI characterization, thereby ensuring the efficiency of 3D camera movement.

[0049] An image to be processed can be divided into dozens or hundreds of image blocks, which can ensure that the representation of each image block is relatively simple, and that the representation time does not increase instead of decrease due to the large number of image blocks.

[0050] In one implementation, the image to be processed is divided into overlapping blocks to obtain a first set of image blocks. This includes: starting from the first vertex of the image to be processed, using the two edges containing the first vertex as starting edges, and sequentially overlapping the first image blocks with boxes of the same size towards the interior of the image to be processed; if the box exceeds the end edge of the image to be processed, the first image block is then drawn towards the interior of the image to be processed, starting from the intersection of the box and the end edge, with the end edge being the two edges of the image to be processed other than the two starting edges. In this implementation, overlapping the image blocks with boxes of fixed size is simple and convenient. For boxes that would exceed the boundary of the image to be processed without adjustment, their relative positions are adjusted to ensure that all image blocks are of the same size and that the content within each image block is the original content of the image to be processed. Figure 3 (c) is an example of this implementation, and box A is an example of a box of the same size as above.

[0051] Overlapping bounding boxes inevitably result in overlapping areas between adjacent image patches, and the proportion of this overlap affects subsequent rendering. Excessive overlap leads to too many image patches, increasing the number of subsequent MPI representations, while insufficient overlap prevents the full preservation of the relationships between elements in the image. Overlap can be understood as the percentage of the overlapping area within an image patch.

[0052] In one example, the overlap of adjacent first image patches is all within a preset overlap range. In this example, controlling the overlap within the preset range allows for a relatively good balance between the quality and efficiency of MPI representation, avoiding both excessive and insufficient representation. This preset overlap range can be determined based on the size of the first image patch, or it can be determined by setting different overlap levels for a certain number of test image patches and then basing the preset overlap range on the MPI representation results of these test image patches.

[0053] In one example, the preset overlap range is the interval between the first overlap and the second overlap, including both the first and second overlaps, where the first overlap is less than the second overlap. This preset overlap range includes a minimum first overlap and a maximum second overlap. The first overlap corresponds to the boundary between sufficient and insufficient MPI representation, i.e., the boundary point for the minimum required MPI representation quality. The second overlap corresponds to the boundary between an appropriate and excessive amount of MPI representation, i.e., the boundary point for the highest required amount of MPI representation.

[0054] In one example, the first overlap is 0.25 and the second overlap is 0.5. This example represents a suitable range determined through experimental testing on the test images, within which MPI characterization performs best, with good quality and high efficiency.

[0055] It should be noted that the above example mainly uses the same bounding box for the overlap operation. However, it should be understood that in practice, multiple bounding boxes of different sizes can also be used sequentially, although the operation will be more complex and may affect the execution speed. For example, a preset sequence of bounding box sizes can be set, which includes multiple bounding boxes of different sizes. Following the order indicated by this sequence and the execution steps of the overlap block operation described above, multiple image blocks of different sizes are sequentially bounded, and subsequent MPI characterization steps are performed. However, the overlap setting may also need to be adjusted accordingly. Different overlap settings may be set for larger and smaller image blocks. Of course, not adjusting it can also ensure the overall effectiveness of the solution in this application to a certain extent.

[0056] S402. Perform MPI characterization on multiple first image blocks respectively to obtain multiple first MPI characterization images.

[0057] Each first image block corresponds to a first MPI representation image.

[0058] It should be understood that while the size of each first MPI representation image corresponds to the size of its corresponding image patch, this does not mean that the two sizes are the same. This is because the size of the image after MPI representation is also affected by the depth of the MPI layer. The deeper the layer, the larger the size of the MPI representation image may be. Figure 2 As can be seen.

[0059] It should also be understood that the first MPI representation image is a block-based MPI representation image, not the MPI representation image corresponding to the entire image to be processed. In traditional schemes, there is only one MPI representation image per depth (each MPI layer), and the size of this MPI representation image is determined by the size of the original image and the depth of the MPI layer, such as... Figure 2 As shown in (a). The first MPI representation image may contain multiple MPI representation images within a single MPI layer, such as... Figure 2 As shown in (b).

[0060] In one implementation, the method further includes: padding multiple first MPI representation images to obtain multiple third MPI representation images; and performing homography transformation on the multiple third MPI representation images to obtain multiple second MPI representation images of the new viewpoint image. In this implementation, padding the first representation images makes the transitions between them more natural when all representation images are subsequently fused, effectively eliminating edge lines. Related content will be combined with... Figure 5 Detailed explanations will not be repeated here.

[0061] Filling also requires consideration of two issues: how much to fill and what to fill. Too little fill won't effectively achieve a smooth transition between images representing different MPI values, while too much may incorrectly alter or distort the original image information. As for what to fill, the simplest approach is to fill with fixed values, while further approaches include filling with boundary values, and even more advanced approaches include filling with values ​​within a certain range near the boundary, essentially creating a small-scale copy.

[0062] In one example, the padding size used is greater than 0 and less than the maximum padding size, which indicates the size corresponding to overfilling. This effectively avoids edge line problems caused by homography transformation sampling while preventing the introduction of other artificially added elements. It should be understood that padding is used to smooth the boundaries between MPI representation images corresponding to different image blocks, facilitating the fusion of multiple MPI representation images and preventing a sense of discontinuity or edge lines at the boundaries when merging different MPI representation images. It also avoids overfilling that could lead to incorrect or distorted image content. However, it should be understood that the padding requirement in this application's scheme stems from new problems introduced by the steps included in this application's scheme, primarily caused by homography transformation and sampling when obtaining the second MPI representation image. To illustrate the concept of overfilling, consider an extreme example: suppose each MPI representation image is 60x60 pixels. If the padding size is also 60 pixels, it's equivalent to adding 60 pixels to the outer edge of each image. After padding, the MPI representation image becomes 180x180 pixels. The actual effective content is only the original 60x60 pixels, with the rest being artificial padding data. This can cause the content of other adjacent MPI representation images to be obscured or affected, which is counterproductive.

[0063] In one example, multiple first MPI representation images are filled to obtain multiple third MPI representation images. This includes: filling the area to be filled by copying the color values ​​of a first region in each first MPI representation image to the area to be filled. The first region is defined as a region whose distance from the boundary of the first MPI representation image is less than or equal to a preset distance threshold, which is determined based on the size of the area to be filled; and filling the area to be filled with zero opacity. In this example, filling by color copying makes the transition more natural and the effect better. Related content can also be found at [reference needed]. Figure 5 I will not go into details.

[0064] S403. Perform homography transformation on multiple first MPI characterization images to obtain multiple second MPI characterization images of the new perspective image.

[0065] Homography transformation is one of the most common coordinate system transformations. This step is to map the obtained first MPI representation image to the coordinate system of the new viewpoint image. It can be seen that the scheme in this application only involves translation and does not involve other complex transformations such as rotation.

[0066] In one implementation, homography transformation is performed on multiple first MPI representation images to obtain multiple second MPI representation images of a new viewpoint image. This includes: determining the coordinates and optical centers of the first vertices of multiple second image blocks corresponding to the multiple first image blocks in the new viewpoint image based on the coordinates and optical centers of the first vertices of the multiple first image blocks; and performing homography transformation and sampling on the multiple first MPI representation images based on the coordinates and optical centers of the first vertices of the multiple second image blocks to obtain the multiple second MPI representation images. In this implementation, the updated positions in the new image (new viewpoint image) are determined based on the positions of the image blocks in the original image, thereby determining the relative positions of the first MPI representation images in the new image coordinate system.

[0067] S404. According to the depth of multiple first MPI characterization images, fuse multiple second MPI characterization images into a new perspective image.

[0068] It should be understood that the depth of the first MPI characterization image corresponds to the depth of the image patch to which the first MPI characterization image is located. Therefore, step S404 can also be considered as fusing multiple second MPI characterization images into a new perspective image according to the depth of the image patch.

[0069] In one implementation, multiple second MPI representation images are fused into a new perspective image according to the depths of multiple first MPI representation images, including: sorting the multiple second MPI representation images in ascending or descending order of depth according to the depths of the multiple first MPI representation images; and fusing the sorted multiple second MPI representation images into a new perspective image. In this implementation, the second MPI representation images are sorted by depth, so that the adjacent relationship between different MPI representation images can be determined, thereby making the fusion more efficient.

[0070] Figure 4 The method shown mainly performs MPI representation after dividing the image into blocks, maps the MPI representation images of each obtained image block to the coordinate system of the new perspective image based on the homography transformation, and synthesizes the new perspective image based on the depth of the image block (the depth of the MPI representation image). While effectively reducing the number of MPI layers and the size, this solution also realizes the generation of the new perspective image by means of simple coordinate transformation and depth-based synthesis, thereby effectively improving the efficiency of 3D camera movement.

[0071] Figure 5 is a comparison diagram of new perspective images generated based on different padding sizes in an embodiment of the present application. As Figure 5 shown in (a) is the original image. Figure 5 shown in (b) is a schematic diagram of the k-th frame new perspective image when the padding size (render_padsz) is zero (no padding). In the k-th frame image, there are long and thin black lines, which are caused by the hard splicing at the boundary when different block MPI images are combined into the new perspective image. As Figure 5 shown in (c) is a schematic diagram of the m-th frame new perspective image when the padding size is n1 (for example, a value between 10 and 20 or other values) pixels. There are no black lines in the m-th frame image and no other redundant elements in the image. As Figure 5 shown in (d) is a schematic diagram of the p-th frame new perspective image when the padding size is n2 (for example, a value between 60 and 80 or other values) pixels. There are no black lines in the p-th frame image but there are redundant elements. All the above variables are positive integers, and n1 < n2. In practical applications, the appropriate padding size can be determined based on the size of the block MPI representation image, that is, to achieve the effect shown by n1 pixels for the determined padding size.

[0072] It should be understood that Figure 5 the color values in a small range are copied to the padding area, and this padding method can make the effect after padding better and the coherence of the image better compared with directly using the boundary values.

[0073] Figure 6This is a schematic flowchart illustrating another method for generating images from a new perspective according to an embodiment of this application. Figure 6 for Figure 4 An example.

[0074] S601. Perform small-range color copying and filling and small-range zero-filling of opacity on the block MPI representation image of the original image.

[0075] The segmented MPI representation image in step S601 can be obtained by using... Figure 4 The image is represented after the overlap block operation in the original image, which will not be elaborated further. The original image is represented by src, which is short for source. Small-range copy padding means copying the color values ​​(represented by src_mpi_color) of the MPI representation image corresponding to each image block of the original image to the corresponding area to be filled, according to the fill size indicator. Small-range zero padding means setting the opacity src_mpi_alpha of the area to be filled in the block MPI representation image to zero.

[0076] S602, Determine the top-left coordinates and update the optical center of the image tile.

[0077] It should be understood that Figure 6 The scheme shown is for generating a new viewpoint image (i.e., the target image, which can be abbreviated as tgt). Therefore, this step is to map the padded MPI representation image src_mpi from step S601 to the new viewpoint image. Since each image patch corresponds to an MPI representation image, it can be determined based on the image patch.

[0078] S603. Through homography transformation, the block MPI representation image in the new perspective image is obtained by sampling.

[0079] That is, the coordinate system is transformed from the coordinate system of the original image to the coordinate system of the new viewpoint image through homography transformation, and the color map (which can be represented by tgt_mpi_color) and opacity map (which can be represented by tgt_mpi_alpha) of each MPI characterization image in the new viewpoint image are obtained by sampling.

[0080] Homography transformation sampling can be represented as grid_sample. It should be understood that the lack of padding in this application's scheme may result in edge lines (…). Figure 5 The thin black line shown in (b) is caused by grid_sample. This is a new problem brought about by the solution of the traditional MPI representation complexity. Therefore, other MPI representation methods that do not adopt the solution of this application may not have edge line problems, and there is no need to perform small-area filling.

[0081] S604. Sort all image patches in the new viewpoint image into block MPI representation images (tgt_mpi_color and tgt_mpi_alpha) according to the depth of the image patches in the original image (src) from near to far.

[0082] S605. Combine the sorted block MPI representation images into a new perspective image.

[0083] Here, based on alpha synthesis, the new perspective image can satisfy the formula. in, Let t represent the new view image at frame t, N be the total number of MPI layers, c represent color, α represent opacity, (1-α) represent occlusion, and c′ and α′ represent the estimates of c and α, respectively.

[0084] It should be understood that Figure 6 yes Figure 4 The example shown is an example of a scheme, and specifically an example of a combination of some of its implementations. Therefore... Figure 4 The relevant technical effects of the solution shown can be cited in Figure 6 The plan will not be elaborated further.

[0085] The methods of the embodiments of this application have been described above with reference to the accompanying drawings. It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially, these steps are not necessarily executed in the order shown in the figures. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps. The apparatus of the embodiments of this application will now be described with reference to the accompanying drawings.

[0086] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Figure 7 As shown, the device 2000 includes a block unit 2001 and a processing unit 2002. The device 2000 can be an electronic device such as a mobile phone, tablet computer, laptop computer, computer, or vehicle infotainment system.

[0087] The device 2000 can be used to execute any of the methods described above for generating images from new perspectives. For example, the segmentation unit 2001 can be used to execute step S401, and the processing unit 2002 can be used to execute steps S402-S404. The processing unit 2002 can also be used to execute... Figure 6 The method shown.

[0088] In one implementation, the device 2000 may further include a storage unit for storing relevant data. This storage unit may be integrated into any of the aforementioned units, or it may be a unit independent of all of the aforementioned units.

[0089] Figure 8 This is a schematic diagram of the system architecture of an electronic device according to an embodiment of this application. The layered architecture divides the system into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom: the application layer, the application framework layer, the hardware abstraction layer, the driver layer, and the hardware layer. The application layer may include a series of application packages. In this embodiment, the application packages may include a camera, a gallery, etc.

[0090] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. In this embodiment, the application framework layer may include a camera access interface, which may include camera management and camera devices. The camera access interface is used to provide application programming interfaces and programming frameworks for camera applications.

[0091] The hardware abstraction layer is an interface layer located between the application framework layer and the driver layer, providing a virtual hardware platform for the operating system. In this embodiment, the hardware abstraction layer may include a camera hardware abstraction layer and a camera algorithm library.

[0092] The camera hardware abstraction layer can provide virtual hardware for camera device 1, camera device 2, or more camera devices. The camera algorithm library may include runtime code and data that implement the shooting methods provided in the embodiments of this application.

[0093] The driver layer is the layer between hardware and software. It includes drivers for various hardware components, such as camera drivers, digital signal processor drivers, and image processor drivers.

[0094] The camera device driver is used to drive the camera sensor to acquire images and to drive the image signal processor to preprocess the images. The digital signal processor driver is used to drive the digital signal processor to process images. The image processor driver is used to drive the graphics processor to process images.

[0095] The method for generating new perspective images in this application embodiment is described in detail below, based on the above system structure:

[0096] In response to a user's action of opening the camera application, such as clicking the camera application icon, the camera application calls the camera access interface in the application framework layer to launch the camera application. It then sends a command to start the camera by calling the camera device (Camera Device 1 and / or other camera devices) in the camera hardware abstraction layer. The camera hardware abstraction layer sends this command to the camera device driver in the kernel layer. This camera device driver can then start the corresponding camera sensor and acquire image light signals through the sensor. One camera device in the camera hardware abstraction layer corresponds to one camera sensor in the hardware layer.

[0097] Then, the camera sensor can transmit the acquired image light signal to the image signal processor for preprocessing to obtain the image electrical signal (raw image), and transmit the raw image to the camera hardware abstraction layer through the camera device driver.

[0098] The camera hardware abstraction layer can send the original image to the camera algorithm library. The camera algorithm library stores program code that implements the method for generating new perspective images provided in the embodiments of this application. Based on a digital signal processor and an image processor, the camera algorithm library executes the above code to achieve the aforementioned ability to generate new perspective images.

[0099] The camera algorithm library can detect and send raw images captured by the camera to the camera hardware abstraction layer. The camera hardware abstraction layer can then display these images. Simultaneously, the camera algorithm library can continuously output and display images from new perspectives; by sequentially playing these new perspective images, the desired effect can be achieved... Figure 1 The 3D camera movement effect shown.

[0100] Figure 9 This is a schematic diagram of the hardware structure of another electronic device according to an embodiment of this application. Figure 9As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0101] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0102] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0103] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0104] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0105] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0106] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, electronic device 100 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0107] In this embodiment, the electronic device 100 displays the original image captured by the camera, generates a new perspective image after executing the solution of this application, and continuously displays the generated new perspective images in sequence to achieve a 3D camera movement effect, which relies on the display functions provided by the GPU, the display screen 194, and the application processor.

[0108] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0109] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0110] Camera 193 is used to capture still images or videos. An object passes through the lens to generate an optical image that is projected onto a photosensitive element. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Internet Service Provider) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for processing. The DSP converts the digital image signal into image signals in standard formats such as RGB and YUV. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0111] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0112] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0113] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0114] Random access memory (RAM) can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data. Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into RAM for direct read and write by the processor 110.

[0115] In this embodiment, the code implementing the method described in this embodiment can be stored in non-volatile memory. When running a camera application, the electronic device 100 can load the executable code stored in the non-volatile memory into random access memory.

[0116] The pressure sensor 180A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be disposed on the display screen 194.

[0117] The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 about three axes (i.e., the x, y, and z axes). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and causes the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization.

[0118] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of electronic device 100, in a different position than display screen 194.

[0119] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] This application also provides an electronic device, which includes: one or more processors and a memory; the memory is coupled to one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to enable the electronic device to perform the steps in any of the above methods.

[0122] This application also provides a chip system applied to an electronic device. The chip system includes one or more processors, which invoke computer instructions to cause the electronic device to perform the steps in any of the methods described above. Optionally, the chip system further includes a memory electrically connected to the processor. Optionally, the chip system may also include a communication interface.

[0123] This application also provides a computer-readable storage medium storing instructions that, when executed by an electronic device, can implement any of the methods described above. This computer-readable medium may include at least: any entity or device capable of carrying computer program code (instructions) to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0124] This application also provides a computer program product, which includes a computer program that, when executed by an electronic device, can implement any of the above-described methods. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] In the embodiments provided in this application, it should be understood that the disclosed apparatus / devices and methods can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0130] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0131] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0132] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0133] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating images from new perspectives, characterized in that, include: An overlap block operation is performed on the image to be processed to obtain a first image block set. The first image block set includes multiple first image blocks, and the boundary of each first image block does not exceed the boundary of the image to be processed. Multi-plane image MPI representation is performed on the plurality of first image blocks respectively to obtain a plurality of first MPI representation images, with each first image block corresponding to a first MPI representation image; Homography transformation is performed on the plurality of first MPI characterization images to obtain a plurality of second MPI characterization images of the new viewpoint image; According to the depth of the plurality of first MPI characterization images, the plurality of second MPI characterization images are fused into the new perspective image.

2. The method according to claim 1, characterized in that, The image to be processed is subjected to an overlap block division operation to obtain a first image block set, including: Starting from the first vertex of the image to be processed, and taking the two sides where the first vertex is located as the starting sides, the first image block is sequentially overlapped and framed into the image to be processed using boxes of the same size. If the bounding box extends beyond the end edge of the image to be processed, the first image block will be drawn out into the image to be processed, starting from the intersection of the bounding box and the end edge. The end edge is the other two edges of the image to be processed besides the two starting edges.

3. The method according to claim 2, characterized in that, The overlap between adjacent first image blocks is within a preset overlap range.

4. The method according to claim 1, characterized in that, The step of fusing the multiple second MPI characterization images into the new viewpoint image according to the depth of the multiple first MPI characterization images includes: According to the depth of the plurality of first MPI characterization images, the plurality of second MPI characterization images are sorted in order from near to far or from far to near. The sorted multiple second MPI characterization images are fused into the new perspective image.

5. The method according to claim 1, characterized in that, The step of performing homography transformation on the plurality of first MPI characterization images to obtain a plurality of second MPI characterization images of the new viewpoint image includes: Based on the coordinates and optical centers of the first vertices of the plurality of first image blocks, determine the coordinates and optical centers of the first vertices of the plurality of second image blocks corresponding to the plurality of first image blocks in the new perspective image; Based on the coordinates of the first vertices and the optical center of the plurality of second image blocks, homography transformation and sampling are performed on the plurality of first MPI characterization images to obtain the plurality of second MPI characterization images.

6. The method according to claim 1, characterized in that, The method further includes: The plurality of first MPI characterization images are filled to obtain a plurality of third MPI characterization images; Homography transformation is performed on the plurality of third MPI characterization images to obtain a plurality of second MPI characterization images of the new perspective image.

7. The method according to claim 6, characterized in that, The fill size used during filling is greater than 0 and less than the maximum fill size, which is used to indicate the size corresponding to overfilling.

8. The method according to claim 6, characterized in that, The process of filling the plurality of first MPI characterization images to obtain a plurality of third MPI characterization images includes: The area to be filled is filled by copying the color values ​​of a first region in each first MPI characterization image to the area to be filled. The first region is a region that is less than or equal to a preset distance threshold from the boundary of the first MPI characterization image. The preset distance threshold is determined based on the size of the area to be filled. Fill the opacity value of the area to be filled with zero.

9. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 8.

10. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the method as described in any one of claims 1 to 8.

Images