A method for generating a grid picture, a wallpaper generation method, and an electronic device

Abstract

This application relates to the field of terminal technology, providing a method for generating a gridded image, a wallpaper generation method, and an electronic device. The method for generating a gridded image includes: obtaining a first depth boundary corresponding to a first image, wherein the first depth boundary is the boundary of a first depth image corresponding to the first image, and the first depth boundary includes at least one first pixel; setting at least one grid structure on the first depth image, such that the first pixel is distributed within the grid structure; given a preset positional correspondence between the first pixel and the grid structure, determining at least one first activation point based on the grid structure, wherein the first activation point includes at least one of the vertices and center points of each grid in the grid structure; and constructing a target grid structure for the first activation point, thereby generating a gridded image based on the target grid structure. The technical solution of this application can solve the problem of high computational load caused by dense grids in existing methods.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method for generating gridded images, a method for generating wallpapers, and an electronic device. Background Technology

[0002] With the development of electronic technology, mobile terminals are increasingly using 3D rendering to enhance realism and interactivity. In the 3D rendering process of 2D images, it is typically necessary to construct a 3D mesh from the 2D image to obtain a corresponding meshed image. Then, based on the meshed image, lighting models and rendering techniques are applied to give the meshed image a realistic visual effect, resulting in the final 3D image. Therefore, meshing 2D images is a crucial step in the 3D image generation process.

[0003] Existing image meshing methods typically construct a dense mesh structure using each pixel of the image as a vertex. This often results in millions of mesh faces, placing a significant burden on the storage and rendering capabilities of mobile devices. While mesh simplification algorithms can simplify the dense mesh structure and reduce the number of mesh faces, the simplification process itself is computationally intensive and time-consuming. Therefore, existing image meshing methods are computationally demanding and resource-intensive, making them particularly unsuitable for mobile devices with limited computing resources. Summary of the Invention

[0004] This application provides a method for generating gridded images, a method for generating wallpapers, and an electronic device to solve the problem of high computational load in existing gridded image generation methods.

[0005] To achieve the above objectives, in a first aspect, embodiments of this application provide a method for generating a gridded image, the method comprising:

[0006] Obtain the first depth boundary corresponding to the first image, the first depth boundary is the boundary of the first depth image corresponding to the first image, and the first depth boundary includes at least one first pixel; set at least one grid structure on the first depth image so that the first pixel is distributed in the grid structure; when the first pixel and the grid structure have a preset positional correspondence, determine at least one first activation point based on the grid structure, the first activation point includes at least one of the vertices and center points of the grid structure; construct a target grid based on the first activation point to generate a gridded image.

[0007] The mesh image generation method illustrated in this application determines activation points by utilizing the positional relationship between pixels at the image depth boundary and the mesh structure. These activation points are then used to determine the mesh vertices in the target mesh structure, thereby constructing the target mesh structure. Through this design, this application avoids constructing a dense mesh structure by treating every pixel as a vertex, and also eliminates the need for further simplification of the dense mesh structure. Therefore, the method proposed in this application has a lower computational load and consumes fewer computational resources, making it suitable for mobile terminals.

[0008] In one implementation, at least one grid structure is set on the first depth image, including: setting at least one level on the first depth image based on different resolutions, such that each level has a corresponding grid structure, wherein each resolution corresponds to one level, and each grid in the same level has the same size. Using this implementation, setting multiple grid structures based on different resolutions allows for analysis of the first depth image at different resolutions, simultaneously balancing computational efficiency and image detail.

[0009] In one implementation, at least one layer is set on the first depth image based on different resolutions, so that each layer has a corresponding grid structure. This includes: numbering the layers according to resolution from largest to smallest; determining the side length of the grid corresponding to each layer based on the layer number, so as to form a grid structure based on the side length; wherein the side length is the sum of 2 raised to the power of N and 1, and N is the layer number corresponding to the side length. Using this implementation, the layers are numbered according to the resolution, thereby determining the side length of the grid. In this way, different resolutions correspond to different grid side lengths; the higher the resolution, the smaller the grid side length and the denser the grid structure. Therefore, it is possible to achieve layer-by-layer refinement of the grid structure.

[0010] In one implementation, after setting at least one mesh structure on the first depth image, the method further includes: storing the first coordinates corresponding to the center of the edge of the mesh in the mesh structure, and the second coordinates corresponding to the center point of the mesh; determining whether the first pixel has a preset positional correspondence with the mesh structure based on the first and second coordinates, wherein the positional correspondence includes: the first pixel is located at the center point of the mesh in the mesh structure, and / or, the first pixel is located at the center of any edge of the mesh in the mesh structure. Using this implementation, the coordinates of the center point of the mesh and the coordinates of the center points of its edges can be pre-calculated and saved. Thus, when determining whether the first pixel has a positional correspondence with the mesh structure, it is only necessary to extract the pre-selected and saved coordinates and traverse them, improving the speed of determining the positional correspondence. Furthermore, this implementation pre-sets the positional correspondence to the first pixel being at the center of the mesh or the center of an edge. Since the center position has special characteristics, this reduces computational complexity and also facilitates bisection of the triangle based on the center in subsequent steps.

[0011] In one implementation, a target mesh structure is constructed for a first activation point to generate a meshed image based on the target mesh structure. This includes: determining a first pixel as a second activation point; forming an activation point set based on the first and second activation points; and constructing the target mesh structure based on the activation point set. This implementation constructs the activation point set based on the first pixel and the first activation point. Since the first pixel is a point on the depth boundary, and the first activation point is determined based on the first pixel, this implementation utilizes points on the depth boundary and the activation points radiating from them to construct the target mesh structure. This approach focuses the construction on the region corresponding to the depth boundary while avoiding overlooking other regions.

[0012] In one implementation, if the first pixel has a preset positional correspondence with the mesh structure, at least one first activation point is determined based on the mesh structure. This includes: traversing the meshes in the mesh structure in ascending order of their hierarchical numbers; if a mesh has a positional correspondence with an activation point in the activation point set, then the vertex or center point of the mesh is determined as the first activation point, and the first activation point is added to the activation point set. Using this implementation, the positional correspondence between the first pixel and the mesh structure is determined, and the activation point set is obtained through this pre-calculated positional relationship. Subsequent operations such as triangulation only require querying the activation point set, resulting in faster speed and making it more suitable for edge-side operation.

[0013] In one implementation, if there is a positional correspondence between the mesh and the activation points in the activation point set, then a vertex or center point in the mesh is determined as the first activation point. This includes: if the center point of the mesh in the mesh structure coincides with any activation point in the activation point set, then a vertex of the mesh is determined as the first activation point; and / or, if the center of any edge of the mesh in the mesh structure coincides with any activation point in the activation point set, then both the vertex and the center point of the mesh are determined as the first activation points. By adopting this implementation, for mesh regions that have a positional correspondence with activation points in the activation point set (i.e., the center of the mesh or the center of a mesh edge coincides with an activation point), the first activation point is determined, increasing the number of activation points in that region. Therefore, the mesh density in that region can be increased, i.e., the meshing accuracy of that region can be improved.

[0014] In one implementation, before constructing the target mesh structure based on the set of activation points, the method further includes: uniformly dividing the first depth image into multiple meshes at preset intervals; using the vertices of the uniformly divided multiple meshes as third activation points, and adding the third activation points to the set of activation points. This implementation uniformly sets several third activation points throughout the entire first depth image, ensuring that non-depth boundary regions are also covered by activation points, preventing the mesh in non-depth boundary regions from becoming too sparse, and improving the accuracy of meshing.

[0015] In one implementation, constructing a target mesh structure based on an activation point set includes: dividing the first depth image into two triangles using the diagonal of the first depth image; determining whether each triangle matches a preset condition, the preset condition being: the midpoint of the base of the triangle is an activation point in the activation point set, and the length of the base of the triangle is greater than... If a triangle matches the preset conditions, connect the midpoint of its base to the right-angle vertex of the triangle to divide it into two new triangles, and return to the step of determining whether each triangle matches the preset conditions. If a triangle does not match the preset conditions, determine the vertex connection relationships of the mesh in the target mesh structure on the two-dimensional plane based on the vertex connection relationships between multiple triangles. Based on the first depth image, determine the information of the vertices of the mesh in the target mesh structure in the depth direction. Based on the information in the depth direction and the vertex connection relationships on the two-dimensional plane, construct the target mesh structure in three-dimensional space. This implementation method uses triangular bisection to divide the mesh, obtaining the vertex connection relationships of the target network structure on the two-dimensional plane. Simultaneously, combined with the depth image, the vertex positions and connection relationships in three-dimensional space are obtained. In this constructed target network structure, adjacent mesh faces share edges, preventing cracks and avoiding holes during rendering.

[0016] In one implementation, the vertex connection relationships of the target mesh structure on the two-dimensional plane are determined based on the vertex connection relationships between multiple triangles. This includes: traversing a binary tree constructed based on triangles to obtain multiple leaf nodes on the binary tree. Each node on the binary tree corresponds to a triangle, and the node information of each node includes at least one of the following: vertex coordinates, midpoint coordinates of the base of the triangle corresponding to the node, child node information, and flag information. The flag information is used to identify whether the node is a leaf node. The vertex connection relationships of the triangles corresponding to the leaf nodes are determined as the vertex connection relationships of the target mesh structure on the two-dimensional plane. This implementation utilizes a binary tree to store the node information of the triangles and participates in the calculation. Because binary tree search is highly efficient, it can effectively improve the computational efficiency of the meshed image generation algorithm.

[0017] In one implementation, before traversing the binary tree constructed based on the triangles, the method further includes: constructing a binary tree with the triangles obtained from the first depth image partitioning as the root nodes; after dividing the triangles into two new triangles, constructing two child nodes on the corresponding nodes of the triangles, and updating the node information of the corresponding nodes and child nodes, where the two child nodes are the nodes corresponding to the two new triangles respectively. This implementation uses each node of the binary tree to represent a triangle and utilizes the hierarchical relationship of the binary tree to represent the containment relationship between triangles. The structure is simple and easy to understand and implement.

[0018] In one implementation, obtaining the first depth boundary corresponding to the first image includes: performing a depth estimation algorithm on the first image to obtain a first depth image corresponding to the first image, wherein the depth estimation algorithm is performed using a monocular depth estimation network, and the monocular depth estimation network includes at least one of MiDaS network, DORN network, Monodepth network, and DepthGAN network; and obtaining the boundary of the first depth image as the first depth boundary corresponding to the first image. Using this implementation, the depth of the first image is estimated using a monocular depth estimation network to obtain its corresponding depth image. Then, in subsequent steps, the boundary of the first image is obtained based on the depth corresponding to the pixels in the image.

[0019] In one implementation, obtaining the boundary of the first depth image includes: calculating the gradient of each pixel in the first depth image using the Sobel operator; constructing an exponential function using the gradient and a preset adjustment coefficient; processing each pixel in the first depth image using the exponential function to obtain a processed first depth image; and obtaining the boundary in the processed first depth image. This implementation uses the Sobel operator to perform edge detection on the first depth image to extract the depth boundary in the image. Furthermore, in subsequent steps, targeted mesh construction methods are applied to both boundary and non-boundary locations.

[0020] In one implementation, after obtaining the processed first depth image, the method further includes: for each pixel in the processed first depth image, adjusting gray values ​​greater than or equal to a preset threshold to 1, and gray values ​​less than the preset threshold to 0, to obtain a padded first depth image; in the padded first depth image, determining the pixel with a gray value of 1 as the first pixel. Using this implementation, based on the preset threshold and the relationship between pixel gray values, pixels with weaker boundaries are set to 0, considered not to be on the boundary, thus achieving the padded first depth image. This results in clearer boundaries. Furthermore, since the refinement of depth boundaries conforms to physical laws, it improves the accuracy of boundary recognition and ensures rendering quality.

[0021] In one implementation, after constructing the target mesh structure based on the first activation point, the method further includes: using the color information of the first image to color the target mesh structure, obtaining a colored meshed image. This implementation colors each mesh in the target mesh structure, assigning corresponding colors and textures to the mesh faces, thereby converting the black-and-white meshed image into a color image.

[0022] Secondly, this application provides a wallpaper generation method, which includes: in response to a wallpaper selection instruction, determining a pre-selected wallpaper image from at least one image in a gallery, wherein the pre-selected wallpaper image is a two-dimensional image; processing the pre-selected wallpaper image using the mesh image generation method as described in the first aspect and any implementation thereof, to obtain a target network structure corresponding to the pre-selected wallpaper image; performing 3D rendering on the target network structure corresponding to the pre-selected wallpaper image using a renderer to obtain a colored mesh image, and using the colored mesh image as a three-dimensional wallpaper image.

[0023] The wallpaper generation method illustrated in this application embodiment utilizes the aforementioned method for generating gridded images to generate corresponding gridded images based on two-dimensional images. The gridded images are then rendered to output three-dimensional wallpaper images with a more realistic 3D effect. Compared to traditional two-dimensional wallpapers, this application embodiment enhances the user's visual experience.

[0024] Thirdly, this application also provides an electronic device, including a memory and a processor; the memory and the processor are coupled; wherein the memory is used to store computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, it causes the electronic device to perform the gridded image generation method as described in the first aspect and any implementation thereof, or the wallpaper generation method as described in the second aspect thereof.

[0025] Fourthly, this application also provides a chip system, which includes a processor; the processor is coupled to a memory, the memory being used to store computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, the method for generating a gridded image as described in the first aspect and any implementation thereof or the method for generating a wallpaper as described in the second aspect thereof is executed.

[0026] Fifthly, this application also provides a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the gridded image generation method as described in the first aspect and any implementation thereof, or the wallpaper generation method as described in the second aspect.

[0027] Sixthly, this application also provides a computer program product, which includes: a computer program or instructions that, when executed on a computer, cause the computer to perform the gridded image generation method as described in the first aspect and any implementation thereof, or the wallpaper generation method as described in the second aspect.

[0028] Understandably, the beneficial effects that the technical solutions provided in the third to sixth aspects above can achieve can be referred to the beneficial effects in the first aspect and any of its optional implementation methods, and will not be repeated here. Attached Figure Description

[0029] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 It is a display effect of a 3D wallpaper image from different viewing angles;

[0031] Figure 2 This is a schematic diagram illustrating the effect of constructing a triangular mesh structure using a quadtree-based triangulation method.

[0032] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application;

[0034] Figure 5 This is the first flowchart of the method for generating a gridded image provided in the embodiments of this application;

[0035] Figure 6 This is a schematic diagram of a depth image before processing and a depth image after processing provided in an embodiment of this application;

[0036] Figure 7 This is a schematic diagram of a first pixel distributed in a grid structure according to an embodiment of this application;

[0037] Figure 8 This is a schematic diagram of a hierarchical arrangement provided in an embodiment of this application;

[0038] Figure 9 This is a schematic diagram of grid structure division at each level provided in an embodiment of this application;

[0039] Figure 10 This is a schematic diagram of a first coordinate and a second coordinate provided in an embodiment of this application;

[0040] Figure 11 This is the second flowchart of the method for generating a gridded image provided in the embodiments of this application;

[0041] Figure 12 This is a schematic diagram of an activation strategy provided in an embodiment of this application;

[0042] Figure 13 This is a schematic diagram of a uniformly divided grid provided in an embodiment of this application;

[0043] Figure 14 This is the third flowchart of the method for generating a gridded image provided in the embodiments of this application;

[0044] Figure 15 This is a schematic diagram of a triangulation process provided in an embodiment of this application;

[0045] Figure 16 This is a schematic diagram of an activation point and a corresponding schematic diagram of vertex connection relationships provided in an embodiment of this application;

[0046] Figure 17 This is the fourth flowchart of the method for generating a gridded image provided in the embodiments of this application;

[0047] Figure 18 The embodiments provided in this application are based on Figure 15 A schematic diagram of the binary tree constructed by triangulation is shown;

[0048] Figure 19 This is a flowchart of a wallpaper generation method provided in an embodiment of this application;

[0049] Figure 20 This is a schematic diagram illustrating the processing of a pre-selected wallpaper image using the aforementioned image meshing method, as provided in an embodiment of this application.

[0050] Figure 21 This is provided by the embodiments of this application. Figure 20 Enlarged view of the target mesh structure;

[0051] Figure 22 This is a schematic diagram of a user interface for a 3D wallpaper setting scene provided in an embodiment of this application;

[0052] Figure 23 This is a schematic diagram of a gridded image generation device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the protection scope of this application.

[0054] In the following description, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0055] Furthermore, in this application, directional terms such as "upper," "lower," "inner," and "outer" are defined relative to the indicated placement of the components in the accompanying drawings. It should be understood that these directional terms are relative concepts, used for relative description and clarification, and can change accordingly depending on the placement of the components in the accompanying drawings.

[0056] The following explanations of the technical terms mentioned in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0057] Triangular mesh structures, used in computer graphics and 3D modeling, are a widely applied data structure in fields such as computer graphics, computer-aided design (CAD), and geographic information systems (GIS). A triangular mesh structure consists of a series of interconnected triangular meshes that collectively cover a two-dimensional region or a three-dimensional surface. In this embodiment, the triangular mesh structure can be used to generate 3D wallpapers.

[0058] A patch, used in computer graphics and 3D modeling, is one of the basic units that constitute a 3D mesh structure. Specifically, a patch is a planar polygon formed by connecting three or more vertices, used to represent the surface area of ​​a 3D object. For example, patches can be laid on each triangle in a triangular mesh structure to construct complex 3D meshes and models. In the embodiments of this application, laying patches on the mesh of the constructed target network structure yields a meshed image.

[0059] The application scenarios of the embodiments of this application will be described below with reference to the accompanying drawings.

[0060] With the development of electronic technology, electronic devices can use 3D rendering technology to enhance realism and interactivity. For example, 3D rendering technology can be used to generate 3D wallpapers for personalized screen displays. 3D wallpapers include three dimensions: length, height, and depth. Compared to traditional 2D wallpapers, 3D wallpapers add a depth dimension. This allows for different images to be displayed from different viewing angles, enabling viewers to perceive the three-dimensional position and shape of objects in the image. The added depth dimension makes wallpaper images more vivid and realistic.

[0061] Figure 1 It is a 3D wallpaper image displayed from different perspectives.

[0062] like Figure 1 As shown, this 3D wallpaper image depicts the image of a dog. Figure 1 As shown in Figure A, this represents the display effect when viewed from the front. Figure 1 As shown in B, this is the display effect viewed by a user from a certain angle on the right. It can be seen that the display effect of the 3D wallpaper image changes with the change of viewing angle.

[0063] The aforementioned 3D wallpaper image can be obtained by performing 3D rendering on a 2D image. Specifically, the 3D rendering process of a 2D image typically involves meshing the image. Then, based on the generated 3D mesh image, lighting models and rendering techniques are applied to give the mesh image a realistic visual effect, resulting in the final 3D image.

[0064] Meshing can be used to explicitly represent 3D geometry using triangular mesh structures, quadrilateral mesh structures, etc., with triangular mesh structures being the most widely used. A triangular mesh structure consists of a series of interconnected triangular meshes. Triangular planes (hereinafter referred to as triangular mesh patches) are laid on these triangular meshes, and a 3D model can be constructed by piecing together these triangular mesh patches. For example, a cube can be composed of 6 faces, and each face's square shape is composed of 2 triangular mesh patches; that is, 12 triangular mesh patches are pieced together to form a cube. It can be understood that complex 3D images may be composed of tens of thousands of triangular mesh patches, each of which is part of its surface.

[0065] However, the image meshing method based on triangular meshes has the following problems.

[0066] Currently, mesh structures are typically constructed by directly using each pixel of an image as the vertex of each mesh. For example, for a 1000×1000 pixel image, using all pixels as mesh vertices, 1000×1000 vertices can construct a mesh structure containing 999×999×2 triangles. It is evident that this method of using each pixel as a mesh vertex results in a very dense mesh, requiring a large number of triangular mesh faces. Therefore, a significant amount of storage space is needed to store the information of mesh vertices, edges, and faces (i.e., triangular mesh faces). Furthermore, during rendering, performing calculations on a large number of triangles consumes substantial computational resources. Mobile devices, in particular, are designed with portability and energy efficiency in mind, and their storage space and computing performance are typically weaker than those of computers. Therefore, mobile devices are even less able to handle the computational burden of processing a massive number of triangular mesh faces, which can cause lag or even crashes.

[0067] Considering the above problems, mesh simplification algorithms can be used to simplify the dense mesh structure. Mesh simplification algorithms can reduce the number of vertices, edges, and faces in complex mesh structures. Commonly used mesh simplification algorithms include: vertex extraction algorithms, which remove vertices and their adjacent faces from the mesh structure and then re-triangulate the holes created by the deletion operation; vertex clustering algorithms, which cluster multiple vertices into one and then update the mesh faces based on the clustered vertices; and edge shrinking algorithms, which shrink an edge into a single point and remove the face corresponding to that edge. These mesh simplification algorithms can reduce the complexity of the mesh and the number of triangular mesh faces. However, the simplification process for dense mesh structures also requires significant computational resources.

[0068] In addition, some solutions use quadtree-based triangulation methods to construct triangular mesh structures. However, this method uses both active and inactive nodes to build the triangular mesh structure, and gaps exist between the faces corresponding to active and inactive nodes. This means that adjacent faces in the generated mesh structure do not share edges. These gaps between adjacent faces can lead to holes in the final meshed image.

[0069] Figure 2 This is a schematic diagram illustrating the effect of constructing a triangular mesh structure using a quadtree-based triangulation method.

[0070] Figure 2 A is a planar schematic diagram of a triangular mesh structure constructed using a quadtree-based triangulation method. Figure 2 The diagram shows a 3D effect of constructing a triangular mesh structure using a quadtree-based triangulation method. Figure 2 The white triangles in the center represent triangular mesh panels, and the shaded areas represent the gaps between these panels. Figure 2From A, we know that adjacent triangular meshes are flush with each other in the two-dimensional plane. From Figure 2 As shown in B, there are gaps between adjacent triangular meshes in 3D space. Therefore, the scheme for constructing triangular mesh structures based on quadtree triangulation requires additional patching to fill the gaps and prevent holes from appearing in the rendered image.

[0071] To address the aforementioned issues, this application provides a method for generating gridded images.

[0072] The method for generating a meshed image provided in this application first determines activation points by utilizing the positional relationship between pixels on the image depth boundary and a pre-set mesh structure. Then, mesh vertices are determined from these activation points to construct a target mesh structure containing multiple meshes. Since this application does not use every pixel as a mesh vertex to construct the mesh structure, the constructed mesh structure has a smaller number of meshes, eliminating the need for further simplification. Therefore, the method proposed in this application consumes fewer computational resources and is more suitable for mobile terminals with limited storage space and computing performance. Furthermore, since all mesh vertices in this application are activation points with the same resolution, there is no issue of adjacent faces not sharing edges, thus eliminating the need for additional repair of cracks between adjacent faces.

[0073] The triangular mesh construction method provided in this application can be applied to electronic devices. In some embodiments, the electronic device may be a mobile phone, tablet computer, handheld computer, personal computer (PC), ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, vehicle device, and other mobile terminals. This application does not impose any special restrictions on the specific type of electronic device.

[0074] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0075] like Figure 3As shown, the electronic device 100 may include a processor 110, a memory 120, an antenna 1, an antenna 2, a mobile communication module 130, a wireless communication module 140, a sensor module 150, a display screen 160, etc. The sensor module 150 may include a pressure sensor 150A, a touch sensor 150B, a fingerprint sensor 150C, an ambient light sensor 150D, a temperature sensor 150E, a gyroscope sensor 150F, a proximity sensor 150G, etc. In this embodiment, wallpaper selection commands from the user can be received through the pressure sensor 150A and the touch sensor 150B.

[0076] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0077] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0078] The memory 120 can be used to store computer executable program code, including instructions. The memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the foldable electronic device 100 (such as audio data, phonebook, etc.). Furthermore, the memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the foldable electronic device 100 by running instructions stored in the memory 120 and / or instructions stored in memory located within the processor.

[0079] In this embodiment, the code implementing the gridded image generation method and wallpaper generation method described in this embodiment can be stored in non-volatile memory. When setting a 3D wallpaper, the electronic device 100 can load the executable code stored in the non-volatile memory into random access memory.

[0080] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 130, wireless communication module 140, modem, and baseband processor.

[0081] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0082] The mobile communication module 130 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 130 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 130 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 130 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 130 and at least some modules of the processor 110 may be housed in the same device.

[0083] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device or displays an image or video through the display screen 160. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 130 or other functional modules.

[0084] The wireless communication module 140 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 140 can be one or more devices integrating at least one communication processing module. The wireless communication module 140 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 140 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0085] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 130, and antenna 2 is coupled to wireless communication module 140, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0086] Electronic device 100 implements display functions through a GPU, a display screen 160, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 160 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0087] In this embodiment, the electronic device 100 implements the triangular mesh construction method and wallpaper generation method provided in this embodiment, which mainly rely on the image computing and processing capabilities provided by the GPU.

[0088] Display screen 160 is used to display images, videos, etc. Display screen 160 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 160, where N is a positive integer greater than 1.

[0089] In this embodiment of the application, the electronic device 100 displays a gallery to enable users to select preset wallpaper images from the gallery, and the ability to display three-dimensional wallpaper images, which relies on the display functions provided by the GPU, the display screen 160, and the application processor.

[0090] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0091] Generally, the implementation of the functions of electronic device 100 requires not only hardware support but also software cooperation. The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes the layered architecture Android system as an example to illustrate the software structure of electronic device 100.

[0092] Figure 4 This is a schematic diagram of the software structure of the electronic device provided in the embodiments of this application.

[0093] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0094] The application layer can include a series of application packages.

[0095] like Figure 4 As shown, the application package can include applications such as camera, calendar, map, WLAN, music, SMS, gallery, call, navigation, video, and wallpaper.

[0096] A gallery application is an application used to manage and browse pictures and videos. The gallery can respond to user commands, selecting and displaying images stored on an electronic device for the user to browse and choose from.

[0097] A wallpaper application is an application that provides wallpapers. It can provide a download path for wallpaper resources, allowing users to download the corresponding wallpaper images and directly set them as the background of their electronic devices. Furthermore, a wallpaper application can also provide the function of setting images stored on the electronic device as wallpapers. In this embodiment, the wallpaper application can process a two-dimensional image selected by the user through a gallery to obtain a three-dimensional wallpaper image corresponding to that two-dimensional image.

[0098] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0099] like Figure 4 As shown, the application framework layer may include a window manager, content provider, phone manager, resource manager, notification manager, view system, wallpaper management service (WallpaperManagerService), etc.

[0100] The content provider stores and retrieves data, making that data accessible to applications. This data may include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0101] Wallpaper management services provide wallpaper resources, download paths for wallpaper resources, and also manage and set wallpapers. They can include features such as wallpaper libraries, personalized recommendations, downloads and settings, community features, and user uploads.

[0102] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0103] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0104] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0105] System libraries can include multiple functional modules. For example: surface manager, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), media libraries, etc.

[0106] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0107] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0108] A 2D graphics engine is a graphics engine for 2D drawing.

[0109] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0110] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0111] The following example, using a 3D wallpaper setting scene, illustrates the workflow of the software and hardware of electronic device 100.

[0112] First, the wallpaper application calls the wallpaper management service in the application framework layer to launch the wallpaper application. Then, the wallpaper application calls the content provider in the application framework layer to access images stored in the local gallery of the electronic device. After the user selects an image from the local gallery, the wallpaper application calls the 3D graphics processing library in the system library to perform the meshing processing operation provided in this embodiment on the selected image, ultimately obtaining a 3D wallpaper image.

[0113] Figure 5 This is the first flowchart of the method for generating a gridded image provided in the embodiments of this application.

[0114] like Figure 5 As shown, the method includes the following steps S11-S18.

[0115] Step S11: Obtain the first depth image corresponding to the first image.

[0116] In this step, the 2D image to be meshed is used as the first image, and its corresponding depth image, i.e., the first depth image, is obtained. The first image is a 2D image, which can be a color image, a grayscale image, or a black and white image. The depth image is a grayscale image; the grayscale value of each pixel in the depth image indicates the distance information (i.e., depth information) between that point and the camera. For example, a pixel with no depth information has a grayscale value of 0. Through the first depth image, the distance from each point in the first image to the camera can be determined.

[0117] In one implementation, step S11 may include the following steps S111-S112.

[0118] Step S111: Obtain the first image.

[0119] In step S111, a first image is first acquired so that it can be meshed in subsequent steps. The method of acquiring the first image can be selected by the user according to the actual application scenario, and this application embodiment does not limit it. For example, the first image can be acquired by taking a photo, or it can be determined from the images stored in the local map library, or it can be downloaded from the Internet or local area network.

[0120] Step S112: Perform a depth estimation algorithm on the first image to obtain the first depth image corresponding to the first image.

[0121] In step S112, a first depth image is obtained using a depth estimation algorithm. Specifically, the depth estimation algorithm can infer the distance between each pixel in a single two-dimensional image and the camera, thereby obtaining information about the image in the depth direction. Considering that two-dimensional images are usually captured by a monocular camera, a monocular depth estimation network can be used for depth estimation of the two-dimensional target image in this embodiment.

[0122] For example, a Mixed-scale Dense Depth (MiDaS) network can be used to perform depth estimation algorithms to estimate the depth of a single image. The MiDaS network is a monocular depth estimation model based on a neural network that can process input color images using a pre-trained neural network and output a corresponding depth map. In this embodiment, inputting a two-dimensional first image into the MiDaS network yields the corresponding depth image, i.e., the first depth image. Besides the MiDaS network, other monocular depth estimation networks can also be used to perform depth estimation algorithms, such as DORN networks, Monodepth networks, and DepthGAN networks.

[0123] It should be noted that steps S111-S112 only provide a method for obtaining the first depth image based on a monocular depth estimation network. In some implementations, other methods can also be used to obtain the first depth image. For example, a pre-trained conditional generative adversarial network (such as Pix2Pix or CycleGAN) can be used to generate the first depth image by utilizing the mapping from image to depth map learned during training. It is understood that the above is only an exemplary description of the method for obtaining the first depth image. In practical applications, other methods for obtaining the first depth image can also be used, and this application does not limit them.

[0124] Step S12: Obtain the boundary of the first depth image as the first depth boundary corresponding to the first image. The first depth boundary includes at least one first pixel.

[0125] In step S12, it can be understood that during the capture of a 2D image, the actual distance between different positions on the captured 3D object and the lens varies, especially at the boundaries between objects, and at points where the object's contours and textures change. This is reflected in the depth image as different pixels having different depths, particularly at boundaries, contours, and texture changes, where significant depth variations may occur. When rendering a 2D image into a 3D image, locations with larger depth differences (i.e., more significant depth changes) are more prone to discontinuities or unnatural appearances. For example, contours and boundaries with significant depth variations may exhibit stretching artifacts and other defects.

[0126] Considering that areas with larger depth differences in 2D images are more prone to artifacts during rendering, this application implements targeted meshing for these artifact-prone areas to reduce the number of meshes while maintaining rendering quality. Specifically, denser meshes are used for areas with larger depth differences. Dense meshes can accommodate more faces, thus reducing artifacts. For areas without depth differences, relatively sparse meshes are used to reduce the number of faces and thus lower computational load. This application's approach of determining mesh density based on depth differences achieves a balance between rendering quality and computational efficiency.

[0127] Based on this, embodiments of this application obtain the locations where depth differences exist, and then set a dense grid at those locations. Specifically, the locations where depth differences exist in the first image are the boundaries of the first depth image (i.e., the first depth boundary), such as the outline of an object in the aforementioned analysis.

[0128] In specific application scenarios, visual and image processing techniques can be used to obtain depth boundaries. This application embodiment does not specifically limit the method for obtaining depth boundaries. For example, for the first image in this application embodiment, deep learning methods can be used, employing deep learning models such as convolutional neural networks to predict the depth of each pixel in the first image, thereby obtaining depth boundaries where depth differences exist. For example, a specific deep learning model can also be used to process the first image to obtain a corresponding depth image, from which depth boundaries can be extracted. For example, images from two or more viewpoints can be used to infer the depth of each pixel in the first image using parallax information, thereby determining the depth boundaries based on the depth of each pixel. For example, multiple images of the same scene can be captured under different lighting conditions, and the depth of each pixel in the first image can be inferred by analyzing changes in reflected light.

[0129] In one implementation, step S12 may include the following steps S121-S126.

[0130] Step S121: Calculate the gradient of each pixel in the first depth image using the Sobel operator.

[0131] In step S121, edge detection is performed on the depth image obtained in the preceding steps based on the gradients of each pixel in the first depth image to obtain the depth boundary locations where defects are likely to occur. It can be understood that since there is a depth difference at the depth boundary locations, i.e., the depth value changes, and the significance of the numerical change can usually be represented by the gradient, the boundaries of the depth image can be obtained based on the gradient values ​​of each pixel in the depth image. Specifically, gradient-based edge detection algorithms such as the Sobel algorithm, the Prewitt algorithm, and the Canny algorithm can be used.

[0132] For example, if the Sobel algorithm is used, the gradient value corresponding to each pixel in the first depth image is first calculated using the Sobel operator, that is, the gradient of the gray value of each pixel. The Sobel operator is an operator used in the Sobel edge detection algorithm. It can use convolution operations to filter the depth image, thereby obtaining the gradient information of the gray value of each pixel in the depth image.

[0133] Step S122: Construct an exponential function using the gradient and a preset adjustment coefficient.

[0134] In step S122, the gradient value corresponding to each pixel can be used as the exponential parameter, and the exponential parameter can be adjusted using a pre-set adjustment coefficient to obtain the final exponential parameter. An exponential function is constructed with e as the base and the exponential parameter obtained in the previous steps as the exponent.

[0135] Step S123: Process each pixel in the first depth image using an exponential function to obtain the processed first depth image.

[0136] In step S123, since each pixel in the depth image has a corresponding gradient value, substituting the gradient value into the exponential function constructed in the previous steps yields the exponential function value corresponding to each pixel. Using the exponential function value as the grayscale value corresponding to that pixel, a new depth image, i.e., the depth boundary image, can be obtained.

[0137] Specifically, the boundary of the depth image D can be calculated using the following formula (1) to obtain the processed depth image, which is the depth boundary image A:

[0138] (1)

[0139] in To adjust the coefficients of the exponential function, one possible solution is... . This is the Sobel operator, used to calculate the gradient of a pixel in an image. express The norm, that is The length of the vector.

[0140] Figure 6 This is a schematic diagram of a depth image before processing and a depth image after processing provided in an embodiment of this application.

[0141] Figure 6 In this image, A represents the depth image before processing. Figure 6 B in the text is related to Figure 6 The A in the image corresponds to the processed depth image, also known as the depth boundary map.

[0142] like Figure 6 As shown in Figure A, because the horse is closer to the lens, it will appear as a lighter color, while the ground, trees, and other background elements will appear as darker colors because they are farther from the lens. This is understandable, as it's for easier explanation. Figure 7 In image A, the horse is depicted as white, and the entire background is rendered as black. However, in real-world applications, different parts of the horse's body are at varying distances from the camera, resulting in depth and color differences. For example, the horse's nose and eyes might be closer to the camera than other parts of its body, thus appearing lighter in the depth map.

[0143] like Figure 6 As shown in B, Figure 6 The number B in the image clearly shows the horse's outline. Understandably, this is done for easier depiction. Figure 6 The B map only shows the outline of the horse, but in real-world applications, the depth boundary map may also show details of various parts of the horse's body, such as the ears, eyes, and nose.

[0144] From the aforementioned formula (1) and Figure 6 It can be seen that in the processed depth image (i.e., depth boundary image A), the gray value of each pixel contains the saliency information of the depth change at the location of the pixel, rather than the depth information of the pixel itself.

[0145] It should be noted that steps S121-S123 only provide a method for obtaining depth boundary images based on gradients. In some implementations, other methods can also be used to obtain depth boundary images. For example, the Laplacian algorithm can be used, employing a convolution kernel with second derivatives to obtain the depth image boundary. The Difference of Gaussians (DoG) algorithm can also be used, calculating the difference between the results of two Gaussian filters at different scales to obtain the depth image boundary. It is understood that the above are merely illustrative descriptions of depth boundary image acquisition methods; in practical applications, other depth boundary image acquisition methods can be used, and this application does not limit these methods.

[0146] Step S124: For the grayscale value of each pixel in the processed first depth image, adjust the grayscale value that is greater than or equal to the preset threshold to 1, and adjust the grayscale value that is less than the preset threshold to 0, to obtain the padded first depth image.

[0147] Step S125: In the padded first depth image, determine the pixel with a gray value of 1 as the first pixel.

[0148] In steps S124-S125, considering the influence of factors such as image noise, uneven illumination, and algorithm limitations, the depth boundary obtained in the aforementioned steps may be inaccurate. To address this, the depth image processed in step S123 can be padded to ensure its resolution meets the requirements of practical application scenarios, such as achieving the resolution of the first image.

[0149] Specifically, image padding can be performed based on the relationship between the grayscale value of each pixel and a preset threshold. It can be understood that the larger the grayscale value of a pixel, the greater the depth difference at that pixel's location, the more significant the depth change, and the greater the probability that the pixel is located at a depth boundary. Conversely, the smaller the grayscale value, the less likely the pixel is to be located at a depth boundary. However, for cases where the grayscale value is neither particularly large nor particularly small, it is impossible to accurately determine whether a pixel is at a depth boundary. To address the inability to accurately determine whether a pixel is at a depth boundary, this embodiment pre-sets a threshold. If the grayscale value is greater than or equal to the threshold, the grayscale value is considered sufficiently large, and the pixel corresponding to that grayscale value is within the depth boundary. Otherwise, the grayscale value is considered too small, and the pixel corresponding to that grayscale value is not within the depth boundary.

[0150] For example, the first depth image (i.e., the depth boundary image) processed in the aforementioned steps The grayscale value of each pixel in the image can be greater than or equal to the threshold. The grayscale value is adjusted to 1, which is less than the threshold. The grayscale value is adjusted to 0. At this point, it can be considered that the first depth image after padding (i.e., the adjusted depth boundary image) is... In this context, pixels with a grayscale value of 1 are points within the depth boundary, while pixels with a grayscale value of 0 are not. Therefore, a pixel with a grayscale value of 1 can be considered the first pixel, while a pixel with a grayscale value of 0 is not considered the first pixel.

[0151] It can be understood that a larger preset threshold results in more points being considered within the depth boundary. Therefore, there are more first pixels, leading to more detailed depiction of the depth boundary, a denser grid at the depth boundary, and a better-quality meshed image. However, this increases the number of first pixels, slowing down the computation. Conversely, a smaller preset threshold results in a sparser grid at the depth boundary, but also speeds up the computation. Therefore, a more reasonable preset threshold can be set by considering both computational efficiency and the desired level of detail in the meshed image. For example, a preset threshold of 0.5 could be used.

[0152] It is understandable that the purpose of steps S124-S125 is to improve the accuracy of the depth boundaries, thereby improving the rendering effect of the final meshed image. In some implementations, steps S124-S125 may be omitted, and other processing methods that can improve the accuracy of the depth boundaries may be used instead. For example, morphological operations (such as dilation and erosion) may be used to smooth the depth boundaries, or edge enhancement algorithms may be used to highlight the depth boundaries. In other implementations, to improve computational speed and reduce computational load, steps S124-S125 may be omitted, and step S126 may be executed directly after step S123.

[0153] Step S126: In the processed first depth image, obtain the boundary, the first depth boundary including at least one first pixel.

[0154] In step S126, the processed first depth image (i.e., the depth boundary image) can be used as a basis. This involves obtaining the depth boundary. Specifically, in the depth boundary image, the grayscale value of each pixel can contain significant information about the depth change at the pixel's location. Since the depth change at the boundary is relatively significant, the boundary can be obtained based on the grayscale value of each pixel in the depth boundary image.

[0155] Furthermore, if steps S124-S125 were performed before step S126, then the first depth image after padding (i.e., the adjusted depth boundary image) can be used as a basis. This allows us to obtain depth boundaries. Specifically, we can use an image... In this model, pixels with a grayscale value of 1 are used as points in the depth boundary, and all points with a grayscale value of 1 form the boundary of the depth image.

[0156] After obtaining the boundary of the depth image, several first pixel points can be determined within the boundary. For example... Figure 6 As shown in B, point P is a first pixel. Multiple first pixels can outline... Figure 6 The outline of the horse in B. It can be understood that the first pixel must be a point within the depth boundary, but a point within the depth boundary is not necessarily the first pixel.

[0157] It should be noted that steps S121-S126 only provide one method for obtaining the first depth boundary. In some implementations, other methods can also be used to obtain the first depth boundary. For example, an encoder-decoder structure such as U-Net can be used to estimate the depth of the first image. Then, based on the depth estimation result, an edge detection algorithm is applied to extract the depth boundary. Alternatively, an image segmentation method can be used to segment the first image into regions with similar features (such as color and texture), and then the first depth boundary can be determined based on the boundaries of each region. Furthermore, the color and texture features of the first image can be combined with the depth features of the first depth image to form a comprehensive feature representation. Using a trained deep learning model, the color and texture features and depth features are processed simultaneously to obtain the first depth boundary. It is understood that the above are merely illustrative descriptions of depth boundary acquisition methods; in practical applications, other depth boundary acquisition methods can also be used, and this application does not limit them.

[0158] Step S13: Set at least one grid structure on the first depth image so that the first pixel points are distributed in the grid structure.

[0159] In step S13, a grid structure is pre-set in the depth image, and the first pixel is distributed within the grid structure, thus forming a certain positional relationship between the first pixel and the pre-set grid structure. Subsequently, the target grid structure can be constructed based on the positional relationship between the first pixel and the pre-set grid structure.

[0160] Each network structure can be pre-defined based on certain rules. For example, the grid in the network structure can be set as a square, triangle, parallelogram, etc., with a preset side length. The rules between different network structures can be different. For example, if two grid structures are pre-defined in the depth image, namely grid structure 1 and grid structure 2, then the side length of the grid in network structure 1 and the side length of the grid in network structure 2 can be different, or the shape of the grid in network structure 1 and the shape of the grid in network structure 2 can be different.

[0161] Figure 7 This is a schematic diagram of a first pixel distributed in a grid structure according to an embodiment of this application.

[0162] like Figure 7 As shown, Figure 7 The depth image in the image contains multiple first pixels (such as...) Figure 7 A grid structure is set on this depth image (with black dots in the center). Therefore, the first pixels are distributed in the grid structure, and each first pixel has a relative position with the grid cells in the grid structure, forming a certain positional relationship.

[0163] In one implementation, step S13 may include the following step S131.

[0164] Step S131: Based on different resolutions, set at least one layer on the first depth image so that each layer has a corresponding grid structure.

[0165] In step S131, one or more layers are set on the first depth image, and the resolutions of the multiple layers are different. By setting a corresponding layer for each resolution, a grid structure corresponding to the resolution of that layer can be set in each layer, thereby enabling the analysis of the first depth image based on different resolutions in different layers.

[0166] Within this mesh structure, each grid cell has the same size, while grid cells in different structures have different sizes. It can be understood that the higher the resolution of a layer, the denser the mesh structure needs to be to represent more detail, and therefore the smaller the size of each grid cell. Conversely, the lower the resolution of a layer, the sparser the mesh structure needs to be, and the larger the size of each grid cell.

[0167] Figure 8 This is a schematic diagram of a hierarchical setup provided in an embodiment of this application.

[0168] like Figure 8 As shown, based on different resolutions, three layers are set on the first depth image, each layer having a grid structure. Specifically, the resolution of 1920×1080 corresponds to layer 1, which has a 3x3 grid structure; the resolution of 1280×720 corresponds to layer 2, which has a 2x2 grid structure; and the resolution of 800×600 corresponds to layer 3, which has a grid structure containing only one grid.

[0169] In one implementation, step S131 may include the following steps S1311-S1312.

[0170] Step S1311: Number the levels according to their resolution from largest to smallest.

[0171] In step S1311, the layers are numbered according to a certain order based on multiple different resolutions, for example, numbered in descending order of resolution. In this way, the first depth image can be analyzed layer by layer according to the layer number, that is, analyzed layer by layer in order from fine to coarse.

[0172] For example, as described above Figure 8The three layers in the image are numbered as layer 1, layer 2, and layer 3, in descending order of resolution. When analyzing the first depth image according to the layer number, layer 1 with a resolution of 1920×1080 can be analyzed first, followed by layer 2 with a resolution of 1280×720, and finally layer 800×600.

[0173] It is understandable that the more layers there are, the more detailed the analysis of the first-depth image will be.

[0174] Alternatively, the layers can be numbered in other orders based on multiple different resolutions. For example, they can be numbered in ascending order of resolution.

[0175] Step S1312: Determine the side length of the grid corresponding to each level according to the level number, so as to form a grid structure based on the side length.

[0176] In step S1312, different grid side lengths are set for the grid structure in each layer. It can be understood that the level of image detail varies with resolution. The higher the resolution of a layer, the more detailed the image display at that layer. To achieve a more detailed display, the grid structure at that layer needs to be denser. Therefore, the grid structure contains more grid cells. Conversely, the lower the resolution of a layer, the sparser the grid structure at that layer, and the fewer grid cells it contains.

[0177] Therefore, based on the correlation between resolution and the density of the preset mesh structure, the side length of the mesh in the network structure can be determined according to the resolution. Since the resolution and layer number correspond in the aforementioned step S1311, the side length of the mesh structure in the layer can also be determined according to the layer number. For example, the side length of the mesh in a mesh structure can be set to the sum of 2 raised to the power of N and 1, where N is the layer number corresponding to this mesh structure.

[0178] For example, let the resolution of the first depth image be... Then, the levels can be numbered sequentially according to their resolution from largest to smallest to obtain a level list. The list contains several different levels. ,in And the hierarchical structure within it. Its resolution is the highest of all levels, for In the numbered hierarchy In the middle, the grid structure consists of multiple quadrilateral grids. Composition, i.e., hierarchy ,in This represents the number of grid cells in the grid structure. In this grid structure, each quadrilateral grid cell... The side length is determined according to the hierarchical number. Confirmed, for .

[0179] Figure 9 This is a schematic diagram of the grid structure division in each level provided in an embodiment of this application.

[0180] Figure 9 In this context, A represents the level. Schematic diagram of the grid structure. Figure 9 B in the text represents the level. Schematic diagram of the grid structure.

[0181] like Figure 9 As shown, Figure 9 A mesh structure is created for a 5×5 resolution depth image. The specific process is as follows: Result in the number of levels Therefore, two layers are set for this depth image. and .like Figure 9 As shown in A, in the hierarchy middle, Quadrilateral grid The side length is .like Figure 9 As shown in B, in the hierarchy middle, Quadrilateral grid The side length is .

[0182] S14: Stores the first coordinates corresponding to the center of the edge of the grid in the grid structure, and the second coordinates corresponding to the center point of the grid.

[0183] In step S14, for each level of the grid structure, the coordinates used to determine the positional correspondence are recorded. This allows for direct retrieval of the pre-saved coordinates for calculations in subsequent steps of determining positional correspondences, thus improving computational efficiency.

[0184] S15: Based on the first coordinate and the second coordinate, determine whether the first pixel has a preset positional correspondence with the grid structure. The positional correspondence includes: the first pixel is located at the center point of the grid in the grid structure, and / or the first pixel is located at the center of any edge of the grid in the grid structure.

[0185] In step S15, for example, if the positional correspondence includes the first pixel being located at the center point of the grid in the grid structure, then the coordinates of the grid center point can be pre-stored. If the positional correspondence includes the first pixel being located at the center of an edge of the grid in the grid structure, then the coordinates of the center of the grid edge can be pre-stored.

[0186] It is understandable that coordinates of other locations can also be stored for other location correspondences. For example, if the location correspondence includes the first pixel being located at a vertex of any grid in the grid structure, then the coordinates of the grid vertices can be pre-stored.

[0187] Figure 10 This is a schematic diagram of a first coordinate and a second coordinate provided in an embodiment of this application.

[0188] like Figure 10 As shown, a quadrilateral grid contains a center point and the midpoints of the four sides. The coordinates of the center of the quadrilateral grid's sides (i.e., the first coordinates) and the coordinates of the grid's center point (i.e., the second coordinates) are stored. When the positional correspondence is determined later, the specific values ​​of each coordinate can be substituted into the calculation to determine whether the active point is also located at that coordinate position.

[0189] It is understandable that the purpose of steps S14-S15 is to pre-store the coordinates required for the calculation, thereby improving the calculation speed. Therefore, in some implementations, steps S14-S15 can be omitted, and step S16 can be executed directly after step S13 to reduce the required storage space.

[0190] Step S16: When the first pixel has a preset positional correspondence with the grid structure, at least one first activation point is determined based on the grid structure, so as to use the first activation point as the set of activation points.

[0191] In step S16, based on the positional correspondence between the first pixel and the mesh structure, a corresponding first activation point is determined in the mesh structure, and an activation point set is constructed. This operation increases the number of activation points while ensuring that the added activation points are related to the first pixel, i.e., related to the depth boundary. Therefore, a relatively dense mesh structure can be constructed in the depth boundary region.

[0192] The position of the first activation point can be determined based on the grid corresponding to the first pixel. For example, if the relative position of the first pixel to the grid structure conforms to a preset positional correspondence, then the vertex of the grid corresponding to the first pixel is taken as the first activation point. For example, if the relative position of the first pixel to the grid structure conforms to a preset positional correspondence, then the center of the grid corresponding to the first pixel is taken as the first activation point.

[0193] Furthermore, the first activation point can also be set to a point at another location in the grid corresponding to the first pixel, such as the midpoint of a grid edge or a point on the grid diagonal. In this way, if the relative position of the first pixel to the grid structure conforms to a preset positional correspondence, the midpoint of the edge of the grid corresponding to the first pixel is used as the first activation point, or a point at a specific location on the grid diagonal is used as the first activation point. Alternatively, the first activation point can also be set to a point in another grid, such as a point in a grid adjacent to the grid corresponding to the first pixel.

[0194] Furthermore, if the positional relationship between the first pixel and the grid in the grid structure does not match the preset positional correspondence, no operation will be performed.

[0195] Step S17: Construct a target mesh structure for the set of activation points to generate a meshed image based on the target mesh structure.

[0196] In step S17, the target network structure is constructed using the activation points in the activation point set, and patches are added to each grid of the target network structure to obtain a meshed image. Alternatively, each activation point in the activation point set can participate in the computation to construct the target mesh structure. Or, several representative activation points can be selected from the activation point set, and these selected activation points can then be used to obtain the target network structure.

[0197] For example, activation points in the activation point set can be used as mesh vertices to construct a target mesh structure containing multiple meshes. Based on the distribution of activation points in different regions, the target mesh structure can be made relatively dense in the depth boundary region and relatively sparse in the non-depth boundary region.

[0198] For example, directly connecting activation points can yield a target mesh structure containing polygonal meshes such as triangles, quadrilaterals, and hexagons. Alternatively, based on specific optimization criteria or geometric constraints, such as maximum side length constraints, connecting activation points can yield a target mesh structure containing regular or irregular meshes. Algorithms such as minimum spanning tree and shortest path algorithms can also be used to connect activation points to obtain the target mesh structure. Furthermore, other methods can be used to connect activation points, which will not be elaborated upon here.

[0199] In one implementation, step S17 may be followed by step S18.

[0200] Step S18: Using the color information of the first image, color the target grid structure to obtain a colored gridded image.

[0201] In step S18, considering that the depth boundary does not contain color information from the first image, the target network structure obtained based on the depth boundary also does not contain color information. To obtain a colored mesh image, the color information from the first image can be used to color the target mesh structure. The color information from the first image can be RGB (Red, Green, Blue), HSV (Hue Saturation Value), HSL (Hue Saturation Lightness), etc., and is not limited here.

[0202] It's understandable that the target network structure consists of multiple triangular meshes, each mesh representing a hole, which cannot be colored. Therefore, the first step is to add a facet to each triangular mesh within the target network structure to fill the holes. Then, each facet can be colored individually to obtain a colored mesh image.

[0203] Figure 11 This is the second flowchart of the method for generating a gridded image provided in the embodiments of this application.

[0204] like Figure 11 As shown, the method includes the following steps S21-S29.

[0205] Step S21: Obtain the first depth image corresponding to the first image.

[0206] Step S22: Obtain the boundary of the first depth image as the first depth boundary corresponding to the first image. The first depth boundary includes at least one first pixel.

[0207] Step S23: Set at least one grid structure on the first depth image so that the first pixel points are distributed in the grid structure.

[0208] Steps S21-S23 are the same as steps S11-S13 mentioned above, and will not be repeated here.

[0209] Step S24: Determine the first pixel as the second activation point. Construct an activation point set based on the second activation point.

[0210] In step S24, an activation point set is constructed based on the first pixel. Unlike the previous embodiment, which determines the first activation point based on the first pixel and then constructs the activation point set based on the first activation point, this embodiment also uses the first pixel as an activation point in the activation point set.

[0211] Step S25: Traverse the grids in the grid structure in ascending order of the level number.

[0212] In step S25, the vertex activation strategy is executed by traversing each mesh sequentially. Specifically, a double loop can be performed based on both the layer and the mesh. For example, first, the meshes in the mesh structure corresponding to layer 1 are traversed, then the meshes in the mesh structure corresponding to layer 2 are traversed, until the meshes corresponding to all layers have been traversed.

[0213] Step S26: If the mesh and the activation points in the activation point set have a positional correspondence, then determine the vertex or center point in the mesh as the first activation point and add the first activation point to the activation point set.

[0214] In step S26, an activation strategy is executed. Specifically, during the mesh traversal, for each mesh visited, the positional correspondence between the mesh and each activation point in the activation point set is analyzed, and a first activation point is obtained based on the analysis results. The activation point set may include the second activation point from step S24, and may also include the first activation point determined based on the positional correspondence in the current step S26.

[0215] In one implementation, step S26 may include steps S261 and / or S262.

[0216] Step S261: If the center point of the grid in the grid structure coincides with any active point in the set of active points, then the vertex of the grid is determined as the first active point.

[0217] Step S262: If the center of any edge of the mesh in the mesh structure coincides with any active point in the set of active points, then the vertex of the mesh and the center point of the mesh are determined as the first active point.

[0218] In steps S261 and S262, the positional correspondence can be that the center point of the grid in the network structure coincides with an activation point in the set of activation points, or that the center of any edge of the grid in the network structure coincides with an activation point in the set of activation points. Different activation strategies can be adopted for different positional correspondences. For example, for the positional correspondence where the center point of the grid coincides with an activation point, an activation strategy of activating the grid vertex can be adopted to determine the grid vertex as the first activation point. For the positional correspondence where the center of the grid edge coincides with an activation point, an activation strategy of activating both the grid vertex and the center point can be adopted to determine both the grid vertex and the center point as the first activation points.

[0219] Figure 12 This is a schematic diagram of an activation strategy provided in an embodiment of this application.

[0220] Figure 12 A is a schematic diagram of determining the first activation point when there is a positional correspondence between the grid and the activation point. Figure 12B is a schematic diagram of determining the first activation point when the grid and the activation point have another positional correspondence.

[0221] like Figure 12 As shown in Figure A, if an activation point a is exactly at the center of a quadrilateral grid, then the four vertices A1-A4 of the quadrilateral grid are activated, that is, these four vertices are determined as the first activation points. Figure 12 In A, the arrow points to the first activation point.

[0222] like Figure 12 As shown in Figure B, if an activation point b falls exactly at the center of an edge of a quadrilateral grid (as shown in the figure, at the center of the left side of the quadrilateral grid), then the four vertices B1-B4 of the quadrilateral grid and the center point B5 of the quadrilateral grid are activated. In other words, these four vertices and the center point are all determined as the first activation points. Figure 12 In B, the arrow points to the first activated point.

[0223] It can be understood that steps S25-S26 essentially involve analyzing the positional relationship between activation points and the meshes in the mesh structure. Therefore, in some implementations, the mesh traversal method described in S25-S26 can be used. The mesh traversal method specifically involves traversing the mesh and determining whether any activation point exists at the center of each mesh or the center of a mesh edge. In other implementations, the activation point traversal method can also be used. This method involves traversing the activation points in the activation point set and determining whether each activation point is at the center of any mesh or the center of a mesh edge. In practical execution, considering computational efficiency, the mesh traversal method can be used.

[0224] For example, the following is an activation method based on traversing activation points: Let the set of activation points be... The first pixel is used as the second activation point to form an initial set of activation points. Therefore, in the initial stage of implementing the activation strategy, there exists The correspondence. Traverse the set of activation points. For each activation point, determine its positional relationship with each grid cell in the grid structure at each level. If an activation point is at the center of any grid cell, then the vertex of that grid cell is designated as the first activation point, and the first activation point is added to the activation point set. In the process of determining whether an activation point is located at the center of an edge in any grid, both the vertex and the center point of that grid are designated as the first activation point, and this first activation point is added to the set of activation points. middle.

[0225] For example, the following is an activation method based on traversing a grid:

[0226] In this activation method, the input is the initial set of activation points. Hierarchical List Uniformly divide the grid point set The output is a set of activation points. .

[0227] The pseudocode for this activation method is as follows:

[0228]

[0229] for do

[0230]

[0231] for in do

[0232] if The center point is at then

[0233] Will Add four vertices to

[0234] else if The center point of the edge is at then

[0235] Will Add the four vertices and the center point to

[0236] else

[0237] continue

[0238]

[0239] return

[0240] The above is the pseudocode for the activation method.

[0241] In the pseudocode of the activation method, " "Indicates: will" Assign to The set of activation points at this time equal to the initial set of activation points "for "do" indicates: based on hierarchical numbering Analyze each level in ascending order. "Indicates: hierarchy" Assignment. "for in "do" indicates: analyze the hierarchy sequentially. Each quadrilateral grid in "if The center point is at "then" and "will" Add four vertices to Indicates: If the quadrilateral grid The center point in the set of activation points In the middle, the quadrilateral grid is used. The four vertices are added to the activation point set as activation points. middle. "else if The center point of the edge is at "then" and "will" Add the four vertices and the center point to "Indicates: If the quadrilateral grid..." The center point of the edge is in the set of activation points In the middle, the quadrilateral grid is used. The four vertices and the center point are added to the set of activation points as activation points. In the middle, "else" and "continue" indicate: if the quadrilateral grid... The center point and the center point of the edge are not in the set of activation points. If the loop continues, then the loop will continue. "Indicates: Set of activation points" and set The union of the two sets serves as the new set of activation points. The dot (.) indicates "return". "Indicates the set of output activation points" .

[0242] In the pseudocode of the aforementioned activation method, the first pixel is used as the second activation point to form the initial set of activation points. and will The set of activation points before the traversal begins During the traversal, newly determined first activation points are added to the activation point set in real time, updating the activation point set. That is, the final set of activation points. It is formed based on the first and second activation points, resulting in the final set of activation points. Used to construct the target network structure.

[0243] In the pseudocode of the aforementioned activation method, the input includes a uniformly divided set of grid points. For uniformly divided grid point sets The settings can be configured using the following steps S27-S28.

[0244] Step S27: Divide the first depth image into multiple grids evenly according to a preset interval.

[0245] In step S27, it can be understood that the first pixel is distributed in the depth boundary region. Generally, the closer to the center of the depth boundary region, the denser the distribution of the first pixel. Conversely, the closer to the non-depth boundary region, the sparser the distribution of the first pixel. Since the first and second activation points are determined based on the first pixel, the first and second activation points are also usually more densely distributed in the depth boundary region and more sparsely distributed in the non-depth boundary region.

[0246] To address the issue of an oversimplified network structure in non-deep boundary regions due to a limited number of activation points, some activation points can be set in these regions as well.

[0247] Specifically, a grid is uniformly divided in the first depth image at certain intervals to set activation points. The interval can be set based on the resolution of the first image; for example, for a 1000×1000 pixel image, the interval can be set to 10 pixels. A straight line is drawn every 10 pixels along both the x-axis and y-axis, thus creating a 100-row, 100-column grid structure in the first depth image. In addition to square grids, rectangular, triangular, and parallelogram grids can also be created. For example, drawing a straight line every 10 pixels along the x-axis and every 15 pixels along the y-axis results in a rectangular grid structure.

[0248] Figure 13 This is a schematic diagram of a uniformly divided grid provided in an embodiment of this application.

[0249] like Figure 13 As shown, in the first depth image, nine straight lines are drawn horizontally and vertically at certain intervals, with the distance between adjacent lines being the same. This division method yields a 10x10 grid structure, where each square grid cell has the same size. Connecting the diagonals of each square grid cell creates a grid structure containing multiple triangular grid cells, all of the same size.

[0250] Step S28: Use the vertices of the evenly divided grids as the third activation points and add the third activation points to the activation point set.

[0251] In step S28, after uniformly dividing the mesh, the vertices of the mesh are added to the set of activation points as third activation points. As described above. Figure 13In this approach, the vertices of a uniformly divided triangular mesh can be used as third activation points and added to the activation point set. The activation point set then includes the first, second, and third activation points. Therefore, the activation points in non-depth boundary regions will not be too sparse. Consequently, the network structure built based on the activation points in the activation point set will not be excessively sparse in non-depth boundary regions, preventing oversimplification in those regions.

[0252] In addition to using the vertices of a uniformly divided grid as the third activation point, other points on the grid, such as the center point of the grid, can also be selected as the third activation point.

[0253] It is understandable that the purpose of steps S27-S28 is to increase the mesh density in non-depth boundary regions, avoiding an overly coarse image caused by sparse meshes in these regions. In some implementations, steps S27-S28 can be omitted, and step S29 can be executed directly after step S26. For example, in cases where the background is relatively simple, setting a sparse mesh in the background region has little impact on the image's fineness and can even improve the overall processing speed. For instance, if a 2D image includes a solid-color background and the image of a horse, then there is no need to meticulously depict the solid-color background. Therefore, for this 2D image, a third activation point can be omitted, and only the first and second activation points need to be set.

[0254] Step S29: Construct the target mesh structure for the set of activation points.

[0255] For step S29, please refer to step S18 above, and it will not be repeated here.

[0256] In one implementation, step S29 may be followed by the following steps.

[0257] Using the color information from the first image, the target mesh structure is colored to obtain a colored mesh image.

[0258] For details on this step, please refer to step S19 above, which will not be repeated here.

[0259] Figure 14 This is the third flowchart of the method for generating a gridded image provided in the embodiments of this application.

[0260] like Figure 14 As shown, the aforementioned step S19 includes the following steps S19a-S19f.

[0261] Step S19a: Divide the first depth image into two triangles using the diagonal of the first depth image.

[0262] In step S19a, the vertices of the processed first depth image (i.e., the depth boundary image) can be used as the starting points for triangulation. A diagonal line is drawn in the depth image to obtain two isosceles right triangles. Then, based on these two isosceles right triangles, a triangular mesh is formed.

[0263] Step S19b: Determine whether each triangle matches the preset conditions. The preset conditions are: the midpoint of the base of the triangle is an active point in the set of active points, and the length of the base of the triangle is greater than... .

[0264] Step S19c: If the triangle matches the preset conditions, connect the midpoint of the base to the right-angle vertex of the triangle to divide the triangle into two new triangles, and return to step S19b.

[0265] In steps S19b-S19c, triangular mesh generation is performed. Specifically, a cyclical method is used to determine whether each triangle matches a preset condition. If it does not match, no operation is performed on that triangle. If it matches, the triangle is divided into two smaller triangles, and then each of the two smaller triangles is checked to see if it matches the preset condition. Further triangular mesh generation is then performed on each of the two smaller triangles. This process is repeated until every triangle no longer matches the preset condition.

[0266] Figure 15 This is a schematic diagram of a triangulation process provided in an embodiment of this application.

[0267] Figure 15 In the diagram, A represents a schematic of dividing the first depth image into two triangles, a and b. Figure 15 B in the diagram is a schematic diagram of dividing triangles a and b into smaller triangles a1, a2 and b1, b2 respectively. Figure 15 C in the diagram is a schematic diagram of dividing triangle a1 into two smaller triangles a11 and a12.

[0268] like Figure 15 As shown in Figure A, the four dots represent the four vertices A1-A4 of the first depth image. The figure contains a diagonal line from the upper left to the lower right (from A1 to A4). This diagonal line divides the first depth image into two isosceles right triangles a and b, and the vertices of isosceles right triangles a and b are also vertices of the first depth image.

[0269] like Figure 15 As shown in Figure B, the diamond-shaped black dots B1 and B2 represent the activation points. Figure 15As shown in point B, the midpoints of the bases of isosceles right triangles a and b coincide with the activation point B1. Therefore, we can connect the right-angle vertex and the midpoint of the base of each triangle. That is, connect the lower left corner point A3 to the center rhombus black dot B1, and connect the upper right corner point A2 to the center rhombus black dot B1. In this way, isosceles right triangle a is divided into two smaller triangles a1 and a2, and isosceles right triangle b is divided into two smaller triangles b1 and b2. It can be seen that smaller triangles a1, a2 and b1, b2 are also isosceles right triangles.

[0270] like Figure 15 As shown in C, the midpoint of the base of small triangle a1 coincides with the activation point B2. Therefore, we can connect the right-angle vertex B2 of triangle a1 to the midpoint of its base B1, dividing triangle a1 into two smaller triangles, a11 and a12. It can be seen that triangles a11 and a12 are also isosceles right triangles. Since neither of the midpoints of the bases of smaller triangles a11 and a12 has an activation point, we do not perform a bisection operation on triangles a11 and a12. Similarly, the midpoints of the bases of the other triangles a2, b1, and b2 do not have activation points, so we also do not perform a bisection operation on triangles a2, b1, and b2.

[0271] This implementation method understandably performs a bisection operation on the triangle when the midpoint of the triangle's base coincides with an activation point. If the midpoint of the triangle's base is not an activation point, the triangle cannot be bisectioned. Therefore, using the midpoint of the triangle's base as an activation point in the set of activation points can be used as a preset condition. If the triangle does not match this preset condition, the bisection operation on that triangle should be stopped.

[0272] Furthermore, considering that the minimum pixel value of each side of the quadrilateral is 1, the isosceles right triangles formed by dividing the quadrilateral have two legs of length 1 and a base of length 1. That is, the base length of the smallest isosceles right triangle is... Therefore, the side length of the base of the triangle can be greater than... As a prerequisite, if the length of the base of the triangle is not greater than... If the triangle is too small to be bisected, then the bisecting operation on the triangle is stopped.

[0273] Step S19d: If the triangles do not match the preset conditions, determine the vertex connection relationships of the meshes in the target mesh structure on the two-dimensional plane based on the vertex connection relationships between multiple triangles.

[0274] In step S19d, when none of the triangles in the diagram satisfy the bisection condition, the triangular mesh division is stopped. The vertex connections of each triangle at this point are obtained. These vertex connections are then used as the vertex connections of the target mesh structure on the two-dimensional plane. In other words, the projection of the target network structure onto the two-dimensional plane coincides with the lines connecting the vertices of the triangles.

[0275] Figure 16 This is a schematic diagram of an activation point and a corresponding vertex connection relationship provided in an embodiment of this application.

[0276] Figure 16 A in the diagram is an activation point. Figure 16 B in the text is related to Figure 16 The diagram shows the vertex connection relationship corresponding to A in the diagram.

[0277] like Figure 16 As shown in Figure A, the graph contains several dots, which are the activation points in the set of activation points. Figure 16 As shown in B, draw a diagonal line from the top left to the bottom right corner of the image, dividing the square image into two isosceles right triangles. Then, using recursion, divide each triangle that meets the preset conditions into two new smaller triangles, until the base of all triangles no longer meets the preset conditions. After all triangles have been divided, the vertex connections of the triangles in the final network structure represent the vertex connections of the mesh in the target network structure on the two-dimensional plane. It can be seen that the final network structure consists of several triangles, each triangle being a projection of a triangular mesh in the target network structure onto the two-dimensional plane.

[0278] Step S19e: Based on the first depth image, determine the information of the vertices of the mesh in the target mesh structure in the depth direction.

[0279] Step S19f: Based on the information in the depth direction and the vertex connection relationship on the two-dimensional plane, construct the target mesh structure in three-dimensional space.

[0280] In steps S19e-S19f, it can be understood that the triangle division in the aforementioned steps is performed on the processed depth image (i.e., the depth boundary image). The depth boundary image only contains the depth difference information of each pixel in the image, not the depth information itself. Therefore, the divided triangles also do not contain the depth information of the first image. That is, the vertex connection relationships of the triangles are connections on a two-dimensional plane, not connections in the depth dimension. The purpose of this solution is to generate a three-dimensional target network structure. Therefore, in addition to using the vertex connection relationships of the triangles as the vertex connection relationships of the network structure on a two-dimensional plane, depth information also needs to be extracted from the first depth image as the depth dimension information of the target network structure. Specifically, firstly, based on the vertex connection relationships of the target network structure on a two-dimensional plane, the positions of the mesh vertices on the two-dimensional plane can be determined. Then, based on the depth information of each pixel contained in the first depth image, the depth information of the pixels at the positions of the mesh vertices can be determined as the depth information of the mesh vertices. In this way, by combining the information on the two-dimensional plane and the information in the depth dimension, the information of each mesh in the target mesh structure in three-dimensional space can be obtained, thus obtaining the three-dimensional target mesh structure.

[0281] Figure 17 This is the fourth flowchart of the method for generating a gridded image provided in the embodiments of this application.

[0282] like Figure 17 As shown, the aforementioned step S19 includes the following steps S19g-S19o.

[0283] Step S19g: Divide the first depth image into two triangles using the diagonal of the first depth image.

[0284] Step S19g is the same as step S19a mentioned above, and will not be repeated here.

[0285] Step S19h: Construct a binary tree using the triangles obtained from the first depth image as root nodes.

[0286] In step S19h, considering that a binary tree can clearly demonstrate the process of bisectioning the triangle in the aforementioned steps, and that the process of bisectioning the triangle matches the recursive definition of a binary tree, the binary tree can be used to perform the triangular mesh partitioning process. Based on this, a binary tree is constructed using triangles. First, the two largest triangles, i.e., the two triangles obtained by partitioning using the diagonal of the first depth image, are taken as root nodes. In this way, two binary trees are obtained, with the root node of each binary tree corresponding to a triangle.

[0287] Step S19i: Determine whether each triangle matches the preset conditions. The preset conditions are: the midpoint of the base of the triangle is an active point in the set of active points, and the length of the base of the triangle is greater than 10 ... .

[0288] Step S19j: If the triangle matches the preset conditions, connect the midpoint of the base to the right-angle vertex of the triangle to divide the triangle into two new triangles.

[0289] Steps S19i-S19j are the same as steps S19b-S19c mentioned above, and will not be repeated here.

[0290] Step S19k: After dividing the triangle into two new triangles, construct two child nodes on the corresponding nodes of the triangles, update the node information of the corresponding nodes and child nodes, and return to step S19i.

[0291] In step S19k, for each large triangle divided into two smaller triangles, two new nodes are created in the binary tree. For example, if triangle a is divided into triangles a1 and a2, two child nodes are created on the node corresponding to triangle a, corresponding to triangles a1 and a2 respectively. In this way, the parent-child relationship between nodes in the binary tree can be used to represent the relationship between the large and small triangles.

[0292] Figure 18 The embodiments provided in this application are based on Figure 15 The diagram shows a binary tree constructed through triangulation.

[0293] Figure 18 The diagram shows the construction process of a binary tree with triangle a as a node, where A, B, and C are the nodes. Figure 18 D in the diagram is a schematic diagram of two binary trees constructed with triangle a and triangle b as root nodes respectively.

[0294] like Figure 18 As shown in A, firstly, two triangles are divided using the diagonal of the first depth image, which are then used as the root nodes of the binary tree, and the node information of the root nodes is stored. As for the aforementioned... Figure 16 Given an isosceles right triangle a in A, construct a binary tree with a as the root node. At this point, the binary tree contains only the root node a.

[0295] like Figure 18As shown in B, after dividing triangle a into two new triangles a1 and a2, two child nodes are created for the nodes corresponding to triangle a. These two child nodes are the nodes corresponding to the two new triangles a1 and a2, respectively. That is, the root node a is the parent node of triangles a1 and a2, and triangles a1 and a2 are the two child nodes of the root node a. At the same time, the node information corresponding to triangle a is updated, and the node information corresponding to triangles a1 and a2 is stored separately.

[0296] like Figure 18 As shown in C, after dividing triangle a1 into a11 and a12, triangles a11 and a12 are set as the two child nodes of node a1. This process is repeated recursively until all triangles matching the preset conditions have been divided. Furthermore, the same method is applied to triangle b. Figure 18 The binary tree is constructed using methods A through C.

[0297] like Figure 18 As shown in D, these are the two final binary trees. The root nodes of these two binary trees are the nodes corresponding to triangles a and b, respectively, and each triangle has a corresponding node in the binary tree.

[0298] In a binary tree, each node's information includes one or more of the following: vertex coordinates, midpoint coordinates of the base of the corresponding triangle, child node information, and flag information. The vertex coordinates determine the positions of the three vertices of the triangle. The midpoint coordinates determine whether the midpoint coincides with an active point in the set of active points, and if so, the midpoint is used to bisection the triangle. The child node information contains information about the smaller triangles resulting from each subdivision, recording the hierarchical relationship of the triangular bisection. The flag information identifies whether the node is a leaf node. The initial flag information for each node can be set to true. After constructing the child nodes of a node, the child node information can be updated, and the node's flag information can be changed to false to indicate that the node is not a leaf node.

[0299] The following is a triangulation method based on binary tree partitioning:

[0300] In this algorithm, the input is a set of activation points. The initial triangular node set consisting of two root nodes. The output is a binary tree with two triangular nodes. , .

[0301] The pseudocode for this triangulation method is as follows:

[0302] for in do

[0303] if It is a leaf node and The length of the base is greater than and The center point of the bottom edge is at then

[0304] Connect the center point of the base to the right-angle vertex, and... Divided into two triangular nodes

[0305] renew Child node information, and add two child nodes to

[0306] The leaf node flag is set to false.

[0307] else

[0308] continue

[0309] by A binary tree with triangular nodes is obtained from the root node. ,

[0310] return ,

[0311] The above is the pseudocode for the triangulation method.

[0312] In the pseudocode of this triangulation method, "for in "do" means: traverse the set of triangle nodes. All nodes "if It is a leaf node and The length of the base is greater than and The center point of the bottom edge is at "then", "connect the center point of the base to the right-angle vertex, and..." Divide into two triangle nodes, update Child node information, and add two child nodes to "and" "Leaf node flag set to false" means: if the node It is a leaf node, and the node If the corresponding triangle meets the preset conditions, then the node will be... The corresponding triangle is divided into two smaller triangles, resulting in two child nodes. The nodes are then updated. Information about child nodes is collected, and the child nodes are added to the triangle node set. In the middle, the last updated node Specifically, the node information is updated by setting the leaf node flag to false. "else" and "continue" indicate that if the node is not a leaf node or does not meet the preset conditions, the loop continues. "return" , "Indicates: Output binary tree" and .

[0313] It is understandable that steps S19h and S19k utilize the special data structure of a binary tree to store and perform operations on the triangular nodes. In some implementations, steps S19h and S19k can be omitted, and other data structures such as linked lists can be used instead of binary trees; the specific steps will not be elaborated here.

[0314] Step S19l: If the triangle does not match the preset conditions, traverse the binary tree constructed based on the triangle to obtain multiple leaf nodes on the binary tree.

[0315] In step S191, after constructing the binary tree based on the triangle, the tree is traversed starting from the root node to obtain all leaf nodes. The traversal order can be depth-first, such as preorder, inorder, and postorder, or breadth-first, such as level-order. As described above... Figure 19 Given two binary trees obtained from D, for the binary tree with root node 'a', traverse starting from root node 'a'. Taking a depth-first preorder traversal as an example, check nodes a, a1, a11, a12, a2 ​​in sequence to obtain leaf nodes a11, a12, and a2. For the binary tree with root node 'b', traverse starting from root node 'b'. Taking a depth-first preorder traversal as an example, check nodes b, b1, b2 in sequence to obtain leaf nodes b1 and b2. Therefore, determine the vertex connections between triangles a11, a12, a2, b1, and b2.

[0316] Step S19m: Determine the vertex connection relationship of the triangle corresponding to the leaf node as the vertex connection relationship of the mesh in the target mesh structure on the two-dimensional plane.

[0317] In step S19m, the vertex connectivity of the mesh in the target network structure is determined using leaf nodes. From the aforementioned... Figure 16 As can be seen from C, the vertex connection relationships between triangles a11, a12, a2, b1, and b2 are determined, which means the triangle is... Figure 16The mesh structure in C. Based on this, the vertex connection relationship of the triangles corresponding to the leaf nodes of a binary tree can be used as the vertex connection relationship of the mesh in the target mesh structure on the two-dimensional plane.

[0318] Step S19n: Based on the first depth image, determine the information of the vertices of the mesh in the target mesh structure in the depth direction.

[0319] Step S19o: Based on the information in the depth direction and the vertex connection relationship on the two-dimensional plane, construct the target mesh structure in three-dimensional space.

[0320] Steps S19n-S19o are referred to in the aforementioned steps S19e-S19f, and will not be repeated here.

[0321] Figure 19 This is a flowchart of a wallpaper generation method provided in an embodiment of this application.

[0322] like Figure 19 As shown, the method includes the following steps S31-S33.

[0323] Step S31: In response to the wallpaper selection instruction, determine a pre-selected wallpaper image from at least one image in the gallery, wherein the pre-selected wallpaper image is a two-dimensional image.

[0324] In step S31, the user selects a pre-selected wallpaper image from the gallery. Specifically, the user first issues a wallpaper selection command via touch, pressing, voice control, etc., and the electronic device responds by displaying two-dimensional images stored in the gallery. Then, the user selects an image from the two-dimensional images displayed by the electronic device as the pre-selected wallpaper image. The pre-selected wallpaper image is the image the user wishes to set as a three-dimensional wallpaper.

[0325] Step S32: Process the pre-selected wallpaper image to obtain the target network structure corresponding to the pre-selected wallpaper image.

[0326] In step S32, the pre-selected wallpaper image is processed into a grid using the gridded image generation method described in the foregoing embodiments to obtain a three-dimensional target grid structure. For detailed processing methods, please refer to steps S11-S18, S21-S29, S19a-S19f, and S19g-S19o in the foregoing embodiments; they will not be repeated here.

[0327] Step S33: Use a renderer to perform 3D rendering on the target network structure corresponding to the pre-selected wallpaper image to obtain a colored mesh image, and use the colored mesh image as a 3D wallpaper image.

[0328] In step S33, the three-dimensional target mesh structure and the two-dimensional wallpaper are input into the renderer, and the three-dimensional wallpaper image is obtained by using the color information of the two-dimensional wallpaper and the three-dimensional target mesh structure.

[0329] Figure 20 This is a schematic diagram illustrating how the aforementioned image meshing method is used to process a pre-selected wallpaper image, as provided in an embodiment of this application.

[0330] Figure 21 This is provided by the embodiments of this application. Figure 20 A magnified view of the target mesh structure.

[0331] like Figure 20 and Figure 21 As shown, firstly, depth estimation is performed on the input color 2D image (i.e., the color pre-selected wallpaper image) to obtain the corresponding depth image. Then, edge detection is performed on the depth image to obtain the depth boundary of the pre-selected wallpaper image. Based on the pixels in the depth boundary, multiple activation points are obtained. These activation points are used as input to a triangulation algorithm to divide the image into triangles. Based on the vertex connection relationships of the divided triangles, the connection relationships of the grid vertices on the 2D plane are obtained.

[0332] Finally, based on the color information (such as RGB information), depth image, and vertex connectivity of the input 2D image, a 3D wallpaper image is obtained. Specifically, based on the connectivity of the mesh vertices in the 2D plane and the depth image, a 3D target mesh structure without color information can be obtained. For example... Figure 21 The image shows a 3D target mesh structure without color information. Then, using a renderer, based on the color information of the input 2D image, a 3D rendering of the target mesh structure without color information is performed. In this way, a colored 3D mesh image is obtained through 3D rendering and used as a 3D wallpaper image.

[0333] In one implementation, after obtaining the 3D wallpaper image, it can be set as the wallpaper of an electronic device, saved in the local wallpaper library of the wallpaper management service, or uploaded to the wallpaper resource community of the wallpaper management service.

[0334] Figure 22 This is a schematic diagram of the user interface for a three-dimensional wallpaper setting scene provided in an embodiment of this application.

[0335] like Figure 22 As shown in Figure A, the main interface 10 of the electronic device can display various types of application icons, including a wallpaper application icon 101. In response to a user's action of opening the wallpaper application, such as a first click or first touch 11 on the wallpaper application icon 101, the electronic device invokes the wallpaper management service of the application framework layer to launch the wallpaper application.

[0336] like Figure 22 As shown in B, this is the wallpaper application interface 20 displayed on the electronic device after the wallpaper application is launched. The wallpaper application interface 20 can display various types of wallpaper icons, such as two-dimensional wallpaper icons and three-dimensional wallpaper icons, and can also display a recommendation list. In response to a second click or second touch operation 12 by the user on the three-dimensional wallpaper icon 102, the user is redirected to the three-dimensional wallpaper module of the wallpaper application.

[0337] like Figure 22 As shown in C, this is the 3D wallpaper module interface 30 of the wallpaper application. In the 3D wallpaper module interface 30, in response to the user's request to generate a 3D wallpaper, for example... Figure 5 In C, the third click operation or third touch operation 13 of the generate 3D wallpaper button 103 enables the wallpaper application to access and display images stored in the local gallery of the electronic device.

[0338] like Figure 22 As shown in Figure D, this is the image display interface 40 of the electronic device's local image library. In the image display interface 40, in response to the user's fourth click or fourth touch operation 14 on the pre-selected wallpaper image 104, the wallpaper application performs a gridding process on the pre-selected wallpaper image, ultimately obtaining a three-dimensional wallpaper image. After the three-dimensional wallpaper image is generated, the wallpaper application can display a three-dimensional wallpaper preview interface.

[0339] like Figure 22 As shown in E, this is the 3D wallpaper preview interface 50. In the 3D wallpaper preview interface 50, in response to user touch or click operations on various buttons, tasks corresponding to those buttons are executed. For example, in response to the user's fifth click or fifth touch operation 15 on the "Set as Wallpaper" button 105, the 3D wallpaper image is set as the wallpaper of the electronic device. Furthermore, in response to click or touch operations on other buttons, the generated 3D wallpaper image can be previewed, or uploaded to a community, etc.

[0340] Understandable. Figure 22 This is only one implementation method of the embodiment of this application. The setting of three-dimensional wallpaper can also be achieved in other ways. For example, you can enter the program to generate three-dimensional wallpaper by clicking the wallpaper menu of the theme application; or you can enter the program to generate three-dimensional wallpaper by clicking the settings button and selecting desktop and wallpaper in the pop-up settings list.

[0341] Other embodiments of this application provide an apparatus for generating a gridded image.

[0342] Figure 23 This is a schematic diagram of a gridded image generation device provided in an embodiment of this application.

[0343] like Figure 23 As shown, the gridded image generation device may include: a display screen 1001, a memory 1002, a processor 1003, and a communication module 1004. These devices can be connected via one or more communication buses 1005. The display screen 1001 may include a display panel 10011 and a touch sensor 10012. The display panel 10011 displays images, and the touch sensor 10012 transmits detected touch operations to the application processor to determine the touch event type, providing visual output related to the touch operation through the display panel 10011. The processor 1003 may include one or more processing units, such as an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor. Different processing units may be independent devices or integrated into one or more processors. The memory 1002 is coupled to the processor 1003 and is used to store various software programs and / or computer instructions. The memory 1002 may include volatile memory and / or non-volatile memory. When the processor executes computer instructions, the gridded image generation apparatus can perform the various functions or steps performed in the above method embodiments.

[0344] When the software program and / or multiple sets of instructions in the memory 1002 are executed by the processor 1003, the meshed image generation device performs the following method steps: obtaining a first depth boundary corresponding to a first image, the first depth boundary being the boundary of a first depth image corresponding to the first image, the first depth boundary including at least one first pixel; setting at least one mesh structure on the first depth image, such that the first pixel is distributed in the mesh structure; when the first pixel and the mesh structure have a preset positional correspondence, determining at least one first activation point based on the mesh structure, the first activation point including at least one of the vertices and center points of each mesh in the mesh structure; constructing a target mesh structure for the first activation point, so as to generate a meshed image based on the target mesh structure.

[0345] This application also provides an electronic device, including: a processor, a memory, and a touch screen; the memory stores program instructions, which, when executed by the processor, cause the electronic device to perform the gridded image generation method or wallpaper generation method in any of the above embodiments.

[0346] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of an electronic device). Or, for example, the interface circuit can be used to send signals to other devices. Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0347] This application also provides a computer-readable storage medium, which includes computer instructions, and the computer instructions are used in the aforementioned electronic device (such as...). Figure 3 When the electronic device 100 shown is run, it causes the electronic device to perform the various functions or steps in the above method embodiments.

[0348] This application also provides a computer program product that, when run on a computer, causes the computer to perform the various functions or steps performed by the mobile phone in the above method embodiments.

[0349] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0350] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0351] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0352] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0353] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0354] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating a gridded image, characterized in that, The method includes: Obtain the first depth boundary corresponding to the first image, wherein the first depth boundary is the boundary of the first depth image corresponding to the first image, and the first depth boundary includes at least one first pixel. At least one grid structure is set on the first depth image so that the first pixel is distributed in the grid structure; When the first pixel has a preset positional correspondence with the mesh structure, at least one first activation point is determined based on the mesh structure. The first activation point includes at least one of the vertices and center points of each mesh in the mesh structure. A target mesh structure is constructed for the first activation point to generate a meshed image based on the target mesh structure; wherein, setting at least one mesh structure on the first depth image includes: Based on different resolutions, at least one layer is set on the first depth image so that each layer has a corresponding mesh structure, wherein each resolution corresponds to one layer, and in the same layer, each mesh in the mesh structure has the same size; The step of setting at least one layer on the first depth image based on different resolutions, so that each layer has a corresponding mesh structure, includes: The levels are numbered from largest to smallest according to the resolution; The side length of the grid corresponding to each level is determined according to the level number, so as to form the grid structure based on the side length; Wherein, the side length is the sum of 2 raised to the power of N and 1, and N is the level number corresponding to the side length.

2. The method according to claim 1, characterized in that, After setting at least one grid structure on the first depth image, the method further includes: Store the first coordinates corresponding to the center of the edge of the grid in the grid structure, and the second coordinates corresponding to the center point of the grid; Based on the first coordinate and the second coordinate, it is determined whether the first pixel has a positional correspondence with the grid structure, wherein the positional correspondence includes: the first pixel is located at the center point of the grid in the grid structure, and / or the first pixel is located at the center of any edge of the grid in the grid structure.

3. The method according to claim 1, characterized in that, The step of constructing a target mesh structure for the first activation point, and generating a meshed image based on the target mesh structure, includes: The first pixel is designated as the second activation point; An activation point set is formed based on the first activation point and the second activation point; The target mesh structure is constructed based on the set of activation points.

4. The method according to claim 3, characterized in that, When the first pixel has a preset positional correspondence with the grid structure, determining at least one first activation point based on the grid structure includes: According to the ascending order of the hierarchy numbers, traverse the grids in the grid structure; If the grid has the positional correspondence with the activation points in the activation point set, then the vertex or the center point of the grid is determined as the first activation point, and the first activation point is added to the activation point set.

5. The method according to claim 4, characterized in that, If the mesh has the positional correspondence with the activation points in the set of activation points, then determining the vertex or the center point of the mesh as the first activation point includes: If the center point of the grid in the grid structure coincides with any of the activation points in the set of activation points, then the vertex of the grid is determined to be the first activation point; and / or, If the center of any edge of the mesh in the mesh structure coincides with any of the activation points in the set of activation points, then the vertex of the mesh and the center point of the mesh are determined as the first activation point.

6. The method according to claim 3, characterized in that, Before constructing the target mesh structure based on the set of activation points, the method further includes: The first depth image is uniformly divided into multiple grids at preset intervals. The vertices of the evenly divided multiple grids are used as third activation points, and the third activation points are added to the set of activation points.

7. The method according to claim 3, characterized in that, The construction of the target mesh structure based on the set of activation points includes: The first depth image is divided into two triangles using the diagonal of the first depth image; Determine whether each triangle matches a preset condition, wherein the preset condition is: the midpoint of the base of the triangle is an activation point in the set of activation points, and the length of the base of the triangle is greater than 1. ; If the triangle matches the preset conditions, connect the midpoint of the base to the right-angle vertex of the triangle to divide the triangle into two new triangles, and return to the step of determining whether each triangle matches the preset conditions; If the triangle does not match the preset conditions, the vertex connection relationship of the mesh in the target mesh structure on the two-dimensional plane is determined according to the vertex connection relationship between the multiple triangles. Based on the first depth image, determine the information of the vertices of the mesh in the target mesh structure in the depth direction; Based on the information in the depth direction and the vertex connection relationships on the two-dimensional plane, the target mesh structure is constructed in three-dimensional space.

8. The method according to claim 7, characterized in that, Determining the vertex connection relationships of the mesh in the target mesh structure on the two-dimensional plane based on the vertex connection relationships between the multiple triangles includes: Traverse the binary tree constructed based on the triangle to obtain multiple leaf nodes on the binary tree. Each node on the binary tree corresponds to one triangle. The node information of each node includes at least one of the vertex coordinates, base midpoint coordinates, child node information, and flag information of the triangle corresponding to the node. The flag information is used to identify whether the node is a leaf node. The vertex connection relationship of the triangle corresponding to the leaf node is determined as the vertex connection relationship of the mesh in the target mesh structure on the two-dimensional plane.

9. The method according to claim 8, characterized in that, Before traversing the binary tree constructed based on the triangle, the method further includes: The binary tree is constructed using the triangle obtained from the first depth image as the root node; After dividing the triangle into two new triangles, two child nodes are constructed on the nodes corresponding to the triangles, and the node information of the nodes corresponding to the triangles and the node information of the child nodes are updated, wherein the two child nodes are the nodes corresponding to the two new triangles respectively.

10. The method according to claim 3, characterized in that, The step of obtaining the first depth boundary corresponding to the first image includes: A depth estimation algorithm is performed on the first image to obtain a first depth image corresponding to the first image. The depth estimation algorithm is performed using a monocular depth estimation network, which includes at least one of MiDaS network, DORN network, Monodepth network, and DepthGAN network. Obtain the boundary of the first depth image, and use it as the first depth boundary corresponding to the first image.

11. The method according to claim 10, characterized in that, The step of obtaining the boundary of the first depth image includes: The gradient of each pixel in the first depth image is calculated using the Sobel operator; An exponential function is constructed using the gradient and a preset adjustment coefficient; The first depth image is obtained by processing each pixel in the first depth image using the exponential function. The boundary is obtained from the processed first depth image.

12. The method according to claim 11, characterized in that, After obtaining the processed first depth image, the method further includes: For the grayscale value of each pixel in the processed first depth image, the grayscale values ​​that are greater than or equal to a preset threshold are adjusted to 1, and the grayscale values ​​that are less than the preset threshold are adjusted to 0, to obtain the padded first depth image; In the first depth image after padding, the pixel with a grayscale value of 1 is identified as the first pixel.

13. The method according to claim 1, characterized in that, After constructing the target mesh structure for the first activation point, the method further includes: Using the color information of the first image, the target mesh structure is colored to obtain the colored meshed image.

14. A method for generating wallpaper, characterized in that, The method includes: In response to a wallpaper selection instruction, a pre-selected wallpaper image is determined from at least one image in the gallery, the pre-selected wallpaper image being a two-dimensional image; The pre-selected wallpaper image is processed using the method described in any one of claims 1-13 to obtain the target network structure corresponding to the pre-selected wallpaper image; The target network structure corresponding to the pre-selected wallpaper image is rendered in 3D using a renderer to obtain a colored mesh image, which is then used as a 3D wallpaper image.

15. An electronic device, characterized in that, It includes a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, the computer program code including computer instructions, which, when the processor executes the computer instructions, cause the electronic device to perform the method for generating a gridded image as described in any one of claims 1-13 or the method for generating a wallpaper as described in claim 14.

Images