Gridding picture generation method, wallpaper generation method and electronic equipment

Abstract

The invention relates to the technical field of terminals, and provides a grid picture generation method, a wallpaper generation method and electronic equipment. The grid picture generation method comprises the steps that a first depth boundary corresponding to a first image is acquired, the first depth boundary is the boundary of a first depth image corresponding to the first image, and the first depth boundary comprises at least one first pixel point; at least one grid structure is arranged on the first depth image, so that the first pixel points are distributed on the grid structure; under the condition that the first pixel point and the grid structure have a preset position corresponding relation, at least one first activation point is determined based on the grid structure, and the first activation point comprises at least one of the vertex and the center point of each grid in the grid structure; and constructing a target grid structure for the first activation point to generate a gridded picture based on the target grid structure. According to the technical scheme, the problem of large calculation amount caused by dense grids in an existing method can be solved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method for generating a gridded image, a method for generating a wallpaper, and an electronic device. Background Art

[0002] With the advancement of electronic technology, mobile devices are increasingly using 3D rendering to enhance realism and interactivity. The 3D rendering process for a 2D image typically requires constructing a 3D mesh from the 2D image to create a corresponding meshed image. Then, based on the meshed image, lighting models and rendering techniques are applied to give the meshed image a realistic visual effect, resulting in the final 3D image. Therefore, meshing the 2D image is a crucial step in the 3D image generation process.

[0003] Existing image meshing methods typically construct a dense mesh structure directly using each pixel in the image as a vertex. The number of mesh facets constructed by this method is often in the millions, which puts tremendous pressure on the storage and rendering of mobile terminals. While mesh simplification algorithms can simplify dense mesh structures and reduce the number of mesh facets, the simplification process of dense network structures by mesh simplification algorithms also requires a large amount of computing resources and is time-consuming. Therefore, existing image meshing methods are computationally intensive and require high device resources, making them particularly unsuitable for mobile terminals with limited computing resources. Summary of the Invention

[0004] The present application provides a grid image generation method, a wallpaper generation method, and an electronic device to solve the problem of large computational complexity in existing grid image generation methods.

[0005] To achieve the above objectives, in a first aspect, an embodiment of the present application provides a method for generating a gridded image, the method comprising:

[0006] Obtain a first depth boundary corresponding to the first image, where the first depth boundary is a boundary of the first depth image corresponding to the first image, and the first depth boundary includes at least one first pixel point; set at least one grid structure on the first depth image so that the first pixel points are distributed in the grid structure; when the first pixel point and the grid structure have a preset position correspondence, determine at least one first activation point based on the grid structure, and the first activation point includes at least one of a vertex and a center point of the grid structure; construct a target grid based on the first activation point to generate a gridded image.

[0007] The gridded image generation method shown in the embodiment of the present application uses the positional relationship between the pixel points of the image depth boundary and the grid structure to determine the activation point. The activation point is then used to determine the grid vertices in the target grid structure to construct the target grid structure. Through such a design, the present application does not need to use each pixel point as a vertex to construct a dense grid structure, and there is no need to simplify the dense grid structure. Therefore, the method proposed in the present application has a small amount of computation, consumes few computing resources, and is suitable for mobile terminals.

[0008] In one implementation, at least one grid structure is provided on the first depth image, including: providing at least one level on the first depth image based on different resolutions, such that each level has a corresponding grid structure, wherein each resolution corresponds to a level, and within the same level, each grid in the grid structure has the same size. This implementation, by providing multiple grid structures based on different resolutions, allows analysis of the first depth image based on different resolutions, balancing computational efficiency and image detail.

[0009] In one implementation, at least one level is set on the first depth image based on different resolutions so that each level has a corresponding grid structure, including: numbering the levels from large to small according to the resolution; determining the side length of the grid corresponding to each level according to the level number, so as to form a grid structure based on the side length; wherein the side length is the sum of 2 to the Nth power and 1, and N is the level number corresponding to the side length. Using this implementation, the levels are numbered according to the resolution size, and then the side length of the grid is determined. In this way, the grid side lengths corresponding to different resolutions are different. The higher the resolution, the smaller the grid side length and the denser the grid structure. Therefore, the layer-by-layer refinement of the grid structure can be achieved.

[0010] In one implementation, after setting at least one grid structure on the first depth image, the method further includes: storing a first coordinate corresponding to the center of an edge of a grid in the grid structure, and a second coordinate corresponding to the center point of the grid; and determining whether a first pixel point has a preset positional correspondence with the grid structure based on the first coordinate and the second coordinate, wherein the positional correspondence includes: the first pixel point is located at the center point of the grid in the grid structure, and / or the first pixel point is located at the center of any edge of the grid in the grid structure. Using this implementation, the coordinates of the center point of the grid and the center point of its edges can be pre-calculated and saved. In this way, when determining whether a first pixel point has a positional correspondence with the grid structure, it is only necessary to extract the pre-selected and saved coordinates and traverse them, thereby increasing the speed of determining the positional correspondence. In addition, this implementation pre-sets the positional correspondence as the first pixel point being at the center of the grid or the center of an edge. Due to the special nature of the position of the center, this can reduce computational complexity and facilitate the bisection of the triangle based on the center in subsequent steps.

[0011] In one implementation, a target grid structure is constructed for the first activation point to generate a gridded image based on the target grid structure, including: determining the first pixel point as the second activation point; forming an activation point set based on the first activation point and the second activation point; and constructing the target grid structure based on the activation point set. In this implementation, an activation point set is constructed based on the first pixel point and the first activation point. Since the first pixel point is a point on the depth boundary, and the first activation point is determined based on the first pixel point, this implementation utilizes the points on the depth boundary and the activation points diffused therefrom to construct the target grid structure, which can focus the construction on the area corresponding to the depth boundary while avoiding missing other areas.

[0012] In one implementation, when the first pixel point has a preset positional correspondence with the grid structure, at least one first activation point is determined based on the grid structure, including: traversing the grids in the grid structure in ascending order of the hierarchical numbers; if the grid has a positional correspondence with the activation points in the activation point set, determining the vertex or center point in the grid as the first activation point, and adding the first activation point to the activation point set. Using this implementation, the positional correspondence between the first pixel point and the grid structure is determined, and the activation point set is obtained by precalculating the positional relationship. When performing subsequent operations such as triangulation, only the activation point set needs to be queried, which is faster and more suitable for running on the end side.

[0013] In one implementation, if the grid has a positional correspondence with an activation point in the activation point set, the vertex or center point in the grid is determined as the first activation point, including: if the center point of the grid in the grid structure coincides with any activation point in the activation point set, the vertex of the grid is determined to be the first activation point; and / or, if the center of any edge of the grid in the grid structure coincides with any activation point in the activation point set, the vertex of the grid and the center point of the grid are determined to be the first activation point. Using this implementation, the first activation point is determined for the grid area that has a positional correspondence with the activation point in the activation point set (i.e., the center of the grid or the center of the grid edge coincides with the activation point), thereby increasing the number of activation points in the area. Therefore, the grid density of the area can be improved, that is, the gridding accuracy of the area can be improved.

[0014] In one implementation, before constructing the target mesh structure based on the activation point set, the method further includes: evenly dividing the first depth image into multiple meshes at preset intervals; using vertices of the evenly divided meshes as third activation points, and adding the third activation points to the activation point set. Using this implementation, a number of third activation points are evenly set throughout the first depth image, ensuring that non-depth boundary areas are also covered by the activation points, preventing overly sparse meshes in non-depth boundary areas and improving meshing accuracy.

[0015] In one implementation, constructing a target grid structure based on the activation point set includes: using the diagonal of the first depth image to divide the first depth image into two triangles; determining whether each triangle matches a preset condition, where the preset condition is: the midpoint of the base of the triangle is an activation point in the activation point set, and the length of the base of the triangle is greater than When the triangle matches the preset conditions, connect the midpoint of the base and the right-angled vertex of the triangle to divide the triangle into two new triangles, and return to the step of determining whether each triangle matches the preset conditions; when the triangle does not match the preset conditions, determine the vertex connection relationship of the mesh in the target mesh structure on the two-dimensional plane based on the vertex connection relationship between multiple triangles; determine the information of the vertices of the mesh in the target mesh structure in the depth direction based on the first depth image; construct the target mesh structure in three-dimensional space based on the information in the depth direction and the vertex connection relationship on the two-dimensional plane. Using this implementation method, mesh division is performed by bisection of triangles to obtain the vertex connection relationship of the target network structure on the two-dimensional plane. At the same time, combined with the depth image, the vertex position and connection relationship in the three-dimensional space are obtained. In the target network structure constructed in this way, the facets on adjacent meshes share edges, and there will be no crack problem, which avoids the appearance of holes during rendering.

[0016] In one implementation, the vertex connection relationship of the mesh in the target mesh structure on a two-dimensional plane is determined based on the vertex connection relationship between multiple triangles, including: traversing a binary tree constructed based on triangles to obtain multiple leaf nodes on the binary tree, wherein each node on the binary tree corresponds to a triangle, and the node information of each node includes at least one of the vertex coordinates, base midpoint coordinates, child node information, and flag information of the triangle corresponding to the node, and the flag information is used to identify whether the node is a leaf node; and determining that the vertex connection relationship of the triangle corresponding to the leaf node is the vertex connection relationship of the mesh in the target mesh structure on a two-dimensional plane. With this implementation, a binary tree is used to store the node information of the triangle and participate in the calculation. Since the binary tree search is highly efficient, it can effectively improve the computational efficiency of the gridded image generation algorithm.

[0017] In one implementation, before traversing the binary tree constructed based on the triangles, the method further includes: constructing a binary tree with the triangle obtained by partitioning the first depth image as the root node; after splitting the triangle into two new triangles, constructing two child nodes on the node corresponding to the triangle, and updating the node information of the node and child nodes corresponding to the triangle, wherein the two child nodes are the nodes corresponding to the two new triangles. This implementation method uses each node of the binary tree to represent a triangle, and uses the hierarchical relationship of the binary tree to represent the containment relationship between triangles. The structure is simple and easy to understand and implement.

[0018] In one implementation, obtaining a first depth boundary corresponding to a first image includes: executing a depth estimation algorithm on the first image to obtain a first depth image corresponding to the first image, wherein the depth estimation algorithm is executed using a monocular depth estimation network, and the monocular depth estimation network includes at least one of a MiDaS network, a DORN network, a Monodepth network, and a DepthGAN network; and obtaining a boundary of the first depth image as the first depth boundary corresponding to the first image. Using this implementation, depth estimation is performed on the first image using a monocular depth estimation network to obtain its corresponding depth image. Then, in a subsequent step, the boundary of the first image is obtained based on the depth corresponding to the pixels in the image.

[0019] In one implementation, obtaining the boundaries of the first depth image includes: calculating the gradient of each pixel in the first depth image using a Sobel operator; constructing an exponential function using the gradient and a preset adjustment coefficient; processing each pixel in the first depth image using the exponential function to obtain a processed first depth image; and obtaining the boundaries in the processed first depth image. In this implementation, edge detection is performed on the first depth image using a Sobel operator to extract depth boundaries in the image. In subsequent steps, targeted grid construction methods are employed for boundary and non-boundary locations.

[0020] In one implementation, after obtaining the processed first depth image, the method further includes: for the grayscale value of each pixel in the processed first depth image, adjusting the grayscale value greater than or equal to a preset threshold to 1, and adjusting the grayscale value less than the preset threshold to 0, to obtain the padded first depth image; in the padded first depth image, determining the pixel with a grayscale value of 1 as the first pixel. Using this implementation, based on the relationship between the preset threshold and the grayscale value of the pixel, the grayscale value of the pixel with a weaker boundary is set to 0, and it is considered that it is not on the boundary, so as to achieve the padding of the first depth image. In this way, a clearer boundary can be obtained. In addition, the refinement based on the depth boundary conforms to the laws of physics, thereby improving the accuracy of boundary recognition and ensuring the rendering quality.

[0021] In one implementation, after constructing the target grid structure based on the first activation point, the method further includes: coloring the target grid structure using the color information of the first image to obtain a colored gridded image. This implementation colorizes each grid in the target grid structure, assigning corresponding colors and textures to the grid faces, thereby converting the black and white gridded image into color.

[0022] In a second aspect, the present application provides a wallpaper generation method, the method comprising: in response to a wallpaper selection instruction, determining a preselected wallpaper image from at least one image in a gallery, the preselected wallpaper image being a two-dimensional image; processing the preselected wallpaper image using the gridded image generation method as in the first aspect and any implementation method mentioned above to obtain a target network structure corresponding to the preselected wallpaper image; using a renderer to perform 3D rendering on the target network structure corresponding to the preselected wallpaper image to obtain a colored gridded image, and using the colored gridded image as a three-dimensional wallpaper image.

[0023] The wallpaper generation method illustrated in this embodiment utilizes the aforementioned gridded image generation method to generate a corresponding gridded image based on a two-dimensional image. The gridded image is then rendered to produce a three-dimensional wallpaper image with a more realistic 3D effect. Compared to traditional two-dimensional wallpapers, this embodiment can enhance the user's visual experience.

[0024] In a third aspect, the present application also provides an electronic device comprising a memory and a processor; the memory and the processor are coupled; wherein the memory is used to store computer program code, and the computer program code comprises computer instructions, and when the processor executes the computer instructions, the electronic device executes the method for generating a gridded image as in the first aspect and any implementation method mentioned above, or the method for generating wallpaper as in the second aspect mentioned above.

[0025] In a fourth aspect, the present application also provides a chip system, which includes a processor; the processor is coupled to a memory, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the method for generating a gridded image as in the first aspect and any implementation method above or the method for generating wallpaper as in the second aspect above is executed.

[0026] In the fifth aspect, the present application also provides a computer-readable storage medium, which stores a computer program or instruction. When the computer program or instruction is run on a computer, the computer executes the method for generating a gridded image as in the first aspect and any implementation method mentioned above, or the wallpaper generation method as in the second aspect mentioned above.

[0027] In the sixth aspect, the present application also provides a computer program product, which includes: a computer program or instructions, which, when the computer program or instructions are run on a computer, enables the computer to execute the method for generating a gridded image as in the first aspect and any implementation method mentioned above, or the method for generating wallpaper as in the second aspect mentioned above.

[0028] It can be understood that the beneficial effects that can be achieved by the technical solutions provided in the third to sixth aspects mentioned above can be referred to the beneficial effects in the first aspect and any optional implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0030] Figure 1 It is a three-dimensional wallpaper image displayed at different viewing angles;

[0031] Figure 2 It is a schematic diagram of the effect of constructing a triangular mesh structure based on a quadtree triangulation method;

[0032] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0033] Figure 4 is a schematic diagram of the software structure of the electronic device provided in an embodiment of the present application;

[0034] Figure 5 This is the first flow chart of the method for generating a gridded image provided in an embodiment of the present application;

[0035] Figure 6 is a schematic diagram of a depth image before and after processing provided by an embodiment of the present application;

[0036] Figure 7 This is a schematic diagram of a first pixel point distributed in a grid structure provided by an embodiment of the present application;

[0037] Figure 8 This is a schematic diagram of a hierarchical setting provided by an embodiment of the present application;

[0038] Figure 9 This is a schematic diagram of grid structure division at each level provided in an embodiment of the present application;

[0039] Figure 10 is a schematic diagram of a first coordinate and a second coordinate provided in an embodiment of the present application;

[0040] Figure 11 This is a second flow chart of the method for generating a gridded image provided in an embodiment of the present application;

[0041] Figure 12 This is a schematic diagram of an activation strategy provided in an embodiment of the present application;

[0042] Figure 13 is a schematic diagram of a uniformly divided grid provided in an embodiment of the present application;

[0043] Figure 14 This is the third flow chart of the method for generating a gridded image provided in an embodiment of the present application;

[0044] Figure 15 is a schematic diagram of a triangulation process provided by an embodiment of the present application;

[0045] Figure 16 This is a schematic diagram of an activation point and a corresponding vertex connection relationship diagram provided in an embodiment of the present application;

[0046] Figure 17 This is the fourth flow chart of the method for generating a gridded image provided in an embodiment of the present application;

[0047] Figure 18 The embodiment of this application provides Figure 15 Schematic diagram of the binary tree constructed by triangulation process shown;

[0048] Figure 19 This is a flow chart of the wallpaper generation method provided in an embodiment of the present application;

[0049] Figure 20 This is a schematic diagram of processing a preselected wallpaper image using the aforementioned image gridding method provided by an embodiment of the present application;

[0050] Figure 21 This embodiment of the present application provides Figure 20 A magnified image of the target grid structure;

[0051] Figure 22 This is a schematic diagram of a user operation interface for a three-dimensional wallpaper setting scene provided by an embodiment of the present application;

[0052] Figure 23 This is a schematic diagram of a grid image generation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will clearly describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments of the present application, other embodiments obtained by ordinary technicians in this field without making any creative work are all within the scope of protection of this application.

[0054] Hereinafter, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified with "first," "second," etc., may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0055] In addition, in this application, directional terms such as "upper", "lower", "inner" and "outer" are defined relative to the orientation of the components in the drawings. It should be understood that these directional terms are relative concepts. They are used for relative description and clarification, and they can change accordingly according to changes in the orientation of the components in the drawings.

[0056] The following explains the professional terms mentioned in the embodiments of the present application to facilitate understanding by those skilled in the art.

[0057] A triangular mesh structure, used in computer graphics and 3D modeling, is a widely used data structure in fields such as computer graphics, computer-aided design (CAD), and geographic information systems (GIS). A triangular mesh structure consists of a series of interconnected triangular meshes that collectively cover a two-dimensional area or three-dimensional surface. In embodiments of the present application, the triangular mesh structure can be used to generate 3D wallpaper.

[0058] Facets, used in the fields of computer graphics and 3D modeling, are one of the basic units that make up a 3D mesh structure. Specifically, a facet is a planar polygon connected by three or more vertices, used to represent the surface area of ​​a 3D object. For example, a facet can be laid on each triangular mesh in a triangular mesh structure to construct a complex 3D mesh and model. In an embodiment of the present application, a facet is laid on the mesh of the constructed target network structure to obtain a gridded image.

[0059] The following first describes the application scenarios of the embodiments of the present application with reference to the accompanying drawings.

[0060] With the development of electronic technology, electronic devices can use 3D rendering technology to enhance realism and interactivity. For example, 3D rendering technology can be used to generate three-dimensional wallpaper to achieve personalized screen display. Three-dimensional wallpaper includes three dimensions: length, height, and depth. Compared with traditional two-dimensional wallpaper, three-dimensional wallpaper adds the dimension of depth. In this way, different images can be displayed based on different viewing angles, allowing viewers to perceive the three-dimensional position and shape of objects in the image. The increase in the depth dimension makes the wallpaper image more vivid and realistic.

[0061] Figure 1 It is a three-dimensional wallpaper image displayed at different viewing angles.

[0062] like Figure 1 As shown in FIG, the three-dimensional wallpaper image shows an image of a dog. Figure 1 As shown in A, it is the display effect when the user is watching from the front. Figure 1As shown in B in FIG, the display effect of the user viewing from a certain viewing angle on the right side is shown. As can be seen, as the viewing angle changes, the display effect of the three-dimensional wallpaper image also changes accordingly.

[0063] 3D rendering of a 2D image can produce the aforementioned 3D wallpaper image. Specifically, the 3D rendering process typically involves meshing the 2D image. Then, based on the resulting 3D meshed image, lighting models and rendering techniques are applied to create a realistic visual effect, resulting in the final 3D image.

[0064] Among them, meshing processing can use triangular mesh structure, quadrilateral mesh structure, etc. to explicitly represent three-dimensional geometry, and triangular mesh structure is the most widely used one. The triangular mesh structure is composed of a series of interconnected triangular meshes, and triangular planes (hereinafter referred to as triangular mesh patches) are laid on the triangular meshes. A three-dimensional model can be spliced ​​together through these triangular mesh patches. For example, a cube can be composed of 6 faces, and the square of each face is composed of 2 triangular mesh patches, that is, 12 triangular mesh patches are spliced ​​into a cube. It can be understood that a complex three-dimensional picture may be composed of thousands of triangular mesh patches, each of which is part of its surface.

[0065] However, the image meshing method based on triangular mesh has the following problems.

[0066] Currently, mesh structures are typically constructed directly using each pixel in an image as a vertex for each mesh. For example, for a 1000×1000 pixel image, all pixels are directly used as mesh vertices. Using these 1000×1000 vertices, a mesh structure consisting of 999×999×2 triangular meshes can be constructed. As can be seen, this method of using each pixel as a mesh vertex results in a very dense mesh, and the number of triangular mesh facets required to lay out the mesh is also large. Therefore, a large amount of storage space is required to store information about mesh vertices, edges, and faces (i.e., triangular mesh facets). Furthermore, during the rendering process, performing calculations on a large number of triangles consumes a significant amount of computing resources. This is especially true for mobile devices, which are designed with portability and energy consumption in mind, and whose storage space and computing performance are typically lower than those of computers. Therefore, mobile devices are even more unable to handle the large number of triangular mesh facets, which can cause them to lag or even crash.

[0067] Taking the above problems into consideration, a mesh simplification algorithm can be used to simplify the above dense mesh structure. The mesh simplification algorithm can reduce the number of vertices, edges, and faces in a complex mesh structure. Commonly used mesh simplification algorithms are as follows: vertex extraction algorithm, which can delete vertices and their adjacent faces in the mesh structure, and then re-triangulate the holes caused by the deletion operation; vertex clustering algorithm, which can cluster multiple vertices into one, and then update the mesh faces based on the clustered vertices; edge contraction algorithm, which can contract an edge into a point and delete the face corresponding to this edge. Through the above mesh simplification algorithms, the complexity of the mesh can be reduced and the number of triangular mesh facets can be reduced. However, the simplification process of dense mesh structures also requires a lot of computing resources.

[0068] Some solutions also use quadtree-based triangulation to construct triangular mesh structures. However, in this method, both active and inactive nodes are used to construct the triangular mesh structure, and gaps exist between the faces corresponding to active and inactive nodes. This means that the resulting mesh structure suffers from the problem of adjacent faces not sharing common edges. These gaps between adjacent faces can lead to holes in the resulting meshed image.

[0069] Figure 2 It is a schematic diagram of the effect of constructing a triangular mesh structure based on a quadtree triangulation method.

[0070] Figure 2 A is a schematic diagram of a plane effect of constructing a triangular mesh structure based on a quadtree triangulation method. Figure 2 B is a schematic diagram of a three-dimensional effect of constructing a triangular mesh structure based on a quadtree triangulation method. Figure 2 The white triangles in the middle are triangular mesh patches, and the shaded areas are the cracks between the patches. Figure 2 From A, we can see that adjacent triangle meshes fit together on the two-dimensional plane. Figure 2 As can be seen from B, there are gaps between adjacent triangle meshes in 3D space. Therefore, the quadtree-based triangulation method for constructing triangle mesh structures requires the use of additional patches to fill the gaps and avoid holes in the rendered image.

[0071] In order to solve the above problems, an embodiment of the present application provides a method for generating a gridded image.

[0072] The method for generating a gridded image provided in an embodiment of the present application first determines the activation point by utilizing the positional relationship between the pixel points on the depth boundary of the image and the pre-set grid structure. Then, the grid vertices are determined in the activation points to construct a target grid structure containing multiple grids. Since the embodiment of the present application does not use each pixel point as a grid vertex to construct a grid structure, the number of grids in the constructed grid structure is relatively small, and there is no need to simplify the grid structure. Therefore, the method proposed in the present application consumes less computing resources and is more suitable for mobile terminals with limited storage space and computing performance. In addition, since the grid vertices in the present application are all activation points, their resolutions are the same, and there is no problem of adjacent facets not sharing the same edges, there is no need to additionally repair the cracks between adjacent facets.

[0073] The triangular mesh construction method provided in the embodiments of the present application can be applied to electronic devices. In some embodiments, the electronic device can be a mobile phone, tablet computer, handheld computer, personal computer (PC), ultra-mobile personal computer (UMPC), netbook, as well as a mobile terminal such as a cellular phone, personal digital assistant (PDA), augmented reality (AR) device, virtual reality (VR) device, artificial intelligence (AI) device, wearable device, and vehicle-mounted device. The embodiments of the present application do not impose any particular restrictions on the specific type of the electronic device.

[0074] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0075] like Figure 3 As shown, the electronic device 100 may include a processor 110, a memory 120, an antenna 1, an antenna 2, a mobile communication module 130, a wireless communication module 140, a sensor module 150, a display screen 160, etc. Among them, the sensor module 150 may include a pressure sensor 150A, a touch sensor 150B, a fingerprint sensor 150C, an ambient light sensor 150D, a temperature sensor 150E, a gyroscope sensor 150F, a proximity light sensor 150G, etc. In the embodiment of the present application, a wallpaper selection instruction from the user can be received through the pressure sensor 150A and the touch sensor 150B.

[0076] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0077] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0078] The memory 120 can be used to store computer executable program code, and the executable program code includes instructions. The memory 120 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the foldable electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the foldable electronic device 100 by running instructions stored in the memory 120 and / or instructions stored in a memory provided in the processor.

[0079] In the embodiment of the present application, the code for implementing the gridded image generation method and wallpaper generation method described in the embodiment of the present application can be stored in a non-volatile memory. When setting a three-dimensional wallpaper, the electronic device 100 can load the executable code stored in the non-volatile memory into the random access memory.

[0080] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 130, the wireless communication module 14, the modem and the baseband processor.

[0081] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

[0082] The mobile communication module 130 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 130 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 130 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 130 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 130 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 130 can be set in the same device as at least some of the modules of the processor 110.

[0083] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs the sound signal through the audio device or displays an image or video through the display screen 160. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 130 or other functional modules.

[0084] The wireless communication module 140 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 140 can be one or more devices that integrate at least one communication processing module. The wireless communication module 140 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 140 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0085] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 130 , and antenna 2 is coupled to wireless communication module 140 , so that electronic device 100 can communicate with a network and other devices via wireless communication technology.

[0086] Electronic device 100 implements display functionality through a GPU, display screen 160, and an application processor. A GPU is a microprocessor for image processing that connects display screen 160 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0087] In the embodiment of the present application, the electronic device 100 implements the triangular mesh construction method and wallpaper generation method provided in the embodiment of the present application, mainly relying on the image calculation and processing capabilities provided by the GPU.

[0088] Display screen 160 is used to display images, videos, and the like. Display screen 160 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 160, where N is a positive integer greater than one.

[0089] In an embodiment of the present application, the electronic device 100 displays a gallery to allow the user to select preset wallpaper images in the gallery, and the ability to display three-dimensional wallpaper images depends on the display function provided by the above-mentioned GPU, display screen 160, and application processor.

[0090] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only for illustrative purposes and does not constitute a structural limitation on the electronic device. In other embodiments of the present application, the electronic device may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0091] Generally speaking, the implementation of the functions of the electronic device 100 requires not only hardware support but also software cooperation. The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100.

[0092] Figure 4 It is a schematic diagram of the software structure of the electronic device provided in the embodiment of the present application.

[0093] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0094] The application layer can include a series of application packages.

[0095] like Figure 4 As shown, the application package may include applications such as camera, calendar, map, WLAN, music, short message, gallery, call, navigation, video, wallpaper, etc.

[0096] The gallery application is an application for managing and browsing pictures and videos. The gallery can select and display pictures stored in the electronic device in response to user instructions, so that the user can browse and select any picture.

[0097] A wallpaper application is an application that provides wallpapers. The wallpaper application provides a download path for wallpaper resources, allowing users to download the corresponding wallpaper image through this path and directly set it as the background of their electronic device. Furthermore, the wallpaper application can also provide the function of setting an image stored in the electronic device as wallpaper. In an embodiment of the present application, the wallpaper application can process a two-dimensional image selected by the user through the gallery to obtain a three-dimensional wallpaper image corresponding to the two-dimensional image.

[0098] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0099] like Figure 4 As shown, the application framework layer may include a window manager, a content provider, a phone manager, a resource manager, a notification manager, a view system, a wallpaper management service WallpaperManagerService, and the like.

[0100] The content provider is used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0101] The wallpaper management service is used to provide wallpaper resources and download paths for wallpaper resources. It is also used for wallpaper management and settings, etc. It can include wallpaper library, personalized recommendations, download and settings, community and user upload functions.

[0102] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0103] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0104] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0105] The system library can include multiple functional modules, such as the surface manager, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), media libraries, etc.

[0106] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0107] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0108] A 2D graphics engine is a drawing engine for 2D drawings.

[0109] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0110] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0111] The following describes the workflow of the software and hardware of the electronic device 100 in combination with the three-dimensional wallpaper setting scene.

[0112] First, the wallpaper application calls the wallpaper management service of the application framework layer and starts the wallpaper application through the wallpaper management service. Then, the wallpaper application calls the content provider of the application framework layer and accesses the images stored in the local image library of the electronic device through the content provider. After the user selects an image from the local image library, the wallpaper application calls the 3D graphics processing library in the system library and performs the gridding operation provided by the embodiments of this application on the selected image, ultimately obtaining a 3D wallpaper image.

[0113] Figure 5 This is the first flowchart of the method for generating a gridded image provided in an embodiment of the present application.

[0114] like Figure 5 As shown, the method includes the following steps S11-S18.

[0115] Step S11: Acquire a first depth image corresponding to the first image.

[0116] In this step, the two-dimensional image to be gridded is used as the first image, and its corresponding depth image, that is, the first depth image, is obtained. Among them, the first image is a two-dimensional image, which can be a color image, a grayscale image, or a black and white image. The depth image is a grayscale image, and the grayscale value of each pixel in the depth image indicates the distance information (that is, depth information) between the point and the camera. For example, the grayscale value of a pixel without depth information is 0. Through the first depth image, the distance from each point in the first image to the camera can be known.

[0117] In one implementation, step S11 may include the following steps S111 - S112 .

[0118] Step S111: Acquire a first image.

[0119] In step S111, a first image is first acquired, so that the first image can be gridded in subsequent steps. The method for acquiring the first image can be selected by the user based on the actual application scenario and is not limited in the present embodiment. For example, the first image can be acquired by taking a photo, or by selecting the first image from images stored in the map library, or by downloading the first image via the internet or a local area network.

[0120] Step S112: executing a depth estimation algorithm on the first image to obtain a first depth image corresponding to the first image.

[0121] In step S112, a first depth image is acquired using a depth estimation algorithm. Specifically, the depth estimation algorithm can infer the distance between each pixel in a single two-dimensional image and the camera head, thereby obtaining information about the image in the depth direction. Considering that two-dimensional images are typically captured by a monocular camera, a monocular depth estimation network can be used to perform depth estimation for the two-dimensional target image in the embodiment of the present application.

[0122] Exemplarily, a mixed-scale dense depth (MiDaS) network can be used to execute a depth estimation algorithm to achieve depth estimation of a single image. The MiDaS network is a monocular depth estimation model based on a neural network, which can use a pre-trained neural network to process the input color image and output a corresponding depth map. In an embodiment of the present application, a two-dimensional first image is input into the MiDaS network to obtain a depth image corresponding to the first image, that is, a first depth image. In addition to the MiDaS network, other monocular depth estimation networks can also be used to execute depth estimation algorithms, such as: DORN network, Monodepth network, DepthGAN network, etc.

[0123] It should be noted that steps S111-S112 only provide a method for obtaining a first depth image based on a monocular depth estimation network. In some implementations, other methods may be used to obtain the first depth image. For example, a pre-trained conditional generative adversarial network (such as Pix2Pix, CycleGAN) may be used to generate a first depth image using the mapping from image to depth map learned during training. It will be understood that the above is only an exemplary description of the method for obtaining the first depth image. In actual applications, other methods for obtaining the first depth image may also be used, and this application does not limit this.

[0124] Step S12: Acquire a boundary of the first depth image as a first depth boundary corresponding to the first image, where the first depth boundary includes at least one first pixel point.

[0125] In step S12, it can be understood that during the capture of the two-dimensional image, different positions on the three-dimensional object being captured are at different actual distances from the lens, especially at the junctions between objects, the contours of objects, and locations where the texture changes. This is reflected in the depth image as different depths of different pixels in the depth image, especially at the junctions, contours, and locations where the texture changes. There may be significant depth changes. When rendering a two-dimensional image into a three-dimensional image, the greater the depth difference (that is, the more significant the depth change), the more likely discontinuities or unnatural phenomena will occur. For example, at locations such as contours and junctions where there are significant depth changes, defects such as stretching artifacts may occur.

[0126] Taking into account that during the rendering process, the greater the depth difference in the two-dimensional image, the more likely it is to have defects. In order to reduce the number of grids while ensuring the rendering effect, the embodiment of the present application performs targeted gridding on the positions where defects are prone to occur. Specifically, for positions with larger depth differences, denser grids are set. More facets can be set in a dense grid, so defects can be reduced. For positions without depth differences, relatively sparse grids are set to reduce the number of facets in these positions to reduce the amount of computation. The embodiment of the present application determines the density of the grid based on the depth difference, which can achieve a balance between rendering effect and computational efficiency.

[0127] Based on this, the embodiment of the present application obtains the location where the depth difference exists and sets a dense grid at that location. The location where the depth difference exists in the first image is the location of the boundary of the first depth image (i.e., the first depth boundary), such as the location of the outline of the object in the aforementioned analysis.

[0128] In specific application scenarios, vision and image processing techniques can be used to acquire depth boundaries, and the method for acquiring depth boundaries is not specifically limited in the embodiments of the present application. For example, for the first image in the embodiments of the present application, a deep learning method can be used, and a deep learning model such as a convolutional neural network can be used to predict the depth of each pixel in the first image, thereby obtaining a depth boundary with a depth difference. For example, a specific deep learning model can also be used to process the first image to obtain a corresponding depth image, and then the depth boundary can be extracted from the depth image. For example, the depth of each pixel in the first image can be inferred by using disparity information from images under two or more perspectives to determine the depth boundary based on the depth of each pixel. For example, multiple images of the same scene can be taken under different lighting conditions, and the depth of each pixel in the first image can be inferred by analyzing the changes in reflected light.

[0129] In one implementation, step S12 may include the following steps S121 - S126 .

[0130] Step S121: Calculate the gradient of each pixel in the first depth image using the Sobel operator.

[0131] In step S121, edge detection is performed on the depth image obtained in the previous step based on the gradient of each pixel in the first depth image to obtain the depth boundary position where defects are likely to appear. It can be understood that since the depth boundary position has a depth difference, that is, there is a change in the depth value, and the significance of the value change can usually be represented by a gradient, the boundary of the depth image can be obtained based on the gradient value of each pixel in the depth image. Specifically, a gradient-based edge detection algorithm such as the Sobel algorithm, the Prewitt algorithm, and the Canny algorithm can be used.

[0132] For example, if the Sobel algorithm is used, the Sobel operator is first used to calculate the gradient value corresponding to each pixel in the first depth image, that is, the gradient of the grayscale value of each pixel. The Sobel operator is an operator applied to the Sobel edge detection algorithm, which can filter the depth image using a convolution operation to obtain the gradient information of the grayscale value of each pixel in the depth image.

[0133] Step S122: constructing an exponential function using the gradient and a preset adjustment coefficient.

[0134] In step S122, the gradient value corresponding to each pixel point can be used as an exponential parameter, and the exponential parameter can be adjusted using a preset adjustment coefficient to obtain the final exponential parameter. An exponential function is constructed with e as the base and the exponential parameter obtained in the previous step as the exponent.

[0135] Step S123: using an exponential function to process each pixel in the first depth image to obtain a processed first depth image.

[0136] In step S123, since each pixel in the depth image has a corresponding gradient value, the gradient value is substituted into the exponential function constructed in the previous step to obtain the exponential function value corresponding to each pixel. Using the exponential function value as the grayscale value corresponding to the pixel, a new depth image, also known as the depth boundary image, is obtained.

[0137] Specifically, the boundary of the depth image D can be calculated using the following formula (1) to obtain the processed depth image, that is, the depth boundary image A:

[0138]

[0139] Wherein α is the coefficient of the adjustment exponential function, and one optional solution is α=10. It is the Sobel operator, which is used to calculate the gradient corresponding to the pixel points in the image. express The norm of The length of the vector.

[0140] Figure 6 Schematic diagram of a depth image before and after processing provided in an embodiment of the present application.

[0141] Figure 6 A in it is the depth image before processing, Figure 6 The B in Figure 6 The processed depth image corresponding to A in , is also the depth boundary map.

[0142] like Figure 6 As shown in A, the horse is closer to the camera, so it will appear in a lighter color, while the background such as the ground and trees are farther away from the camera, so they will appear in a darker color. Figure 7 In A, the horse is drawn white and the entire background is drawn black. In real-world scenarios, different parts of the horse's body are at varying distances from the camera, resulting in varying depths and, consequently, colors. For example, a horse's nose and eyes might be closer to the camera than the rest of its body, and therefore appear lighter in the depth map.

[0143] like Figure 6 As shown in B, Figure 6 The B in the figure clearly shows the outline of the horse. It is understandable that for the sake of convenience, Figure 6Only the outline of the horse is shown in B. In actual application scenarios, the depth boundary map may also show details of various parts of the horse's body, such as ears, eyes, nose, etc.

[0144] According to the above formula (1) and Figure 6 It can be seen that in the processed depth image (ie, depth boundary image A), the grayscale value of each pixel contains the significance information of the depth change at the location of the pixel, rather than the depth information of the pixel.

[0145] It should be noted that steps S121-S123 only provide a method for obtaining a depth boundary image based on a gradient. In some implementations, other methods may be used to obtain a depth boundary image. For example, a Laplacian algorithm may be used to obtain a depth image boundary using a convolution kernel of a second-order derivative. A Difference of Gaussians (DoG) algorithm may also be used to obtain a depth image boundary by calculating the difference between the results of two Gaussian filters of different scales. It will be understood that the above is only an exemplary description of a depth boundary image acquisition method. In actual applications, other depth boundary image acquisition methods may also be used, and this application does not limit this.

[0146] Step S124: For the grayscale value of each pixel in the processed first depth image, the grayscale value greater than or equal to the preset threshold is adjusted to 1, and the grayscale value less than the preset threshold is adjusted to 0, to obtain the padded first depth image.

[0147] Step S125: in the padded first depth image, determine a pixel with a grayscale value of 1 as a first pixel.

[0148] In steps S124-S125, the depth boundary obtained in the above steps may be inaccurate due to factors such as image noise, uneven lighting, and algorithm limitations. In this case, the depth image processed in step S123 can be padded to ensure that its resolution meets the requirements of the actual application scenario, for example, to achieve the resolution of the first image.

[0149] Specifically, the image completion operation can be performed based on the size relationship between the grayscale value of each pixel and the preset threshold. It can be understood that the larger the grayscale value of a pixel, the greater the depth difference at the location of the pixel, the more significant the depth change, and the greater the possibility that the pixel is at the depth boundary. Conversely, it can be considered that the possibility of the pixel being at the depth boundary is smaller. For the case where the grayscale value is neither particularly large nor particularly small, it is impossible to accurately define whether the pixel is at the depth boundary. For the case where it is impossible to accurately define whether the pixel is at the depth boundary, the embodiment of the present application pre-sets a threshold. If the grayscale value is greater than or equal to the threshold, it is considered that the grayscale value is large enough, and the pixel corresponding to the grayscale value is in the depth boundary. Otherwise, it is considered that the grayscale value is too small, and the pixel corresponding to the grayscale value is not in the depth boundary.

[0150] For example, for the grayscale value of each pixel in the first depth image (i.e., depth boundary image A) processed in the aforementioned steps, grayscale values ​​greater than or equal to the threshold θ can be adjusted to 1, and grayscale values ​​less than the threshold θ can be adjusted to 0. At this point, it can be considered that in the padded first depth image (i.e., the adjusted depth boundary image A′), pixels with a grayscale value of 1 are points in the depth boundary, and pixels with a grayscale value of 0 are not points in the depth boundary. Therefore, a point with a grayscale value of 1 can be regarded as the first pixel, while a point with a grayscale value of 0 is not regarded as the first pixel.

[0151] Among them, it can be understood that the larger the preset threshold, the more points are considered to be in the depth boundary. Therefore, the more first pixel points there are, the more detailed the depiction of the depth boundary position, the denser the grid at the depth boundary position, and the better the effect of the generated gridded image. However, due to the increase in the number of first pixel points, the computing speed will also slow down. Correspondingly, the smaller the preset threshold, the sparser the grid at the depth boundary position, but the computing speed will also be faster. Therefore, a more reasonable preset threshold can be set by comprehensively considering the computing efficiency requirements and the fineness requirements of the gridded image. Exemplarily, the preset threshold can take a value of 0.5.

[0152] It can be understood that the purpose of steps S124-S125 is to improve the accuracy of the depth boundary in order to improve the rendering effect of the final gridded image. In some implementations, steps S124-S125 may not be performed, and other processing methods that can improve the accuracy of the depth boundary may be used. For example, morphological operations (such as expansion and corrosion) may be used to smooth the depth boundary, or an edge enhancement algorithm may be used to highlight the depth boundary. In other implementations, in order to increase the computing speed and reduce the computing load, steps S124-S125 may not be performed, and step S126 may be performed directly after step S123.

[0153] Step S126: in the processed first depth image, obtain a boundary, where the first depth boundary includes at least one first pixel point.

[0154] In step S126, a depth boundary can be obtained based on the processed first depth image (i.e., depth boundary image A). Specifically, in the depth boundary image, the grayscale value of each pixel can contain significance information of the depth change at the pixel's location. The depth change at the boundary is relatively significant. Therefore, the boundary can be obtained based on the grayscale value of each pixel in the depth boundary image.

[0155] In addition, if steps S124-S125 are executed before step S126, the depth boundary can be obtained based on the padded first depth image (i.e., the adjusted depth boundary image A'). Specifically, pixels with a grayscale value of 1 in image A' can be used as points in the depth boundary, and all points with a grayscale value of 1 form the boundary of the depth image.

[0156] After obtaining the boundary of the depth image, several first pixel points can be determined in the boundary. Figure 6 As shown in B, point P is a first pixel point. Multiple first pixel points can outline Figure 6 The outline of the horse in B. It can be understood that the first pixel point must be a point in the depth boundary, but the point in the depth boundary is not necessarily the first pixel point.

[0157] It should be noted that steps S121-S126 only provide a method for obtaining a first depth boundary. In some implementations, other methods may also be used to obtain the first depth boundary. Exemplarily, an encoder-decoder structure such as U-Net may be used to perform depth estimation on the first image. Then, based on the depth estimation result, an edge detection algorithm is applied to extract the depth boundary. An image segmentation method may also be used to segment the first image into regions with similar features (such as color and texture), and then determine the first depth boundary based on the boundaries of each region. The color and texture features of the first image may also be combined with the depth features of the first depth image to form a comprehensive feature representation. Using a trained deep learning model, the color and texture features and the depth features are processed simultaneously to obtain the first depth boundary. It will be understood that the above is only an exemplary description of the depth boundary acquisition method. In actual applications, other depth boundary acquisition methods may also be used, and this application does not limit this.

[0158] Step S13: setting at least one grid structure on the first depth image so that the first pixels are distributed in the grid structure.

[0159] In step S13, a grid structure is pre-set in the depth image, and the first pixel is distributed in the grid structure, so that the first pixel forms a certain positional relationship with the pre-set grid structure. Thereafter, a target grid structure can be constructed based on the positional relationship between the first pixel and the pre-set grid structure.

[0160] Among them, each network structure can be pre-set based on certain rules. For example, the grid in the network structure can be set to a square, triangle, parallelogram, etc. with a side length of a preset value. The rules between different network structures may be different. For example, two grid structures are pre-set in the depth image, namely grid structure 1 and grid structure 2, then the side length of the grid in network structure 1 and the side length of the grid in network structure 2 can be different, or the shape of the grid in network structure 1 and the shape of the grid in network structure 2 can be different.

[0161] Figure 7 This is a schematic diagram of a first pixel point distributed in a grid structure provided by an embodiment of the present application.

[0162] like Figure 7 As shown, Figure 7 The depth image in contains multiple first pixel points (such as Figure 7 A grid structure is set on the depth image. Therefore, the first pixel points are distributed in the grid structure, and each first pixel point has a relative position with a grid in the grid structure, forming a certain positional relationship.

[0163] In one implementation, step S13 may include the following steps S131 .

[0164] Step S131: Based on different resolutions, at least one level is set on the first depth image, so that each level has a corresponding grid structure.

[0165] In step S131, one or more layers are set on the first depth image, each of which has a different resolution. By setting a corresponding layer for each resolution, a grid structure corresponding to the resolution of the layer can be set in each layer, thereby enabling the first depth image to be analyzed based on different resolutions in different layers.

[0166] Each grid cell in the same grid structure has the same size, while the sizes of grid cells in different grid structures vary. It can be understood that the higher the resolution of a level, the denser the grid structure within that level is to capture more detail, and therefore the smaller the size of each grid cell within that level. Conversely, the lower the resolution of a level, the sparser the grid structure within that level is, and the larger the size of each grid cell within that level is.

[0167] Figure 8This is a schematic diagram of a hierarchical setting provided in an embodiment of the present application.

[0168] like Figure 8 As shown in the figure, based on different resolutions, three levels are set on the first depth image, and each level has a grid structure. Among them, the resolution of 1920×1080 corresponds to level 1, and level 1 has a grid structure of 3 rows and 3 columns; the resolution of 1280×720 corresponds to level 2, and level 2 has a grid structure of 2 rows and 2 columns; the resolution of 800×600 corresponds to level 3, and the grid structure of level 3 contains only one grid.

[0169] In one implementation, step S131 may include the following steps S1311 - S1312 .

[0170] Step S1311: Number the levels from large to small according to resolution.

[0171] In step S1311, the layers are numbered in a certain order according to the multiple different resolutions, for example, numbered in descending order of resolution. In this way, the first depth image can be analyzed layer by layer according to the layer number, that is, layer by layer in a fine-grained to coarse order.

[0172] For example, as mentioned above Figure 8 The three levels in the image are numbered as level 1, level 2, and level 3 in descending order of resolution. When analyzing the first depth image according to the level number, level 1 with a resolution of 1920×1080 may be analyzed first, followed by level 2 with a resolution of 1280×720, and finally by level 800×600.

[0173] It can be understood that the more levels of division there are, the more detailed the analysis of the first depth image is.

[0174] In addition, the levels can be numbered in other orders according to multiple different resolutions, for example, in order from smallest to largest resolution.

[0175] Step S1312: Determine the side length of the grid corresponding to each level according to the level number, so as to form a grid structure based on the side length.

[0176] In step S1312, different grid side lengths are set for the grid structure in each level. It can be understood that the degree of image refinement varies at different resolutions. The higher the resolution of the level, the finer the image display effect at that level. In order to obtain a finer display effect, the grid structure of that level also needs to be denser. Therefore, the number of grids in the grid structure is greater. Conversely, the lower the resolution of the level, the sparser the grid structure of that level, and the fewer the number of grids in the grid structure.

[0177] Therefore, based on the correlation between resolution and the density of the preset grid structure, the side length of the grid in the network structure can be determined according to the resolution. Since the resolution corresponds to the level number in step S1311, the side length of the grid in the grid structure can also be determined according to the level number. For example, the side length of the grid in a grid structure can be set to the sum of 2 raised to the power of N and 1, where N is the level number corresponding to the grid structure.

[0178] For example, the resolution of the first depth image is 2 n +1, the levels can be numbered in descending order of resolution to obtain a level list L = {L1, L2, ..., L n}. The list contains multiple different levels L i , where i∈

[0179] {1,2,…,n}. The resolution of level L1 is the largest among all levels, which is 2 n +1. At level L numbered i i In the grid structure, there are multiple quadrilateral grids S im Composition, that is, level L i ={S i1 ,S i2 ,…S im ,…,s ik}, where k is the number of grids in the grid structure. In this grid structure, each quadrilateral grid S im The side length is determined by the level number i, which is 2 i +1.

[0180] Figure 9 This is a schematic diagram of grid structure division in each level provided in an embodiment of the present application.

[0181] Figure 9 A in FIG is a schematic diagram of the grid structure division in level L1. Figure 9 B in FIG is a schematic diagram of the grid structure division in level L2.

[0182] like Figure 9 As shown, Figure 9 A grid structure is divided for a depth image with a resolution of 5×5. The specific process is as follows: 2 n +1=5, the number of levels n=2. Therefore, two levels L1 and L2 are set for the depth image. Figure 9 As shown in A, in level L1, i=1, quadrilateral mesh S 1m The side length is 2 i +1=2 1 +1=3. Figure 9As shown in B, in level L2, i = 2, quadrilateral mesh S 2m The side length is 2 i +1=2 2 +1=5.

[0183] S14: storing a first coordinate corresponding to the center of the edge of the grid in the grid structure and a second coordinate corresponding to the center point of the grid.

[0184] In step S14, for each grid in the grid structure at each level, the coordinates used to determine the position correspondence are recorded. In this way, in the subsequent step of determining the position correspondence, the pre-saved coordinates can be directly retrieved for calculation, thereby improving calculation efficiency.

[0185] S15: Determine, based on the first coordinate and the second coordinate, whether the first pixel point and the grid structure have a preset position correspondence, where the position correspondence includes: the first pixel point is located at the center point of the grid in the grid structure, and / or the first pixel point is located at the center of any side of the grid in the grid structure.

[0186] In step S15, illustratively, if the position correspondence relationship includes that the first pixel point is located at the center point of a grid in the grid structure, the coordinates of the grid center point may be pre-stored. If the position correspondence relationship includes that the first pixel point is located at the center of an edge of a grid in the grid structure, the coordinates of the center of the grid edge may be pre-stored.

[0187] It is understood that for other position correspondences, coordinates of other positions may also be stored. For example, if the position correspondence includes that the first pixel point is located at a vertex of any grid in the grid structure, the coordinates of the grid vertex may be pre-stored.

[0188] Figure 10 This is a schematic diagram of a first coordinate and a second coordinate provided in an embodiment of the present application.

[0189] like Figure 10 As shown, in a quadrilateral grid, there is a center point and the midpoints of the four sides. The coordinates of the center of the quadrilateral grid side (i.e., the first coordinate) and the coordinates of the grid center point (i.e., the second coordinate) are stored. When the subsequent position correspondence is determined, the specific value of each coordinate can be substituted into the calculation to determine whether the activation point is also located at the position of the coordinate.

[0190] It is understood that the purpose of steps S14-S15 is to pre-store the coordinates required for the calculation, thereby improving the calculation speed. Therefore, in some implementations, steps S14-S15 may not be performed, and step S16 may be performed directly after step S13 to reduce the required storage space.

[0191] Step S16: When the first pixel point and the grid structure have a preset position correspondence, determine at least one first activation point based on the grid structure to use the first activation point as an activation point set.

[0192] In step S16, based on the positional correspondence between the first pixel and the grid structure, a corresponding first activation point is determined in the grid structure, and an activation point set is constructed. This operation not only increases the number of activation points, but also ensures that the added activation points are related to the first pixel, that is, related to the depth boundary, thereby constructing a relatively dense grid structure in the depth boundary area.

[0193] The position of the first activation point can be determined based on the grid corresponding to the first pixel. For example, if the relative position of the first pixel and the grid structure conforms to a preset position correspondence, the vertex of the grid corresponding to the first pixel is used as the first activation point. For example, if the relative position of the first pixel and the grid structure conforms to a preset position correspondence, the center of the grid corresponding to the first pixel is used as the first activation point.

[0194] Furthermore, the first activation point can also be set as a point elsewhere in the grid corresponding to the first pixel, such as the midpoint of a grid edge, a point on a grid diagonal, etc. Thus, if the relative position of the first pixel and the grid structure conforms to a predetermined positional correspondence, the midpoint of the grid edge corresponding to the first pixel will be used as the first activation point, or a point at a specific position on the grid diagonal will be used as the first activation point. Furthermore, the first activation point can also be set as a point in another grid, such as a point in a grid adjacent to the grid corresponding to the first pixel.

[0195] In addition, if the positional relationship between the first pixel point and the grid in the grid structure does not conform to the preset positional correspondence, no operation is performed.

[0196] Step S17: constructing a target grid structure for the activation point set to generate a gridded image based on the target grid structure.

[0197] In step S17, the target network structure is constructed using the activation points in the activation point set, and facets are added to each grid of the target network structure to obtain a gridded image. Each activation point in the activation point set can be involved in the calculation to achieve the target grid structure. Alternatively, a number of representative activation points can be selected from the activation point set and then used to obtain the target network structure.

[0198] For example, the activation points in the activation point set can be used as mesh vertices to construct a target mesh structure containing multiple meshes. Based on the distribution of activation points in different areas, the target mesh structure can be made relatively dense in the depth boundary area and relatively sparse in the non-depth boundary area.

[0199] For example, the activation points can be directly connected to obtain a target mesh structure containing polygonal meshes such as triangles, quadrilaterals, and hexagons. Alternatively, the activation points can be connected based on specific optimization criteria or geometric constraints, such as maximum edge length constraints, to obtain a target mesh structure containing regular or irregular meshes. Alternatively, the activation points can be connected based on algorithms such as minimum spanning trees and shortest path algorithms to obtain a target mesh structure. Furthermore, other methods for connecting activation points are also possible and will not be discussed further here.

[0200] In one implementation, step S17 may be followed by the following step S18.

[0201] Step S18: Using the color information of the first image, color the target grid structure to obtain a colored gridded image.

[0202] In step S18, considering that the depth boundary does not contain the color information of the first image, the target network structure obtained based on the depth boundary does not contain color information. In order to obtain a colored gridded image, the color information of the first image can be used to color the target grid structure. The color information of the first image can be RGB (Red Green Blue) information, HSV (Hue Saturation Value) information, HSL (Hue Saturation Lightness) information, etc., which is not limited here.

[0203] As you can understand, the target network structure is composed of multiple triangular meshes, each of which is a hole, and holes cannot be colored. Therefore, first, add facets to each triangular mesh in the target network structure to fill the holes. Then, you can color each facet individually to obtain a colored mesh image.

[0204] Figure 11 This is the second flow chart of the method for generating a gridded image provided in an embodiment of the present application.

[0205] like Figure 11 As shown, the method includes the following steps S21-S29.

[0206] Step S21: Acquire a first depth image corresponding to the first image.

[0207] Step S22: Acquire a boundary of the first depth image as a first depth boundary corresponding to the first image, where the first depth boundary includes at least one first pixel point.

[0208] Step S23: setting at least one grid structure on the first depth image so that the first pixels are distributed in the grid structure.

[0209] Among them, steps S21-S23 refer to the aforementioned steps S11-S13 and are not repeated here.

[0210] Step S24: Determine the first pixel point as a second activation point and construct an activation point set based on the second activation point.

[0211] In step S24, an activation point set is constructed based on the first pixel. Unlike the previous embodiment, which determines the first activation point based on the first pixel and then constructs the activation point set based on the first activation point, this embodiment also includes the first pixel as an activation point in the activation point set.

[0212] Step S25: traverse the grids in the grid structure in order of level numbers from small to large.

[0213] In step S25, the vertex activation strategy is executed by traversing each mesh one by one. Specifically, a double loop can be performed based on the level and the mesh. For example, the meshes in the mesh structure corresponding to level 1 are traversed first, then the meshes in the mesh structure corresponding to level 2 are traversed, and so on until the meshes corresponding to all levels are traversed.

[0214] Step S26: If the grid has a positional correspondence with the activation point in the activation point set, the vertex or center point in the grid is determined as the first activation point, and the first activation point is added to the activation point set.

[0215] In step S26, the activation strategy is executed. Specifically, during the grid traversal process, each time a grid is visited, the positional correspondence between the grid and each activation point in the activation point set is analyzed, and a first activation point is determined based on the analysis results. The activation point set may include the second activation point from step S24, as well as the first activation point determined based on the positional correspondence in step S26.

[0216] In one implementation, step S26 may include steps S261 and / or S262.

[0217] Step S261: If the center point of a grid in the grid structure coincides with any activation point in the activation point set, then the vertex of the grid is determined to be the first activation point.

[0218] Step S262: If the center of any edge of the grid in the grid structure coincides with any activation point in the activation point set, the vertex of the grid and the center point of the grid are determined as the first activation point.

[0219] In steps S261 and S262, the position correspondence relationship can be that the center point of the grid in the network structure coincides with the activation point in the activation point set, or the center of any edge of the grid in the network structure coincides with the activation point in the activation point set. Different activation strategies can be adopted for different position correspondence relationships. For example, for the position correspondence relationship in which the center point of the grid coincides with the activation point, an activation strategy of activating the grid vertex can be adopted, and the grid vertex is determined to be the first activation point. For the position correspondence relationship in which the center of the grid edge coincides with the activation point, an activation strategy of activating the grid vertex and the center point can be adopted, and the grid vertex and the center point are determined to be the first activation point.

[0220] Figure 12 This is a schematic diagram of an activation strategy provided in an embodiment of the present application.

[0221] Figure 12 A is a schematic diagram of determining the first activation point when the grid and the activation point have a positional correspondence relationship. Figure 12 B is a schematic diagram of determining the first activation point when the grid and the activation point have another positional correspondence relationship.

[0222] like Figure 12 As shown in A in FIG, an activation point a is exactly at the center point of a quadrilateral mesh, then the four vertices A1-A4 of the quadrilateral mesh are activated, that is, these four vertices are determined as the first activation points. Figure 12 The arrow in A points to the first activated point.

[0223] like Figure 12 As shown in Figure 1, an activation point b falls exactly at the center of a quadrilateral edge (the center of the left edge of the quadrilateral in the figure). This activates the four vertices B1-B4 of the quadrilateral and the center point B5 of the quadrilateral. These four vertices and the center point are then determined as the first activation point. Figure 12 The arrow in B points to the first activated point.

[0224] It can be understood that the essence of steps S25-S26 is to analyze the positional relationship between the activation point and the grid in the grid structure. Therefore, in some implementations, the method of traversing the grid in S25-S26 can be adopted. The method of traversing the grid is specifically: traversing the grid and determining whether there is any activation point at the center of each grid or the center of the grid edge. In other implementations, the method of traversing the activation points can also be adopted. The method of traversing the activation points is specifically: traversing the activation points in the activation point set and determining whether each activation point is at the center of any grid or the center of the grid edge. In the specific implementation process, considering the computational efficiency, the method of traversing the grid can be adopted.

[0225] For example, the following is an activation method based on traversing activation points: let the activation point set be P, take the first pixel point as the second activation point, and form the initial activation point set P inii Therefore, in the initial stage of executing the activation strategy, there exists P = P inii The corresponding relationship is traversed through each activation point in the activation point set P, and the positional relationship between the activation point and each grid in the grid structure at each level is determined. If the activation point is at the center of any grid, the vertex of the grid is determined as the first activation point, and the first activation point is added to the activation point set P. If an activation point is at the center of an edge of any grid, both the vertex and the center point of the grid are determined as the first activation point, and the first activation point is added to the activation point set P.

[0226] For example, the following is an activation method based on traversing the grid:

[0227] In this activation method, the input is the initial activation point set P init , level list L, evenly divided grid point set Q; the output is the activation point set P.

[0228] The pseudo code of the activation method is as follows:

[0229]

[0230] The above is the pseudo code of the activation method.

[0231] In the pseudo code of the activation method, “P←P init " means: P init Assign to P, the activation point set P at this time is equal to the initial activation point set P init "for i=1 to n do" means: analyze each level one by one according to the order of level number i from small to large. i =L[i-1]" means: level L i Assignment. “for S im in L i "do" means: analyze the level L in turn i Each quadrilateral mesh S in im . “if S im The center point is in P then" and "S im Four vertices are added to P”: If the quadrilateral mesh S im If the center point of is in the activation point set P, then the quadrilateral mesh S im The four vertices of S are added as activation points to the activation point set P. im The center point of the edge is in P then" and "S imFour vertices and the center point are added to P” to indicate that: if the quadrilateral mesh S im If the edge center point of the quadrilateral mesh S is in the activation point set P, im The four vertices and center point of the quadrilateral mesh S are added as activation points to the activation point set P. "else" and "continue" mean: if the quadrilateral mesh S im If neither the center point nor the edge center point is in the active point set P, the loop continues. "P = P∪Q" means: the union of the active point set P and the set Q is used as the new active point set P. "returnP" means outputting the active point set P.

[0232] In the pseudo code of the activation method above, the first pixel point is used as the second activation point to form the initial activation point set P init , and P init The first activation point is added to the activation point set P before the traversal begins. During the traversal, the newly determined first activation point is added to the activation point set in real time, updating the activation point set P. In other words, the final activation point set P is formed based on the first and second activation points, and the final activation point set P is used to construct the target network structure.

[0233] In the pseudo code of the activation method described above, the input includes a uniformly divided grid point set Q. For setting the uniformly divided grid point set Q, the following steps S27-S28 may be used.

[0234] Step S27: uniformly dividing the first depth image into a plurality of grids according to preset intervals.

[0235] In step S27, it can be understood that the first pixel points are distributed in the depth boundary area. Generally, the closer to the center of the depth boundary area, the denser the first pixel points are distributed. Conversely, the closer to the non-depth boundary area, the sparser the first pixel points are distributed. Since the first activation point and the second activation point are determined based on the first pixel point, the first activation point and the second activation point are usually also densely distributed in the depth boundary area and sparsely distributed in the non-depth boundary area.

[0236] In order to solve the problem that there are fewer activation points in the non-depth boundary area, which leads to over-simplification of the network structure in this area, some activation points can also be set in the non-depth boundary area.

[0237] Specifically, the grid is evenly divided in the first depth image at a certain interval, so as to set the activation point using the evenly divided grid. The interval can be set based on the resolution of the first image. For example, for a 1000×1000 pixel image, the interval can be set to 10 pixels. In both the x-axis and y-axis directions, a straight line is drawn every 10 pixels, so that a grid structure of 100 rows and 100 columns can be divided in the first depth image. In addition, in addition to square grids, rectangular grids, triangular grids, parallelogram grids, etc. can also be divided. For example, in the x-axis direction, a straight line is drawn every 10 pixels; in the y-axis direction, a straight line is drawn every 15 pixels, so that the grid in the grid structure divided is a rectangle.

[0238] Figure 13 A schematic diagram of a uniformly divided grid provided in an embodiment of the present application.

[0239] like Figure 13 As shown in the figure, in the first depth image, nine straight lines are drawn horizontally and vertically at regular intervals, with the distance between adjacent straight lines being equal. This division method results in a grid structure with 10 rows and 10 columns, where each square grid has the same size. Connecting the diagonals of each square grid creates a grid structure consisting of multiple triangular grids, each of which has the same size.

[0240] Step S28: taking the vertices of the multiple evenly divided grids as third activation points, and adding the third activation points to the activation point set.

[0241] In step S28, after the grid is evenly divided, the vertices of the grid are added to the activation point set as the third activation point. Figure 13 In

[15] , the vertices of the evenly divided triangular mesh can be used as third activation points and added to the activation point set. The activation point set now includes both the first and second activation points, as well as the third activation point. Therefore, the activation points in the non-depth boundary region will not be too sparse. Therefore, the network structure constructed based on the activation points in the activation point set will not be too sparse in the non-depth boundary region, resulting in oversimplification in that region.

[0242] In addition, in addition to using the vertices of the evenly divided grid as the third activation point, points at other positions of the grid may also be selected as the third activation point, such as the center point of the grid.

[0243] It can be understood that the purpose of steps S27-S28 is to increase the grid density in the non-depth boundary area to avoid the image being too rough due to sparse grids in the non-depth boundary area. In some implementations, steps S27-S28 may not be performed, and step S29 may be performed directly after step S26. For example, in the case of a relatively simple background, setting a sparse grid in the background area has little effect on the fineness of the image, and can also improve the overall computing speed. For example, if a two-dimensional picture includes a solid background and the image of a horse, there is no need to carefully depict the solid background. Therefore, for this two-dimensional picture, the third activation point may not be set, and only the first activation point and the second activation point may be set.

[0244] Step S29: constructing a target grid structure for the activation point set.

[0245] For step S29, please refer to the aforementioned step S18 and will not be repeated here.

[0246] In one implementation, step S29 may include the following steps.

[0247] The target grid structure is colored using the color information of the first image to obtain a colored gridded image.

[0248] Here, this step refers to the aforementioned step S19 and will not be repeated here.

[0249] Figure 14 This is the third flow chart of the method for generating a gridded image provided in an embodiment of the present application.

[0250] like Figure 14 As shown, the aforementioned step S19 includes the following steps S19a-S19f.

[0251] Step S19a: Divide the first depth image into two triangles using the diagonal line of the first depth image.

[0252] In step S19a, the vertices of the processed first depth image (i.e., the depth boundary image) are used as starting points for triangulation. A diagonal line is drawn in the depth image to obtain two isosceles right triangles. The triangular mesh is then divided based on these two isosceles right triangles.

[0253] Step S19b: Determine whether each triangle matches the preset conditions. The preset conditions are: the midpoint of the base of the triangle is an activation point in the activation point set, and the length of the base of the triangle is greater than

[0254] Step S19c: When the triangle matches the preset conditions, connect the midpoint of the base and the right-angle vertex of the triangle to divide the triangle into two new triangles, and return to step S19b.

[0255] In steps S19b-S19c, the triangular mesh is divided. Specifically, a loop is used to determine whether each triangle matches the preset conditions. If not, no operation is performed on the triangle. If it does, the triangle is divided into two smaller triangles. The two smaller triangles are then each determined to match the preset conditions, and further triangular mesh division is performed on each of them. This process continues until every triangle does not match the preset conditions.

[0256] Figure 15 This is a schematic diagram of a triangulation process provided in an embodiment of the present application.

[0257] Figure 15 A in FIG. 1 is a schematic diagram of dividing the first depth image into two triangles a and b. Figure 15 B in the figure is a schematic diagram of dividing triangle a and triangle b into small triangles a1, a2 and b1, b2 respectively. Figure 15 C in FIG. 1 is a schematic diagram of dividing triangle a1 into two small triangles a11 and a12.

[0258] like Figure 15 As shown in A in the figure, the four dots represent the four vertices A1-A4 of the first depth image. The figure contains a diagonal line running from the upper left to the lower right (from A1 to A4). This diagonal line divides the first depth image into two isosceles right triangles a and b, and the vertices of the isosceles right triangles a and b are all vertices of the first depth image.

[0259] like Figure 15 As shown in B, diamond black dots B1 and B2 represent activation points. Figure 15 As can be seen from B in the figure, the midpoints of the bases of isosceles right triangles a and b coincide with activation point B1. Therefore, we can connect the right-angled vertices and base midpoints of the triangles, respectively. This means connecting the circle A3 in the lower left corner to the diamond-shaped black dot B1 in the center, and connecting the circle A2 in the upper right corner to the diamond-shaped black dot B1 in the center. This divides isosceles right triangle a into two smaller triangles a1 and a2, and isosceles right triangle b into two smaller triangles b1 and b2. It can be seen that the smaller triangles a1 and a2, as well as b1 and b2, are also isosceles right triangles.

[0260] like Figure 15As shown in Figure C, the base midpoint of small triangle a1 coincides with activation point B2. Therefore, we can connect the right-angled vertex B2 and the base midpoint B1 of triangle a1 to split triangle a1 into two small triangles a11 and a12. As can be seen, triangles a11 and a12 are also isosceles right triangles. Since the base midpoints of small triangles a11 and a12 do not have activation points, the binary division operation is not performed on triangles a11 and a12. Furthermore, the base midpoints of the other triangles a2, b1, and b2 do not have activation points, so the binary division operation is not performed on triangles a2, b1, and b2 either.

[0261] It can be understood that this implementation method only performs a bisection operation on a triangle when the midpoint of the triangle base coincides with an activation point. If the midpoint of the triangle base is not an activation point, the triangle cannot be bisected. Therefore, the midpoint of the triangle base can be set as an activation point in the activation point set as a preset condition. If the triangle does not meet this preset condition, the bisection operation on the triangle is terminated.

[0262] In addition, considering that the minimum pixel of each side of the quadrilateral is 1, for the isosceles right triangle divided by the quadrilateral, the lengths of the two right-angled sides are 1 and the length of the base is That is, the length of the base of the smallest isosceles right triangle is Therefore, the base of the triangle can be made longer than As a precondition. If the length of the base of the triangle is not greater than The triangle is too small to be bisected, so stop bisecting the triangle.

[0263] Step S19d: When the triangle does not match the preset condition, the vertex connection relationship of the mesh in the target mesh structure on the two-dimensional plane is determined based on the vertex connection relationship between the multiple triangles.

[0264] In step S19d, if all triangles in the image do not meet the bisection condition, the triangular meshing process stops. The vertex connectivity of each triangle at this point is obtained. This vertex connectivity is used as the vertex connectivity of the target mesh structure on the two-dimensional plane. In other words, the projection of the target mesh structure on the two-dimensional plane coincides with the lines connecting the triangle vertices.

[0265] Figure 16 This is a schematic diagram of an activation point and a corresponding vertex connection relationship provided in an embodiment of the present application.

[0266] Figure 16 A in the figure is a schematic diagram of an activation point. Figure 16 The B in Figure 16 Schematic diagram of the vertex connection relationship corresponding to A in .

[0267] like Figure 16 As shown in A in the figure, the figure contains several dots, which are the activation points in the activation point set. Figure 16 As shown in B in the figure, draw a diagonal line from the upper left corner to the lower right corner in the image to divide the square image into two isosceles right triangles. Then, using a recursive method, the triangles that meet the preset conditions are divided into two new small triangles in sequence until the base of all triangles does not meet the preset conditions. After all triangles are divided, the connection relationship of the triangle vertices in the final network structure is the vertex connection relationship of the mesh in the target network structure on the two-dimensional plane. It can be seen that the final network structure is composed of several triangles, each of which is the projection of a triangular mesh in the target network structure on the two-dimensional plane.

[0268] Step S19e: Determine information about the vertices of the mesh in the target mesh structure in the depth direction according to the first depth image.

[0269] Step S19f: Construct a target grid structure in three-dimensional space based on the information in the depth direction and the vertex connection relationship on the two-dimensional plane.

[0270] In steps S19e-S19f, it can be understood that the triangle division in the aforementioned steps is performed on the processed depth image (i.e., the depth boundary image). The depth boundary image only contains the depth difference information of each pixel in the image, and does not contain depth information. Therefore, the divided triangles do not contain the depth information of the first image. In other words, the vertex connection relationship of the triangle is the connection relationship on the two-dimensional plane, and does not include the connection relationship in the depth dimension. The purpose of this solution is to generate a three-dimensional target network structure. Therefore, in addition to using the vertex connection relationship of the triangle as the vertex connection relationship of the network structure on the two-dimensional plane, it is also necessary to extract depth information from the first depth image as the information of the target network structure in the depth dimension. Specifically, the position of the mesh vertex on the two-dimensional plane can be determined based on the vertex connection relationship of the target network structure on the two-dimensional plane. Then, based on the depth information of each pixel contained in the first depth image, the depth information of the pixel at the location of the mesh vertex is determined as the depth information of the mesh vertex. In this way, the information on the two-dimensional plane and the information on the depth dimension can be combined to obtain the information of each mesh in the target mesh structure in three-dimensional space, that is, the three-dimensional target mesh structure is obtained.

[0271] Figure 17 This is the fourth flow chart of the method for generating a gridded image provided in an embodiment of the present application.

[0272] like Figure 17 As shown, the aforementioned step S19 includes the following steps S19g-S19o.

[0273] Step S19g: Divide the first depth image into two triangles using the diagonal line of the first depth image.

[0274] Among them, step S19g refers to the aforementioned step S19a and is not repeated here.

[0275] Step S19h: construct a binary tree with the triangle obtained by dividing the first depth image as the root node.

[0276] In step S19h, considering that the binary tree can clearly show the process of bisecting the triangle in the previous step, and the process of bisecting the triangle is consistent with the recursive definition of the binary tree, the binary tree can be used to perform the operations of the triangle meshing process. Based on this, the binary tree is constructed using triangles. First, the two largest triangles, that is, the two triangles obtained by dividing the first depth image diagonally, are used as root nodes. In this way, two binary trees are obtained, and the root node of each binary tree corresponds to a triangle.

[0277] Step S19i: Determine whether each triangle matches the preset conditions. The preset conditions are: the midpoint of the base of the triangle is an activation point in the activation point set, and the length of the base of the triangle is greater than

[0278] Step S19j: When the triangle matches the preset conditions, connect the midpoint of the base and the right-angle vertex of the triangle to divide the triangle into two new triangles.

[0279] Among them, steps S19i-S19j refer to the aforementioned steps S19b-S19c and are not repeated here.

[0280] Step S19k: After dividing the triangle into two new triangles, construct two child nodes on the node corresponding to the triangle, and update the node information of the node and child nodes corresponding to the triangle, and return to step S19i.

[0281] In step S19k, each time a large triangle is divided into two smaller triangles, two new nodes are created in the binary tree. For example, to divide triangle a into triangles a1 and a2, two child nodes are created on the node corresponding to triangle a, one for triangle a1 and the other for triangle a2. In this way, the parent-child relationship between binary tree nodes can be used to represent the relationship between large triangles and smaller triangles.

[0282] Figure 18 The embodiment of this application provides Figure 15 Schematic diagram of the binary tree constructed by triangulation processing.

[0283] Figure 18A, B, and C in the figure are schematic diagrams of the construction process of a binary tree with triangle a as a node. Figure 18 D in the figure is a schematic diagram of two binary trees constructed with triangle a and triangle b as root nodes respectively.

[0284] like Figure 18 As shown in A in FIG, first, two triangles are divided by the diagonal line of the first depth image, which are used as the root nodes of the binary tree, and the node information of the root nodes is stored. Figure 16 For the isosceles right triangle a in A, build a binary tree with a as the root node. At this time, the binary tree only contains the root node a.

[0285] like Figure 18 As shown in Figure B, after triangle a is split into two new triangles a1 and a2, two child nodes are created on the node corresponding to triangle a. These two child nodes correspond to the two new triangles a1 and a2, respectively. In other words, root node a becomes the parent node of triangles a1 and a2, and triangles a1 and a2 become the two child nodes of root node a. Simultaneously, the node information corresponding to triangle a is updated, and the node information corresponding to triangles a1 and a2 is stored separately.

[0286] like Figure 18 As shown in C, after triangle a1 is divided into a11 and a12, triangle a11 and triangle a12 are used as two child nodes of node a1. This process is repeated recursively until all triangles that match the preset conditions are divided. In addition, triangle b is also divided into Figure 18 Use methods A to C in the above to construct a binary tree.

[0287] like Figure 18 The two binary trees obtained are shown in D. The root nodes of these two binary trees are the nodes corresponding to triangle a and triangle b respectively, and each triangle has a corresponding node in the binary tree.

[0288] In a binary tree, the node information of each node includes one or more of the vertex coordinates, base midpoint coordinates, child node information, and flag information of the triangle corresponding to the node. Among them, the vertex coordinates of the triangle are used to determine the positions of the three vertices of the triangle. The base midpoint coordinates of the triangle are used to determine whether the base midpoint coincides with an activation point in the activation point set, and in the case of coincidence, the base midpoint is used to divide the triangle into two parts. The child node information contains the information of the small triangles obtained by splitting each triangle, which is used to record the hierarchical relationship of the triangle bisection. The flag information is used to identify whether the node is a leaf node. The initial flag information of each node can be set to true. After constructing the child node corresponding to a node, the child node information of the node can be updated, and the flag information of the node can be modified to false to indicate that the node is not a leaf node.

[0289] The following is a triangulation method based on binary tree partitioning triangles:

[0290] In this algorithm, the input is the activation point set P and the initial triangle node set T = {t1, t2} consisting of two root nodes; the output is two triangle node binary trees Tree1 and Tree2.

[0291] The pseudo code of the triangulation method is as follows:

[0292]

[0293] With t1 and t2 as root nodes, we get the triangle node binary trees Tree1 and Tree2.

[0294] return Tree1,Tree2

[0295] The above is the pseudo code of the triangulation method.

[0296] In the pseudo code of the triangulation method, “for t i in T do" means: traverse all nodes t in the triangle node set T i . “if t i is a leaf node and t i Base length greater than and t i The center point of the bottom edge is in P, then "connect the center point of the bottom edge and the right angle vertex, and t i Divide into two triangle nodes", "update t i Subnode information, and add two subnodes to T" and "t i The leaf node flag position is false" means: if the node t i is a leaf node, and node t iIf the corresponding triangle meets the preset conditions, the node t i The corresponding triangle is divided into two small triangles, two child nodes are obtained, and node t is updated. i The child node information is added to the triangle node set T, and finally the node t is updated. i Specifically, the leaf node flag in the node information is updated to false. "else" and "continue" indicate that if the node is not a leaf or does not meet the pre-set conditions, the loop continues. "return Tree1, Tree2" indicates that the binary trees Tree1 and Tree2 are output.

[0297] It is understood that steps S19h and S19k utilize the unique data structure of a binary tree to store and calculate the triangle nodes. In some implementations, steps S19h and S19k may be omitted and other data structures such as linked lists may be used instead of binary trees. The specific steps are not repeated here.

[0298] Step S191: When the triangle does not match the preset conditions, traverse the binary tree constructed based on the triangle to obtain multiple leaf nodes on the binary tree.

[0299] In step S191, after constructing a binary tree based on the triangle, traverse from the root node to obtain all leaf nodes in the binary tree. Among them, the depth-first traversal order can be used, such as pre-order traversal, in-order traversal, and post-order traversal, or the breadth-first traversal order can be used, such as level-order traversal. Figure 19 For the two binary trees obtained in D, for the binary tree with root node a, traverse from root node a. Taking the depth-first pre-order traversal order as an example, check nodes a, a1, a11, a12, and a2 in sequence, and obtain leaf nodes a11, a12, and a2. For the binary tree with root node b, traverse from root node b. Taking the depth-first pre-order traversal order as an example, check nodes b, b1, and b2 in sequence, and obtain leaf nodes b1 and b2. Therefore, the vertex connectivity relationship between triangles a11, a12, a2, b1, and b2 is determined.

[0300] Step S19m: Determine the vertex connection relationship of the triangle corresponding to the leaf node as the vertex connection relationship of the grid in the target grid structure on the two-dimensional plane.

[0301] In step S19m, the vertex connection relationship of the grid in the target network structure is determined using the leaf nodes. Figure 16 As can be seen from C, the vertex connection relationship between triangles a11, a12, a2, b1, and b2 is determined. Figure 16Based on this, the vertex connection relationship of the triangles corresponding to the binary tree leaf nodes can be used as the vertex connection relationship of the grid on the two-dimensional plane in the target grid structure.

[0302] Step S19n: Determine information about the vertices of the mesh in the target mesh structure in the depth direction according to the first depth image.

[0303] Step S19o: Construct a target mesh structure in three-dimensional space based on the information in the depth direction and the vertex connection relationship on the two-dimensional plane.

[0304] Among them, steps S19n-S19o refer to the aforementioned steps S19e-S19f and are not repeated here.

[0305] Figure 19 This is a flowchart of the wallpaper generation method provided in an embodiment of the present application.

[0306] like Figure 19 As shown, the method includes the following steps S31-S33.

[0307] Step S31: In response to a wallpaper selection instruction, a pre-selected wallpaper picture is determined from at least one picture in a gallery, where the pre-selected wallpaper picture is a two-dimensional image.

[0308] In step S31, the user selects a pre-selected wallpaper image from the gallery. Specifically, the user first issues a wallpaper selection command through touch, press, voice control, or other methods. The electronic device responds to the wallpaper selection command by displaying the 2D images stored in the gallery. The user then selects one of the 2D images displayed by the electronic device as the pre-selected wallpaper image. The pre-selected wallpaper image is the image that the user wishes to set as the 3D wallpaper.

[0309] Step S32: Process the pre-selected wallpaper image to obtain the target network structure corresponding to the pre-selected wallpaper image.

[0310] In step S32, the preselected wallpaper image is gridded using the gridded image generation method described in the previous embodiment to obtain a three-dimensional target grid structure. The detailed processing method is described in steps S11-S18, S21-S29, S19a-S19f, and S19g-S19o of the previous embodiment and will not be repeated here.

[0311] Step S33: Utilize a renderer to perform 3D rendering on the target network structure corresponding to the pre-selected wallpaper image to obtain a colored gridded image, and use the colored gridded image as a three-dimensional wallpaper image.

[0312] In step S33, the three-dimensional target grid structure and the two-dimensional wallpaper are input into a renderer, and a three-dimensional wallpaper image is obtained using the color information of the two-dimensional wallpaper and the three-dimensional target grid structure.

[0313] Figure 20 This is a schematic diagram of an embodiment of the present application for processing a preselected wallpaper image using the aforementioned image gridding method.

[0314] Figure 21 This embodiment of the present application provides Figure 20 A magnified image of the target grid structure.

[0315] like Figure 20 and Figure 21 As shown, first, the depth of the input color two-dimensional image (that is, the color pre-selected wallpaper image) is estimated to obtain the corresponding depth image. Then, edge detection is performed on the depth image to obtain the depth boundary of the pre-selected wallpaper image. Based on the pixel points in the depth boundary, multiple activation points are obtained. The activation points are used as the input of the triangulation algorithm, and triangles are divided based on the activation points. Based on the vertex connection relationship of the divided triangles, the connection relationship of the mesh vertices on the two-dimensional plane is obtained.

[0316] Finally, a three-dimensional wallpaper image is obtained based on the color information (such as RGB information), depth image, and vertex connection relationship of the input two-dimensional image. Among them, the three-dimensional target grid structure without color information can be obtained based on the connection relationship and depth image of the grid vertices on the two-dimensional plane. Figure 21 The figure shows a 3D target mesh structure without color information. The renderer then uses the color information of the input 2D image to perform 3D rendering on the target mesh structure without color information. This 3D rendering process yields a colorful 3D mesh image, which can be used as a 3D wallpaper.

[0317] In one implementation, after obtaining the three-dimensional wallpaper image, it can be set as the wallpaper of the electronic device, saved in the local wallpaper library of the wallpaper management service, or uploaded to the wallpaper resource community of the wallpaper management service.

[0318] Figure 22 This is a schematic diagram of a user operation interface for a three-dimensional wallpaper setting scene provided in an embodiment of the present application.

[0319] like Figure 22 As shown in Figure 1A, the main interface 10 of the electronic device can display multiple types of application icons, including a wallpaper application icon 101. In response to a user's operation of opening a wallpaper application, such as a first click operation or a first touch operation 11 on the wallpaper application icon 101, the electronic device calls the wallpaper management service of the application framework layer and starts the wallpaper application through the wallpaper management service.

[0320] like Figure 22 Figure B shows the wallpaper application interface 20 displayed by the electronic device after the wallpaper application is launched. This interface 20 can display various wallpaper icons, such as 2D wallpaper icons and 3D wallpaper icons, as well as a recommended list. In response to a second click or touch 12 on the 3D wallpaper icon 102, the user is redirected to the 3D wallpaper module of the wallpaper application.

[0321] like Figure 22 As shown in FIG. 3D wallpaper module interface 30 of the wallpaper application, in response to the user's request to generate a three-dimensional wallpaper, for example Figure 5 In C, the third click operation or the third touch operation 13 on the generate three-dimensional wallpaper button 103 causes the wallpaper application to access and display the pictures stored in the local gallery of the electronic device.

[0322] like Figure 22 Figure D shows an image display interface 40 for the local image gallery of the electronic device. In response to the user's fourth click or touch 14 on a pre-selected wallpaper image 104, the wallpaper application performs a gridding process on the pre-selected wallpaper image, ultimately generating a three-dimensional wallpaper image. After the three-dimensional wallpaper image is generated, the wallpaper application can display a three-dimensional wallpaper preview interface.

[0323] like Figure 22 As shown in Figure E, it is a 3D wallpaper preview interface 50. In 3D wallpaper preview interface 50, in response to a user's touch or click of each button, the corresponding task is executed. For example, in response to the user's fifth click or touch 15 on the Set as Wallpaper button 105, the 3D wallpaper image is set as the wallpaper of the electronic device. In addition, in response to clicks or touches of other buttons, the generated 3D wallpaper image can be previewed and uploaded to the community.

[0324] I understand. Figure 22 This is only one implementation method of the embodiment of the present application. The three-dimensional wallpaper setting can also be achieved through other methods, such as entering the program for generating three-dimensional wallpaper by clicking the wallpaper menu of the theme application; or entering the program for generating three-dimensional wallpaper by clicking the settings button and selecting desktop and wallpaper in the pop-up setting item list.

[0325] Other embodiments of the present application provide a device for generating a gridded image.

[0326] Figure 23 This is a schematic diagram of a grid image generation device provided in an embodiment of the present application.

[0327] like Figure 23 As shown, the gridded image generation device may include: a display screen 1001, a memory 1002, a processor 1003, and a communication module 1004. Each of the aforementioned components may be connected via one or more communication buses 1005. Display screen 1001 may include a display panel 10011 and a touch sensor 10012. Display panel 10011 is used to display images. Touch sensor 10012 may transmit detected touch operations to an application processor to determine the type of touch event and provide visual output related to the touch operation via display panel 10011. Processor 1003 may include one or more processing units, such as an application processor, a modem processor, a graphics processor, an image signal processor, a controller, a video codec, a digital signal processor, a baseband processor, and / or a neural network processor. The different processing units may be independent devices or integrated into one or more processors. Memory 1002 is coupled to processor 1003 and is used to store various software programs and / or computer instructions. Memory 1002 may include volatile memory and / or non-volatile memory. When the processor executes the computer instructions, the device for generating a gridded image can perform the various functions or steps performed by the above method embodiments.

[0328] When the software program and / or multiple groups of instructions in the memory 1002 are executed by the processor 1003, the device for generating a gridded image implements the following method steps: obtaining a first depth boundary corresponding to the first image, the first depth boundary is the boundary of the first depth image corresponding to the first image, and the first depth boundary includes at least one first pixel point; setting at least one grid structure on the first depth image so that the first pixel points are distributed in the grid structure; when the first pixel points and the grid structure have a preset position correspondence, determining at least one first activation point based on the grid structure, the first activation point includes at least one of the vertices and center points of each grid in the grid structure; constructing a target grid structure for the first activation point to generate a gridded image based on the target grid structure.

[0329] The present application also provides an electronic device, including: a processor, a memory and a touch screen; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the grid image generation method or wallpaper generation method in any implementation method of the above embodiments.

[0330] An embodiment of the present application also provides a chip system, which includes at least one processor and at least one interface circuit. The processor and the interface circuit can be interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (such as the memory of an electronic device). For another example, the interface circuit can be used to send signals to other devices. Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which is not specifically limited in the embodiment of the present application.

[0331] The embodiment of the present application further provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed in the above-mentioned electronic device (such as Figure 3 When the method is executed on the electronic device 100 shown in the figure, the electronic device is enabled to perform each function or step in the above method embodiment.

[0332] The embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the functions or steps executed by the mobile phone in the above method embodiment.

[0333] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0334] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0335] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0336] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0337] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0338] The above content is only a specific embodiment of this application, but the scope of protection of this application is not limited to this. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for generating a gridded image, characterized in that: The method comprises: Obtaining a first depth boundary corresponding to the first image, where the first depth boundary is a boundary of a first depth image corresponding to the first image, and the first depth boundary includes at least one first pixel point; Setting at least one grid structure on the first depth image so that the first pixels are distributed in the grid structure; In a case where the first pixel point and the grid structure have a preset position correspondence relationship, determining at least one first activation point based on the grid structure, where the first activation point includes at least one of a vertex and a center point of each grid in the grid structure; A target grid structure is constructed for the first activation point to generate a gridded image based on the target grid structure.

2. The method according to claim 1, characterized in that The setting at least one grid structure on the first depth image includes: Based on different resolutions, at least one level is set on the first depth image so that each level has a corresponding grid structure, wherein each resolution corresponds to one level, and in the same level, the size of each grid in the grid structure is the same.

3. The method according to claim 2, characterized in that The step of setting at least one level on the first depth image based on different resolutions so that each level has a corresponding grid structure includes: Numbering the levels from largest to smallest according to the resolution; Determining the side length of the grid corresponding to each level according to the level number, so as to form the grid structure based on the side length; The side length is the sum of 2 raised to the Nth power and 1, and N is the level number corresponding to the side length.

4. The method according to claim 1, wherein After setting at least one grid structure on the first depth image, the method further includes: Storing a first coordinate corresponding to a center of an edge of the grid in the grid structure and a second coordinate corresponding to a center point of the grid; Based on the first coordinate and the second coordinate, determine whether the first pixel point has the position correspondence with the grid structure, wherein the position correspondence includes: the first pixel point is located at the center point of the grid in the grid structure, and / or the first pixel point is located at the center of any side of the grid in the grid structure.

5. The method according to claim 3, characterized in that The constructing a target grid structure for the first activation point to generate a gridded image based on the target grid structure includes: Determining the first pixel point as a second activation point; forming an activation point set based on the first activation point and the second activation point; The target grid structure is constructed based on the activation point set.

6. The method according to claim 5, characterized in that When the first pixel point and the grid structure have a preset position correspondence relationship, determining at least one first activation point based on the grid structure includes: Traversing the grids in the grid structure according to the order of the level numbers from small to large; If the grid has the position correspondence with the activation point in the activation point set, the vertex or the center point of the grid is determined as the first activation point, and the first activation point is added to the activation point set.

7. The method according to claim 6, characterized in that If the grid has the position correspondence with the activation point in the activation point set, determining the vertex or the center point of the grid as the first activation point includes: If the center point of the grid in the grid structure coincides with any activation point in the activation point set, determining the vertex of the grid as the first activation point; and / or, If the center of any edge of the grid in the grid structure coincides with any activation point in the activation point set, the vertex of the grid and the center point of the grid are determined to be the first activation point.

8. The method according to claim 5, characterized in that Before constructing the target grid structure based on the activation point set, the method further includes: Evenly dividing the first depth image into a plurality of grids according to a preset interval; Vertices of the multiple evenly divided meshes are used as third activation points, and the third activation points are added to the activation point set.

9. The method according to claim 5, characterized in that The constructing a target grid structure based on the activation point set includes: Dividing the first depth image into two triangles using a diagonal line of the first depth image; Determining whether each of the triangles matches a preset condition, wherein the preset condition is: a midpoint of the base of the triangle is an activation point in the activation point set, and a length of the base of the triangle is greater than √2; If the triangle matches the preset condition, connecting the midpoint of the base and the right-angled vertex of the triangle to divide the triangle into two new triangles, and returning to the step of determining whether each of the triangles matches the preset condition; In a case where the triangle does not match the preset condition, determining a vertex connectivity relationship of the mesh in the target mesh structure on a two-dimensional plane based on vertex connectivity relationships between a plurality of the triangles; determining, according to the first depth image, information of vertices of the mesh in the target mesh structure in a depth direction; The target grid structure is constructed in three-dimensional space according to the information in the depth direction and the vertex connection relationship on the two-dimensional plane.

10. The method according to claim 9, characterized in that Determining the vertex connectivity relationship of the mesh in the target mesh structure on a two-dimensional plane based on the vertex connectivity relationship between the plurality of triangles includes: Traversing a binary tree constructed based on the triangle to obtain a plurality of leaf nodes on the binary tree, wherein each node on the binary tree corresponds to one of the triangles, and the node information of each node includes at least one of vertex coordinates, base midpoint coordinates, child node information, and flag information of the triangle corresponding to the node, wherein the flag information is used to identify whether the node is a leaf node; Determine the vertex connection relationship of the triangle corresponding to the leaf node as the vertex connection relationship of the grid in the target grid structure on the two-dimensional plane.

11. The method according to claim 10, characterized in that Before traversing the binary tree constructed based on the triangle, the method further includes: Constructing the binary tree with the triangle obtained by dividing the first depth image as a root node; After dividing the triangle into the two new triangles, constructing two child nodes on the node corresponding to the triangle, and updating the node information of the node corresponding to the triangle and the node information of the child nodes, wherein the two child nodes are the nodes corresponding to the two new triangles respectively.

12. The method according to claim 5, characterized in that The obtaining of a first depth boundary corresponding to the first image includes: Executing a depth estimation algorithm on the first image to obtain a first depth image corresponding to the first image, wherein the depth estimation algorithm is executed using a monocular depth estimation network, and the monocular depth estimation network includes at least one of a MiDaS network, a DORN network, a Monodepth network, and a DepthGAN network; A boundary of the first depth image is obtained as the first depth boundary corresponding to the first image.

13. The method according to claim 12, characterized in that The acquiring the boundary of the first depth image includes: Calculating the gradient of each pixel in the first depth image using a Sobel operator; Constructing an exponential function using the gradient and a preset adjustment coefficient; Processing each pixel in the first depth image using the exponential function to obtain a processed first depth image; In the processed first depth image, the boundary is obtained.

14. The method according to claim 12, characterized in that After obtaining the processed first depth image, the method further includes: For each pixel in the processed first depth image, the grayscale value greater than or equal to a preset threshold is adjusted to 1, and the grayscale value less than the preset threshold is adjusted to 0, to obtain the padded first depth image; In the padded first depth image, the pixel point having the grayscale value of 1 is determined to be the first pixel point.

15. The method according to claim 1, wherein After constructing the target grid structure based on the first activation point, the method further includes: The target grid structure is colored using the color information of the first image to obtain the colored gridded image.

16. A wallpaper generation method, characterized in that: The method comprises: In response to the wallpaper selection instruction, determining a preselected wallpaper picture from at least one picture in the gallery, wherein the preselected wallpaper picture is a two-dimensional image; Processing the preselected wallpaper image using the method according to any one of claims 1 to 15 to obtain a target network structure corresponding to the preselected wallpaper image; A renderer is used to perform 3D rendering on the target network structure corresponding to the preselected wallpaper image to obtain a colored gridded image, and the colored gridded image is used as a three-dimensional wallpaper image.

17. An electronic device, characterized in that: It includes a memory and a processor; the memory and the processor are coupled; the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device executes the grid image generation method according to any one of claims 1 to 15 or the wallpaper generation method according to claim 16.

Images