Mapping method and device, storage medium and electronic equipment
By obtaining depth images and processing, the edge outline map of the three-dimensional model map is determined, thereby selecting the target map, so that the midpoint of the target object is located in the center area, solving the problem of distortion of the three-dimensional model display and improving the display effect.
Patent Information
- Application Number
- CN202510119349.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-24
- Publication Date
- 2025-05-27
AI Technical Summary
In the prior art, the three-dimensional model may have a distortion problem after rendering the map, resulting in poor display effect of the three-dimensional model, especially when the midpoint of the target object is located in the edge area of the image.
By acquiring the depth image corresponding to the RGB image, performing binarization processing and floating point calculations, the edge profile map is determined, and the target map is determined from the RGB image based on this, so that the midpoint of the target object is located in the center area, thereby avoiding the selection of the image of the edge area.
It effectively solves the problem of distortion of the display of three-dimensional models, improves the display effect of the three-dimensional model, and makes it more natural and realistic.
Smart Images
Figure CN120047594A_ABST
Abstract
Description
Background Art
[0002] Texture mapping is an important step in generating a three-dimensional model. Generally, it means that after obtaining the mesh of the three-dimensional model of an object, a series of RGB images are pasted on the mesh.
[0003] Since the same vertices in the mesh may appear in multiple RGB images, various factors need to be considered when selecting a suitable RGB image as the final texture map.
[0004] If the midpoint of the target object in the selected RGB image is located in the edge area, it will cause problems of display distortion after texture mapping of the three-dimensional model, resulting in poor display effects of the three-dimensional model.
[0005] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention
[0006] The present disclosure provides a texture mapping method, apparatus, storage medium, and electronic device, which at least to some extent overcome the problem of poor display effects of three-dimensional models in related technologies.
[0007] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be partially learned through the practice of the present disclosure.
[0008] According to one aspect of the present disclosure, there is provided a texture mapping method, including:
[0009] Obtaining a depth image corresponding to an RGB image; there is a corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image;
[0010] Performing binarization processing on the pixel points of the depth image to determine a grayscale image;
[0011] Performing floating-point operations on the pixel points of the grayscale image to determine an edge contour image;
[0012] Based on the edge contour image, using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine a target texture map in which the midpoint of the target object is located in the central area from the RGB image;
[0013] Pasting the target texture map on the mesh vertices of the three-dimensional model and rendering and displaying the three-dimensional model.
[0014] In some embodiments, performing binarization processing on the pixel points of the depth image to determine a grayscale image includes:
[0015] Obtain the depth value corresponding to each pixel point of the depth image;
[0016] Set the pixel value of the pixel points with depth values greater than zero to one, and set the pixel values of the remaining pixel points to zero to determine the grayscale image.
[0017] In some embodiments, perform floating-point operations on the pixel points of the grayscale image to determine the edge contour map, including:
[0018] Set the number of loops; the number of loops starts from zero to calculate the loop steps;
[0019] For the current loop step plus one to get the first value;
[0020] Determine the pixel points with pixel values of one in the grayscale image as the first pixel points;
[0021] If there are pixel points with pixel values other than one in the neighborhood of the first pixel point, then determine the first pixel point as the second pixel point, and perform floating-point operations on the second pixel point to determine the pixel value of the second pixel point;
[0022] End the current loop step and add one to the next loop step, execute the next loop step until the loop step reaches the set number of loops to determine the edge contour map.
[0023] In some embodiments, perform floating-point operations on the second pixel point to determine the pixel value of the second pixel point, including:
[0024] Determine the ratio of the current loop step to the number of loops as the pixel value of the second pixel point.
[0025] In some embodiments, based on the edge contour map, use the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine the target map of the target object located in the central region from the RGB image, including:
[0026] According to the included angle between the pixel value of each pixel point in the edge contour map and the normal line of the pixel point relative to the viewing point and the gradient of the pixel point, determine the cost value of each pixel point in the edge contour map;
[0027] According to the cost value of each pixel point in the edge contour map, use the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine the target map of the target object located in the central region from the RGB image.
[0028] In some embodiments, the pixel points with a pixel value of one in the edge contour map are inside the target object, the pixel points with a pixel value of zero in the edge contour map are outside the target object, and the pixel points with a pixel value between zero and one in the edge contour map are the edges of the target object; the target object is located in the central region of the RGB image;
[0029] According to the cost value of each pixel point in the edge contour map, using the correspondence between the pixel point coordinates of the RGB image and the pixel point coordinates of the depth image, determining a target map of the target object located in the central region from the RGB image, including:
[0030] Determining that the number of pixel points with a cost value greater than a first threshold in the edge contour map is a first number;
[0031] If the first number is greater than a second threshold, determining that the target object is located in the central region of the edge contour map, and determining the edge contour map as the target edge contour map;
[0032] Using the correspondence between the pixel point coordinates of the RGB image and the pixel point coordinates of the depth image, screening out the RGB image corresponding to the target edge contour map from the RGB image as the target map.
[0033] In some embodiments, pasting the target map on the mesh vertices of the three-dimensional model and rendering and displaying the three-dimensional model, including:
[0034] Obtaining the coordinates of the mesh vertices of the three-dimensional model;
[0035] According to the coordinates of the mesh vertices of the three-dimensional model, pasting the target map on the mesh vertices of the three-dimensional model to determine the three-dimensional model after pasting;
[0036] For the three-dimensional model after pasting Figure 3 Rendering the three-dimensional model to display the three-dimensional view of the three-dimensional model.
[0037] According to another aspect of the present disclosure, there is also provided a mapping device, including:
[0038] A depth image acquisition module for acquiring a depth image corresponding to an RGB image; there is a correspondence between the pixel point coordinates of the RGB image and the pixel point coordinates of the depth image;
[0039] A binarization module for binarizing the pixel points of the depth image to determine a grayscale image;
[0040] A floating-point operation module for performing floating-point operations on the pixel points of the grayscale image to determine an edge contour map;
[0041] A target texture mapping module, configured to determine a target texture from the RGB image according to the edge contour map by using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image;
[0042] A texture rendering module, configured to map the target texture to the mesh vertices of a 3D model and render and display the 3D model.
[0043] According to another aspect of the present disclosure, there is also provided an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein, the processor is configured to execute the texture mapping method described in any one of the above by executing the executable instructions.
[0044] According to another aspect of the present disclosure, there is also provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the texture mapping method described in any one of the above is implemented.
[0045] According to another aspect of the present disclosure, there is also provided a computer program product, including a computer program, and when the computer program is executed by a processor, the texture mapping method described in any one of the above is implemented.
[0046] In the embodiments of the present disclosure, the provided texture mapping method, device, storage medium, and electronic device, the method includes: obtaining a depth image corresponding to an RGB image; there is a corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image; performing binarization processing on the pixel points of the depth image to determine a grayscale image; performing floating-point operations on the pixel points of the grayscale image to determine an edge contour map; based on the edge contour map, determining a target texture of the target object whose midpoint is located in the central region from the RGB image by using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image; mapping the target texture to the mesh vertices of a 3D model and rendering and displaying the 3D model. In the embodiments of the present disclosure, based on the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, a mapping relationship is established between the RGB image and the depth image, the depth image is binarized to obtain a grayscale image, and floating-point operations are performed on the grayscale image to obtain an edge contour map. The edge contour of the target object is described by the edge contour map, and then the target texture is determined from the RGB image by using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image. The center point of the target object in the target texture is located in the central region, so as to eliminate the RBG image whose center point of the target object is located in the edge region, map the target texture to the mesh vertices of the 3D model and then render and display the 3D model, thereby ensuring that the rendered and displayed 3D model is closer to the real physical world, making the textured 3D model more natural, and improving the display effect of the 3D model.
[0047] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] The accompanying drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.
[0049] Figure 1 A schematic diagram showing the system structure of a texture mapping method in an embodiment of the present disclosure.
[0050] Figure 2 A schematic diagram showing a texture mapping method in an embodiment of the present disclosure.
[0051] Figure 3 Schematic diagrams showing two different types of RGB images of a texture mapping method in an embodiment of the present disclosure.
[0052] Figure 4 A schematic diagram showing an edge contour map of a texture mapping method in an embodiment of the present disclosure.
[0053] Figure 5 A schematic diagram showing a texture mapping device in an embodiment of the present disclosure.
[0054] Figure 6 A block diagram showing the structure of a computer device for a texture mapping method in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described can be combined in any suitable manner in one or more embodiments.
[0056] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0057] The following will, with reference to the accompanying drawings, elaborate in detail on the specific implementation manners of the embodiments of the present disclosure.
[0058] Figure 1 Fig. shows an exemplary application system architecture diagram to which the texture mapping method in the embodiments of the present disclosure can be applied. As Figure 1 shown, the system architecture may include a terminal device 101, a network 102, and a server 103.
[0059] The network 102 is used to provide a medium for the communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.
[0060] Optionally, the above-mentioned wireless network or wired network uses standard communication technologies and / or protocols. The network is usually the Internet, but can also be any network, including but not limited to any combination of a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a private network, or a virtual private network. In some embodiments, technologies and / or formats including hypertext markup language (HTML), extensible markup language (XML), etc. are used to represent the data exchanged through the network. In addition, conventional encryption technologies such as secure socket layer (SSL), transport layer security (TLS), virtual private network (VPN), and Internet protocol security (IPsec) can be used to encrypt all or some of the links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.
[0061] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, wearable devices, augmented reality devices, virtual reality devices, etc.
[0062] Optionally, the clients of the application programs installed in different terminal devices 101 are the same, or the clients of the same type of application programs based on different operating systems. Depending on the different terminal platforms, the specific form of the client of the application program can also be different. For example, the client of the application program can be a mobile client, a PC client, etc.
[0063] Server 103 can be a server that provides various services. For example, it can be a background management server that supports the devices operated by users using terminal device 101. The background management server can analyze and process data such as received requests, and feedback the processing results to the terminal device.
[0064] Optionally, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication methods, and this application does not make any restrictions here.
[0065] Those skilled in the art can know that Figure 1 the numbers of the terminal devices, networks, and servers in are only illustrative. According to actual needs, there can be any number of terminal devices, networks, and servers. The embodiments of the present disclosure do not make any limitations in this regard.
[0066] Under the above system architecture, an embodiment of the present disclosure provides a sticker pasting method, which can be executed by any electronic device with computing and processing capabilities.
[0067] In some embodiments, the sticker pasting method provided in the embodiments of the present disclosure can be executed by the terminal device of the above system architecture; in some other embodiments, the sticker pasting method provided in the embodiments of the present disclosure can be executed by the server in the above system architecture; in some other embodiments, the sticker pasting method provided in the embodiments of the present disclosure can be implemented by the terminal device and the server in the above system architecture through interaction.
[0068] Figure 2 Shows a schematic diagram of a sticker pasting method in an embodiment of the present disclosure. As Figure 2 shown, the sticker pasting method provided in the embodiments of the present disclosure includes the following steps:
[0069] Step S202: Obtain a depth image corresponding to the RGB image; there is a corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image;
[0070] Step S204: Binarize the pixel points of the depth image to determine a grayscale image;
[0071] Step S206: Perform floating-point operations on the pixel points of the grayscale image to determine the edge contour map;
[0072] Step S208: Based on the edge contour map, use the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine the target map in the RGB image where the midpoint of the target object is located in the central region;
[0073] Step S2010: Paste the target map on the mesh vertices of the 3D model and render to display the 3D model.
[0074] Based on the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, the embodiments of the present disclosure establish a mapping relationship between the RGB image and the depth image. The depth image is binarized to obtain a grayscale image, and floating-point operations are performed on the grayscale image to obtain an edge contour map. The edge contour of the target object is described by the edge contour map. Then, using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, the target map is determined from the RGB image. The center point of the target object in the target map is located in the central region, so as to eliminate the RBG image where the center point of the target object is located in the edge region. After pasting the target map on the mesh vertices of the 3D model, the 3D model is rendered and displayed, thus ensuring that the rendered and displayed 3D model is closer to the real physical world, making the textured 3D model more natural and improving the display effect of the 3D model.
[0075] The RGB image is a common form of digital image. It represents colors through three color channels: red (R), green (G), and blue (B). The RGB image is composed of multiple pixel points. Each pixel point includes three channels: the red channel, the green channel, and the blue channel. Each channel can have 256 different values (from 0 to 255). For example, light blue can be represented as (173, 216, 230), where the red (R) is 173, the green (G) is 216, and the blue (B) is 230. In the RGB image, the pixel point also has pixel coordinates, usually represented as (x1, y2), where x1 is the abscissa and y1 is the ordinate. In the RGB image, each pixel point is represented by the coordinate and the RGB channel values together.
[0076] A depth map is an image that represents the distance from each pixel point in a scene to an observation point (usually a camera). In the fields of computer vision and graphics, depth maps are widely used in areas such as 3D reconstruction, augmented reality, virtual reality, robot navigation, and autonomous driving. In a depth map, each pixel has a depth value, which is the most core information in the depth map and represents the distance from the camera optical center to the surface of the object at the corresponding pixel position in the scene. The depth value is usually a floating-point number or an integer value. A depth map can be a grayscale image, where brighter areas represent closer objects and darker areas represent farther objects; it can also be a pseudo-color image, using different colors to represent different distance ranges to help understand depth information more easily. Each pixel point in the depth map also has pixel coordinates, usually represented as (x2, y2), where x2 is the abscissa and y2 is the ordinate. In a depth image, each pixel point is represented by its coordinates and depth value together.
[0077] When performing 3D modeling, RGB images and depth images can be acquired simultaneously by the same device. For example, RGB images and depth images can be acquired simultaneously through a binocular camera system, and the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image can be output synchronously. Further, RGB images and depth images can also be acquired using different devices. The RGB image is acquired using a camera, and the depth image is acquired using a depth camera. Then, the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image is calculated based on the positional relationship between the camera and the depth camera.
[0078] After obtaining the 3D Mesh grid of an object, texture mapping pastes a series of RGB images onto the Mesh grid. Eventually, each vertex in the Mesh will be assigned an RGB image and texture coordinates of its position in the image. Through some optimization algorithms, each vertex is assigned an RGB image, and at the same time, the 2D position of the vertex in the RGB image is calculated. Since the same vertex in the grid may appear in multiple RGB images, various factors need to be considered when choosing a suitable RGB image as the final texture map. One relatively intuitive rule is that the midpoint of the selected RGB image should not be on the edge of the object contour. Figure 3Two schematic diagrams of different types of RGB images are shown, and the midpoint of the image is set as the nose of the human face, i.e., the shaded area. In the left image, the shaded area is located in the center of the human face and belongs to the central area; while in the right image, the shaded area is located on one side of the human face and belongs to the edge area. If we want the 3D model to be more natural after texture mapping, we need to select the left image as the texture. By combining the depth map corresponding to RGB in the present disclosure, an edge contour map is generated, and the optimization algorithm preferentially selects a non-edge RGB image as the texture based on this edge contour, making the model with the texture more natural.
[0079] In the embodiment, the pixel points of the depth image are binarized to determine a grayscale image, including:
[0080] Obtain the depth value corresponding to each pixel point of the depth image;
[0081] Set the pixel value of the pixel points with depth values greater than zero to one, and set the pixel values of the remaining pixel points to zero to determine the grayscale image.
[0082] Each pixel point of the depth image has a depth value. The depth value corresponding to each pixel point of the depth image is respectively obtained and binarized; specifically, the pixel value of the pixel points with depth values greater than zero is set to 1, and the pixel values of the remaining pixel points are set to zero, obtaining a grayscale image represented by 0 and 1 for pixel points.
[0083] In the embodiment, floating-point operations are performed on the pixel points of the grayscale image to determine an edge contour map, including:
[0084] Set the number of loops; the number of loops starts from zero to calculate the loop steps;
[0085] For the current loop step plus one to obtain a first value;
[0086] Determine the pixel points with pixel values of one in the grayscale image as the first pixel points;
[0087] If there are pixel points with pixel values other than one in the neighborhood of the first pixel point, then determine the first pixel point as the second pixel point, and perform floating-point operations on the second pixel point to determine the pixel value of the second pixel point;
[0088] End the current loop step and add one to the next loop step, and execute the next loop step until the loop step reaches the set number of loops to determine the edge contour map.
[0089] After obtaining the grayscale image, set the number of loops to N times. The number of loops starts from i = 0 and each time the following loop steps are executed:
[0090] (1) Set i = i + 1, where i is the current loop step;
[0091] (2) Determine the pixel points with a pixel value of 1 in the grayscale image as the first pixel points;
[0092] (3) If there are pixel points with non - 1 pixel values in the neighborhood of the first pixel points, then determine the first pixel points as the second pixel points;
[0093] (4) Perform floating - point operations on the second pixel points to determine the pixel values of the second pixel points; specifically, the floating - point operation is to determine the ratio i / N of the current loop step to the number of loop times as the pixel value of the second pixel points.
[0094] Loop and execute the above (1)-(4) until the set number of loop times is reached to determine the edge contour map.
[0095] Furthermore, the above steps can be simplified to the following process:
[0096] Set the number of loop times N, and the loop step i = 0, and loop according to the following steps:
[0097] a) The current step is i = i + 1;
[0098] b) For all points with a value of 1 in the picture, if there are non - 1 pixel points in its neighborhood, then the pixel value of this pixel point is i / N;
[0099] By continuously looping the above a) and b), the edge contour map is finally obtained. The edge contour map belongs to the grayscale image. Among them, the neighborhood of a pixel point refers to the area composed of adjacent pixel points. The neighborhood of a pixel point consists of a four - neighborhood and an eight - neighborhood. Among them, the four - neighborhood refers to the area composed of the four adjacent pixel points above, below, left, and right of the pixel point. The eight - neighborhood refers to the area composed of the eight adjacent pixel points above, below, left, right, upper - left, lower - left, upper - right, and lower - right of the pixel point.
[0100] In the embodiment, based on the edge contour map, using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine the target map of the target object located in the central area from the RGB image, including:
[0101] According to the included angle between the pixel value of each pixel point in the edge contour map and the normal line of the pixel point relative to the view point and the gradient of the pixel point, determine the cost value of each pixel point in the edge contour map;
[0102] According to the cost value of each pixel point in the edge contour map, use the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine the target map of the target object located in the central area from the RGB image.
[0103] After obtaining the edge contour map, for different texture mapping algorithms, the obtained edge contour map can be incorporated into the texture mapping algorithm to optimize the algorithm, so that the texture mapping algorithm avoids selecting an image with the center located at the edge. Specifically, the cost value of each pixel point in the edge contour map can be determined according to the angle between the pixel value of each pixel point in the edge contour map and the normal of the pixel point relative to the viewing point and the gradient of the pixel point; the cost value of each pixel point is used to evaluate whether the pixel point belongs to the central region. According to the cost value of each pixel point in the edge contour map, the target map of the target object located in the central region is determined from the RGB image by using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image.
[0104] In the embodiment, the pixel points with a pixel value of one in the edge contour map are inside the target object, the pixel points with a pixel value of zero in the edge contour map are outside the target object, and the pixel points with a pixel value between zero and one in the edge contour map are the edges of the target object; the target object is located in the central region of the RGB image;
[0105] Determining the target map of the target object located in the central region from the RGB image according to the cost value of each pixel point in the edge contour map by using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image includes:
[0106] Determine that the number of pixel points with a cost value greater than the first threshold in the edge contour map is the first number;
[0107] If the first number is greater than the second threshold, it is determined that the target object is located in the central region of the edge contour map, and the edge contour map is determined as the target edge contour map;
[0108] Using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, the RGB image corresponding to the target edge contour map is screened out from the RGB image as the target map.
[0109] In the above-obtained edge contour map, the pixel points with a pixel value of 1 are the inside of the target object, the pixel points with a pixel value of 0 are the outside of the target object, the pixel points with a pixel value between 0 and 1 are the edges of the target object, the pixel points with a pixel value closer to 0 are closer to the edge of the target object, and the pixel points with a pixel value closer to 1 are closer to the inside of the target object.
[0110] Figure 4 A schematic diagram of an edge contour map is shown, and the number of cycles is N = 5 times. Among them, as 1 / 5, 2 / 5, 3 / 5, and 4 / 5 gradually approach the inside of the target object from the edge of the target object, where 1 / 5 is the area close to the edge of the target object and 4 / 5 is the area close to the inside of the target object. The target object is located in the central region of the RGB image.
[0111] For different texture mapping algorithms, the benchmark of the cost value is different. Taking one case as an example, the greater the cost value, the greater the probability that the image is selected. First, it is necessary to count and determine that the number of pixel points in the edge contour map with a cost value greater than the first threshold is the first number; further, judge the size relationship between the first number and the second threshold. If the first number is greater than the second threshold, it is determined that the target object is located in the central region of the edge contour map, and the edge contour map is determined as the target edge contour map; if the first number is less than the second threshold, it is determined that the target object is located in the edge region of the edge contour map, discard this edge contour map, and re - search for a new RGB image as the texture map; utilize the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, and screen out the RGB image corresponding to the target edge contour map from the RGB image as the target texture map.
[0112] For the obtained target texture map, since the pixel values of the pixel points in the edge region are < 1, the corresponding cost value is lower than the normal value. Thus, the RGB images containing edge points can be excluded, and the RGB images with the target object located in the central region can be retained. As a result, the RGB photos containing edge points are not selected, so the effect of texture mapping can be improved.
[0113] In the embodiment, pasting the target texture map on the mesh vertices of the 3D model and rendering to display the 3D model includes:
[0114] Obtain the coordinates of the mesh vertices of the 3D model;
[0115] According to the coordinates of the mesh vertices of the 3D model, paste the target texture map on the mesh vertices of the 3D model to determine the 3D model after pasting the texture;
[0116] Render the 3D model after pasting... Figure 3 ... and display the 3D view of the 3D model.
[0117] In the embodiment, the 3D model is composed of multiple meshes, and each mesh has mesh vertices. To obtain the coordinates of the mesh vertices of the 3D model, further, the target texture map also has UV coordinates. The UV coordinates are used to specify which part of the 2D texture image should be mapped to which area of the 3D Mesh. Each vertex usually has a corresponding pair of UV coordinates (u, v), where both u and v are floating - point numbers between 0 and 1. Utilize the UV coordinates of the target texture map and the coordinates of the mesh vertices of the 3D model, paste the target texture map on the mesh vertices of the 3D model to determine the 3D model after pasting the texture, and render the 3D model after pasting... Figure 3 ... and display the 3D view of the 3D model.
[0118] It should be noted that in the technical solution of the present disclosure, the acquisition, storage, use, processing, etc. of data all comply with the relevant provisions of national laws and regulations. For various types of data such as personal identity data, operation data, and behavior data related to individuals, customers, and populations obtained in the embodiments of the present disclosure, authorization has been obtained.
[0119] Based on the same inventive concept, an embodiment of the present disclosure also provides a texture mapping device as described in the following embodiments. Since the principle of solving problems in this device embodiment is similar to that of the above method embodiment, the implementation of this device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be elaborated.
[0120] Figure 5 The following shows a schematic diagram of a texture mapping device in an embodiment of the present disclosure, as Figure 5 shown, the device includes:
[0121] A depth image acquisition module 501, configured to acquire a depth image corresponding to an RGB image; there is a corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image;
[0122] A binarization module 502, configured to perform binarization processing on the pixel points of the depth image to determine a grayscale image;
[0123] A floating-point operation module 503, configured to perform floating-point operations on the pixel points of the grayscale image to determine an edge contour image;
[0124] A target texture mapping module 504, configured to determine a target texture mapping from the RGB image according to the edge contour image by using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image;
[0125] A texture mapping rendering module 505, configured to paste the target texture mapping on the mesh vertices of a three-dimensional model and render and display the three-dimensional model.
[0126] It should be noted here that the above depth image acquisition module 501, binarization module 502, floating-point operation module 503, target texture mapping module 504, and texture mapping rendering module 505 correspond to S202 - S2010 in the method embodiment. The examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the content disclosed in the above method embodiment. It should be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.
[0127] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to herein as "circuitry", "module", or "system".
[0128] Reference is now made to Figure 6 to describe the electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The illustrated electronic device 600 is merely an example and should not impose any limitation on the functions and the scope of use of the embodiments of the present disclosure.
[0129] As Figure 6 shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one of the above-mentioned processing units 610, at least one of the above-mentioned storage units 620, and a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610).
[0130] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification. For example, the processing unit 610 may execute the following steps of the above method embodiment: obtaining a depth image corresponding to an RGB image; there is a corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image; performing binarization processing on the pixel points of the depth image to determine a grayscale image; performing floating-point operations on the pixel points of the grayscale image to determine an edge contour image; based on the edge contour image, using the corresponding relationship between the pixel coordinates of the RGB image and the pixel coordinates of the depth image to determine a target map in which the midpoint of the target object is located in the central region from the RGB image; pasting the target map on the mesh vertices of the 3D model and rendering and displaying the 3D model.
[0131] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only storage unit (ROM) 6203.
[0132] The storage unit 620 may further include a program / utilities 6204 having a set (at least one) of program modules 6205. Such program modules 6205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.
[0133] The bus 630 can represent one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, an Accelerated Graphics Port, a processor, or a local bus using any of the various bus architectures.
[0134] The electronic device 600 can also communicate with one or more external devices 640 (such as a keyboard, a pointing device, a Bluetooth device, etc.), and can also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device 600 to communicate with one or more other computing devices. Such communication can be carried out through the input / output (I / O) interface 650. Moreover, the electronic device 600 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 660. As shown in the figure, the network adapter 660 communicates with other modules of the electronic device 600 through the bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0135] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or can be implemented by the way of software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, and the software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
[0136] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, and the computer program product includes: a computer program, and when the computer program is executed by a processor, the above-mentioned mapping method is implemented.
[0137] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, which may be a readable signal medium or a readable storage medium. Stored thereon is a program product capable of implementing the above-described method of the present disclosure. In some possible implementation manners, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.
[0138] More specific examples of the computer-readable storage medium in the present disclosure may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0139] In the present disclosure, the computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
[0140] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.
[0141] In specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as an independent software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, by using an Internet service provider to connect through the Internet).
[0142] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.
[0143] In addition, although the steps of the methods in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in that specific order, or that all the steps shown must be executed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution, etc.
[0144] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the methods according to the embodiments of the present disclosure.
[0145] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and the embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the appended claims.
Claims
1. A mapping method, characterized in that: include: Get the depth image corresponding to the RGB image; The pixel coordinates of the RGB image and the pixel coordinates of the depth image have a corresponding relationship; Binarize the pixels of the depth image to determine a grayscale image; Performing floating point operations on the pixels of the grayscale image to determine an edge contour image; Based on the edge contour map, a target map in which the midpoint of the target object is located in the center area is determined from the RGB image using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image; The target map is pasted on the mesh vertices of the three-dimensional model, and the three-dimensional model is rendered and displayed.
2. The mapping method according to claim 1, characterized in that: Binarizing the pixels of the depth image to determine a grayscale image includes: Obtaining a depth value corresponding to each pixel of the depth image; The pixel values of the pixels whose depth values are greater than zero are set to one, and the pixel values of the remaining pixels are set to zero to determine the grayscale image.
3. The mapping method according to claim 1, characterized in that: Performing floating point operations on the pixels of the grayscale image to determine an edge contour image includes: Setting the number of cycles; the number of cycles is calculated from zero to count the cycle steps; Add one to the current loop step to obtain the first value; Determine a pixel point having a pixel value of 1 in the grayscale image as a first pixel point; If there is a pixel point with a non-one pixel value in the neighborhood of the first pixel point, the first pixel point is determined as the second pixel point, and a floating-point operation is performed on the second pixel point to determine the pixel value of the second pixel point; End the current loop step and add one to the next loop step, execute the next loop step until the loop step reaches the set number of loops, and determine the edge contour map.
4. The mapping method according to claim 3, characterized in that: Performing a floating-point operation on the second pixel to determine a pixel value of the second pixel includes: The ratio of the current cycle step to the cycle number is determined as the pixel value of the second pixel point.
5. The mapping method according to claim 1, characterized in that: Based on the edge contour map, a target map in which the target object is located in the center area is determined from the RGB image using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, including: Determine a cost value of each pixel in the edge contour map according to the angle between the pixel value of each pixel and the normal of the pixel relative to the viewpoint and the gradient of the pixel; According to the cost value of each pixel in the edge contour image, a target map in which the target object is located in the central area is determined from the RGB image using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image.
6. The mapping method according to claim 5, characterized in that: The pixel points with a pixel value of one in the edge contour map are inside the target object, the pixel points with a pixel value of zero in the edge contour map are outside the target object, and the pixel points with a pixel value between zero and one in the edge contour map are edges of the target object; The target object is located in the central area of the RGB image; According to the cost value of each pixel in the edge contour image, a target map where the target object is located in the center area is determined from the RGB image using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, including: Determine the number of pixel points in the edge contour map whose cost value is greater than a first threshold as a first number; If the first number is greater than a second threshold, determining that the target object is located in the central area of the edge contour map, and determining the edge contour map as a target edge contour map; By using the correspondence between the pixel coordinates of the RGB image and the pixel coordinates of the depth image, an RGB image corresponding to the target edge contour image is screened out from the RGB image as a target map.
7. The mapping method according to claim 1, characterized in that: Pasting the target texture on the mesh vertices of the three-dimensional model and rendering and displaying the three-dimensional model, including: Get the coordinates of the mesh vertices of the 3D model; According to the coordinates of the mesh vertices of the three-dimensional model, the target map is pasted on the mesh vertices of the three-dimensional model to determine the three-dimensional model after the mapping; The mapped three-dimensional model is rendered to display a three-dimensional view of the three-dimensional model.
8. A mapping device, characterized in that: include: A depth image acquisition module is used to obtain a depth image corresponding to an RGB image; The pixel coordinates of the RGB image and the pixel coordinates of the depth image have a corresponding relationship; A binarization module, used to perform binarization processing on the pixels of the depth image to determine a grayscale image; A floating point operation module, used for performing floating point operations on the pixels of the grayscale image to determine an edge contour image; A target mapping module, configured to determine a target mapping from the RGB image according to the edge contour map by using a correspondence between pixel coordinates of the RGB image and pixel coordinates of the depth image; The texture rendering module is used to paste the target texture on the mesh vertices of the three-dimensional model and render and display the three-dimensional model.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the mapping method according to any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the mapping method according to any one of claims 1 to 7 is implemented.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the mapping method according to any one of claims 1 to 7 is implemented.