Method and device for creating three-dimensional model of object and electronic equipment

Abstract

The invention discloses a method and a device for creating a three-dimensional model of an object, electronic equipment and a computer readable storage medium. The method for creating the three-dimensional model of the object comprises the following steps: acquiring a plurality of target images corresponding to a target object of a to-be-created three-dimensional model; determining a plurality of groups of image pairs formed by the plurality of target images; based on a pre-trained image processing model, determining a pixel corresponding relation between two images in the image pair and initial point cloud information corresponding to each target image, wherein the image pair comprises two images in the plurality of target images; determining fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and a pixel corresponding relation; and creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each target image. Through the scheme provided by the invention, the three-dimensional model of the object can be efficiently and quickly created according to the image of the object, and the accuracy of the created three-dimensional model is higher.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a method, device, electronic device and computer-readable storage medium for creating a three-dimensional model of an object. Background Art

[0002] With the rapid development of computer vision, 3D modeling of objects has been widely used in many fields, such as e-commerce fields such as product image generation, product display, and consumer personalized matching, as well as virtual reality (VR), game development, cultural heritage protection, industrial design and other fields. Among them, creating a corresponding 3D model of an object based on a photo of the object is currently a more commonly used 3D modeling method. It can create a corresponding 3D model of the object according to the appearance of the object shown in the photo, and apply the created 3D model to the 3D display of the object, so that users can observe and interact from different angles, greatly enriching the user experience and improving work efficiency.

[0003] Traditional 3D modeling technology usually requires users to create a matching 3D model of an object based on an object image using 3D modeling software. However, this 3D modeling method requires users to have high professional skills in 3D modeling, and the modeling process is often time-consuming. In addition, due to the large differences in the technical levels of modelers, the accuracy of the created 3D model of the object is also poor. Summary of the invention

[0004] The present application provides a method, device, electronic device and computer-readable storage medium for creating a three-dimensional model of an object, which can efficiently and quickly create a three-dimensional model of an object based on an image of the object, without requiring the user to have high professional skills in three-dimensional modeling, and the created three-dimensional model has higher accuracy. The specific scheme is as follows: In a first aspect, the present application provides a method for creating a three-dimensional model of an object, the method comprising: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; Determine a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one image in the image pair; Determining fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; A three-dimensional model corresponding to the target object is created according to the fine point cloud information corresponding to each of the target images.

[0005] Optionally, the determining, based on the initial point cloud information of the target image and the pixel correspondence, the fine point cloud information corresponding to the target image includes: estimating initial camera intrinsic parameters corresponding to the target image based on initial point cloud information of the target image; Based on the pixel correspondence, the initial point cloud information corresponding to the target image and the initial camera intrinsic parameters, the fine point cloud information corresponding to the target image is determined.

[0006] Optionally, estimating an initial camera intrinsic parameter corresponding to the target image based on the initial point cloud information of the target image includes: Based on the principle of minimizing the gap between the converted pixel position and the corresponding actual pixel position, the initial camera intrinsic parameters corresponding to the target image are estimated. The converted pixel position is the corresponding pixel position obtained by converting the initial point cloud information corresponding to each pixel point of the target image according to the estimated initial camera intrinsic parameters. The actual pixel position is the corresponding actual pixel position in the target image.

[0007] Optionally, the initial point cloud information corresponding to the pixel points of the target image includes x Dimensional initial point cloud information, y Dimensional initial point cloud information and z Dimensional initial point cloud information; The converted pixel position is obtained as follows: The target image x Dimensional initial point cloud information and y The initial point cloud information of dimensions is z Normalization processing is performed on the initial point cloud information of the dimension to obtain normalized initial point cloud information corresponding to each pixel point of the target image; The normalized initial point cloud information is converted into a conversion pixel position according to the estimated initial camera intrinsic parameters.

[0008] Optionally, determining the fine point cloud information corresponding to the target image based on the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameter includes: estimating the initial camera extrinsic parameters corresponding to the target image based on the pixel correspondence relationship and the initial point cloud information corresponding to the target image; Based on the initial camera extrinsic parameters, the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters, the fine point cloud information corresponding to the target image is determined.

[0009] Optionally, the estimating the initial camera extrinsic parameters corresponding to the target image based on the pixel correspondence relationship and the initial point cloud information corresponding to the target image includes: Based on the principle of minimizing the gap between matching world point cloud information, the initial camera extrinsic parameters corresponding to the target image are estimated. The matching world point cloud information is the point cloud information of the pixel points corresponding to the two images in the image pair after being converted to the world coordinate system according to the estimated initial camera extrinsic parameters.

[0010] Optionally, the estimating the initial camera extrinsic parameters corresponding to the target image based on the principle of minimizing the gap between the matched world point cloud information includes: Determine a weight corresponding to each pixel in the target image, wherein the weight is used to indicate the importance of the pixel for displaying the target object, and the weights of the corresponding pixels in the two images in the image pair are the same; Determine the product of the gap between the matched world point cloud information and the corresponding weight as the first weighted gap of each pair of matched pixel points; The initial camera extrinsic parameters corresponding to the target image are estimated based on the principle of minimizing the sum of the first weighted differences corresponding to the image pair.

[0011] Optionally, determining the fine point cloud information corresponding to the target image based on the initial camera extrinsic parameters, the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters includes: Determining fine point cloud information corresponding to the target image based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point, wherein the matched pixel point is the matched pixel point in the other image of the image pair; The converted pixel position corresponding to the matched pixel point is determined by: Converting the initial point cloud information corresponding to the matched pixel points into world point cloud information based on the initial camera extrinsic parameters, the pixel correspondence relationship and the initial camera intrinsic parameters; The world point cloud information corresponding to the matched pixel point is converted to a two-dimensional image position to obtain a converted pixel position corresponding to the matched pixel point.

[0012] Optionally, the determining of the fine point cloud information corresponding to the target image based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point includes: Determine a weight corresponding to each pixel in the target image, wherein the weight is used to indicate the importance of the pixel for displaying the target object, and the weights of the corresponding pixels in the two images in the image pair are the same; Determine the product of the difference between the actual pixel position of a pixel point of one image in the image pair and the transformed pixel position corresponding to the matched pixel point and the corresponding weight as the second weighted difference for each pair of matched pixels; The fine point cloud information corresponding to the target image is determined based on the principle of minimizing the sum of the second weighted differences corresponding to the image pair.

[0013] Optionally, the camera external parameters include at least one of camera posture and scaling factor, and the camera internal parameters include camera focal length.

[0014] Optionally, determining a plurality of groups of image pairs consisting of the plurality of target images comprises: According to the similarities between the target images, the multiple target images are grouped into multiple groups of image pairs.

[0015] Optionally, the step of grouping the plurality of target images into a plurality of image pairs according to the similarities between the target images comprises: Extracting image coding features corresponding to each of the target images based on a pre-trained image coding model; Calculating the similarity between each target image and other target images according to the image coding features; The multiple target images are grouped into multiple groups of image pairs according to the similarity between each target image and other target images.

[0016] Optionally, grouping the plurality of target images into a plurality of image pairs according to the similarity between each target image and other target images comprises: For each target image, images among other target images whose similarity to the target image is ranked before a preset number are respectively formed into image pairs with the target image, or images among other target images whose similarity to the target image is greater than a preset similarity threshold are respectively formed into image pairs with the target image.

[0017] Optionally, grouping the plurality of target images into a plurality of image pairs according to the similarity between each target image and other target images comprises: Based on the principle that the total number of image pairs does not exceed a second preset number, for each target image, other target images that rank before a preset number of similarities with the target image are respectively formed into image pairs.

[0018] Optionally, the determining the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images based on the pre-trained image processing model includes: The image coding features corresponding to each of the target images and the information of the two images constituting the image pair are input into a pre-trained image processing model to obtain the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images.

[0019] Optionally, the determining of the fine point cloud information corresponding to the target image based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point includes: Based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point, determining the fine point cloud information corresponding to the target image and the precise camera intrinsic parameters corresponding to the target image; The method further comprises: Determine a global normal vector corresponding to the target image according to the precise camera intrinsic parameters and the fine point cloud information corresponding to the target image; Determining an accurate normal vector corresponding to the target image according to a global normal vector corresponding to the target image; The step of creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images comprises: A three-dimensional model corresponding to the target object is created according to the fine point cloud information corresponding to each of the target images and the accurate normal vector corresponding to each of the target images.

[0020] Optionally, the pre-trained image processing model is further used to determine a predicted normal vector corresponding to each of the image pairs during the process of determining the pixel correspondence relationship and the initial point cloud information; The determining, according to the global normal vector corresponding to the target image, the accurate normal vector corresponding to the target image comprises: According to the global normal vector corresponding to the target image and the corresponding predicted normal vectors, the accurate normal vector corresponding to the target image is determined.

[0021] Optionally, determining the accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image and the corresponding predicted normal vectors includes: Calculating the error between each predicted normal vector corresponding to the target image and the corresponding global normal vector; Eliminate pixels whose errors with the global normal vector are greater than a preset error from among the predicted normal vectors corresponding to the target image, and obtain excluded predicted normal vectors corresponding to the target image after excluding the pixels; The accurate normal vector corresponding to the target image is determined according to the mean value of each excluded predicted normal vector corresponding to the target image.

[0022] Optionally, the method further comprises: When a pixel point of the target image is excluded in each of the excluded predicted normal vectors, the normal vector of the pixel point in the global normal vector corresponding to the target image is determined as the accurate normal vector corresponding to the pixel point.

[0023] Optionally, creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images includes: The dense point cloud information corresponding to the target object is determined according to the fine point cloud information corresponding to each of the target images, and a three-dimensional model corresponding to the target object is created according to the dense point cloud information.

[0024] In a second aspect, the present application also provides a method for creating a three-dimensional model of an object, the method comprising: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; Determine a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Determine, based on a pre-trained image processing model, a pixel correspondence relationship between two images in the image pair, initial point cloud information corresponding to each of the target images, and a predicted normal vector corresponding to each of the image pairs, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one of the image pairs; Based on the initial point cloud information of the target image and the pixel correspondence, determining the fine point cloud information corresponding to the target image and the precise camera intrinsic parameters corresponding to the target image; Determine a global normal vector corresponding to the target image according to the precise camera intrinsic parameters and the fine point cloud information corresponding to the target image; Determine the accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image and the corresponding predicted normal vectors; A three-dimensional model corresponding to the target object is created according to the fine point cloud information corresponding to each of the target images and the accurate normal vector corresponding to each of the target images.

[0025] In a third aspect, the present application also provides a method for creating a three-dimensional model of an object, which is applied to a server, and the method includes: Acquire multiple target images sent by the client for displaying the target object, wherein the multiple target images are used to display the target object from multiple viewing angles; Creating a three-dimensional model corresponding to the target object by the method for creating a three-dimensional model of an object according to any one of the first aspects; The three-dimensional model is sent to a client so that the client displays the three-dimensional model.

[0026] In a fourth aspect, the present application further provides a device for creating a three-dimensional model of an object, the device comprising: An acquisition unit, used for acquiring a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used for displaying the target object from a plurality of viewing angles; A pairing unit, used for determining a plurality of image pairs consisting of the plurality of target images, each of the image pairs comprising two different target images; A determination unit, configured to determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position corresponding to an object part in another image corresponding to a pixel position in one image of the image pair; and determine fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; A modeling unit is used to create a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images.

[0027] In a fifth aspect, the present application also provides an electronic device comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor; when the processor executes the computer program instructions, it implements the method described in any one of the first to third aspects.

[0028] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement any one of the methods in the first to third aspects.

[0029] In a seventh aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method as described in any one of the first to third aspects.

[0030] Compared with the prior art, this application has the following advantages: The method for creating a three-dimensional model of an object provided in an embodiment of the present application obtains multiple target images corresponding to the target object of the three-dimensional model to be created, and the multiple target images are used to display the target object from multiple perspectives, and then based on a pre-trained image processing model, the pixel correspondence between two images in an image pair corresponding to the multiple target images and the initial point cloud information corresponding to each of the target images are determined, the image pair includes two images of the multiple target images, and the two images in a group of image pairs can display various parts of the object from different perspectives, so that each group of image pairs can better accurately reflect the information of various parts of the object from the perspective of stereo geometry, the pixel correspondence is used to indicate the pixel position of the object part corresponding to the pixel position of one image in the image pair in the other image, that is, the pixel correspondence can associate and correspond the pixel points belonging to the same object part in the two images, and the pixel correspondence can well reflect the position information of the same object part at different perspectives, so that the pixel correspondence can well reflect the stereoscopic geometric information of the object, and the above-mentioned image processing model is pre-trained, and the image processing model can determine the pixel correspondence between the two images in each group of the image pairs. The initial point cloud information corresponding to the target image and the pixel correspondence relationship can be quickly determined through the pre-trained image processing model, wherein the initial point cloud information reflects the depth image corresponding to each target image, and the initial point cloud information can reflect the geometric information of the object corresponding to a single image. Then, based on the initial point cloud information of the target image and the pixel correspondence relationship, the fine point cloud information corresponding to the target image is determined. Since the pixel correspondence relationship can well reflect the position information of the same object part at different viewing angles, the initial point cloud information is refined through the pixel correspondence relationship, so that the refined point cloud information after the refinement can have higher accuracy in the three-dimensional space, that is, the fine point cloud information can more accurately represent the various parts of the target object in space, and then the three-dimensional model corresponding to the target object is determined to be created according to the fine point cloud information corresponding to each of the target images. Since the fine point cloud information can more accurately represent the various parts of the target object in space, the three-dimensional model corresponding to the target object created according to the fine point cloud information corresponding to each of the target images is more accurate in the three-dimensional space and more matched with the actual target object.

[0031] The solution provided by the present application firstly determines the initial point cloud information corresponding to each target image quickly and efficiently through an image processing model. Since the initial point cloud information is obtained based on a single target image, the initial point cloud information may lack an accurate reflection of the target object in three-dimensional space. The present application then adjusts the initial point cloud information through the pixel correspondence corresponding to each group of images that can well reflect the three-dimensional geometric information of the object, and obtains fine point cloud information that can more accurately represent various parts of the target object in space, so that the three-dimensional model created according to the fine point cloud information corresponding to each of the target images is more matched with the actual target object and has higher accuracy.

[0032] It can be seen that the solution provided by the present application can automatically generate a three-dimensional model corresponding to the target object through a computer, without the need for manual operation of the three-dimensional modeling software. Therefore, it is possible to efficiently and quickly create a three-dimensional model of an object based on the image of the object, without the need for the user to have high professional skills in three-dimensional modeling, and the created three-dimensional model is more accurate and more consistent with the target object. In addition, since the image processing model is trained through a large number of training samples, the training process usually includes both good quality images and poor quality images. Therefore, the trained image processing model has a strong prior, which makes the number of pixel matches predicted by the image processing model larger, and can also achieve good prediction and processing for poor image quality such as low texture, blur, low resolution, and poor lighting conditions, and obtain an accurate three-dimensional model. The initial point cloud information of each target image predicted by the pre-trained image processing model itself also has a relatively high accuracy, can contain rich geometric information, and creates conditions for obtaining more accurate fine point cloud information in the future, so that an accurate three-dimensional model can be obtained in the end. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of an application scenario of the solution for creating a three-dimensional model of an object provided in this application.

[0034] Figure 2 It is a flowchart of an example of a method for creating a three-dimensional model of an object provided in an embodiment of the present application.

[0035] Figure 3 It is a flowchart of another example of the method for creating a three-dimensional model of an object provided in an embodiment of the present application.

[0036] Figure 4 It is a structural schematic diagram of another example of a device for creating a three-dimensional model of an object provided in an embodiment of the present application.

[0037] Figure 5 It is a structural block diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described clearly and completely below in conjunction with the drawings in the embodiments of the present application. However, the present application can be implemented in many other ways different from the following description. Therefore, based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present application.

[0039] It should be noted that the terms "first", "source domain", "third", etc. in the claims, description and drawings of the present application are used to distinguish similar objects and are not used to describe a specific order or sequence. The data used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including", "having" and their variants are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0040] In order to facilitate understanding of the various embodiments of the present application, the application background of the embodiments is described.

[0041] With the rapid development of computer vision, 3D modeling of objects has been widely used in many fields, such as e-commerce fields such as product image generation, product display, and consumer personalized matching. It can also be applied to virtual reality (VR), game development, cultural heritage protection, industrial design and other fields. Among them, creating a corresponding 3D model of an object based on the photo of the object is currently a more commonly used 3D modeling method. It can create a corresponding 3D model of the object according to the appearance of the object shown in the photo, so that the created 3D model can be applied to the 3D display of the object, allowing users to observe and interact from different angles, greatly enriching the user experience and improving work efficiency. 3D reconstruction of objects is a technology that digitally expresses and presents actual objects in 3D. In e-commerce scenarios, 3D reconstruction of products is an emerging and important technology, which is widely used in product design, product image generation, product display, consumer personalized matching and other fields. However, the 3D reconstruction of traditional links relies on professional shooting equipment and venues, and the threshold is high, so it cannot be applied on a large scale.

[0042] For small commodities, merchants can send the commodities to us, and then use professional shooting scenes and equipment to perform high-precision shooting and modeling. However, for large commodities such as furniture, it is not convenient to transport them, and the shooting equipment cannot support the modeling of large-sized objects. In this context, the quality of images taken by users varies, and the scene lighting varies greatly, which makes the 3D reconstruction algorithm face greater challenges.

[0043] Traditional 3D modeling technology usually requires users to create a matching 3D model of an object based on an object image using 3D modeling software. However, this 3D modeling method requires users to have high professional skills in 3D modeling, and the modeling process is often time-consuming. In addition, due to the large differences in the technical levels of modelers, the accuracy of the created 3D model of the object is also poor.

[0044] In related technologies, multimodal 3D model generation can also be achieved with the help of large model capabilities. However, the 3D models generated by large models focus on fast and large-scale generation, and pay attention to the integrity of the model itself, but the effect in terms of realism and precision is poor.

[0045] In order to solve the above problems, the embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for creating a three-dimensional model of an object. The purpose is to efficiently and quickly create a three-dimensional model of an object based on an image of the object, without requiring the user to have high professional skills in three-dimensional modeling, and the created three-dimensional model has higher accuracy.

[0046] The method for creating a three-dimensional model of an object provided in the present application can be used in the fields of three-dimensional display of goods, virtual try-on, object design, product image generation, etc., and the present application does not specifically limit it.

[0047] In order to facilitate understanding of the method embodiments of the present application, the application scenarios are introduced. Figure 1 , Figure 1 The following is a schematic diagram of an application scenario of the solution provided in the embodiment of the present application. The application scenario is a schematic illustration and is not intended to be a specific description of the application scenario. Figure 1 As shown, in this application scenario, a server 102 and a client 101 are provided. In this embodiment, a connection is established between the client 101 and the server 102 via network communication, so as to perform data transmission.

[0048] The client 101 can be an electronic device with display and data processing functions, such as a mobile phone, a tablet computer (pad), a smart watch, a desktop computer, a smart TV, a VR device, a vehicle-mounted device, a wearable device, a laptop computer, etc. The client 101 is used to send a target image corresponding to a target object to the server 102, and obtain a generated three-dimensional model corresponding to the target object from the server 102.

[0049] The server 102 has high computing power. The server 102 can be a server, which has high-speed processor (central processing unit, CPU) computing power, long-term reliable operation, powerful input / output (input / output, I / O) external data throughput capacity and better scalability. The server 102 can be a single server or a server cluster. The server 102 is deployed with an image processing model. The server 102 can generate a three-dimensional model corresponding to the target object based on the image processing model and each target image sent by the client, and send the three-dimensional model to the client 101. The server 102 can also provide other specific services for the client 101, such as user information access, website access, application access, etc., which are not specifically limited in this application.

[0050] The client 101 and the server 102 may communicate with each other using various communication systems, such as a wired communication system or a wireless communication system. The wireless communication system may be, for example, a global system for mobile communications (GSM) system, a code division multiple access (CDMA) system, a wideband code division multiple access (WCDMA) system, a general packet radio service (GPRS), a long term evolution (LTE) system, a LTE frequency division duplex (FDD) system, a LTE time division duplex (TDD) system, a universal mobile telecommunication system (UMTS), a worldwide interoperability for microwave access (WiMAX) communication system, a future fifth generation (5G) system or a new radio (NR), a satellite communication system, etc. Embodiment 1

[0051] The first embodiment of the present application provides a method for creating a three-dimensional model of an object. The method is applied to an electronic device, which may be a server, a laptop computer, a tablet computer, a desktop computer, a mobile phone, a smart TV, or other electronic device with data processing capabilities.

[0052] like Figure 2 As shown, the method for creating a three-dimensional model of an object provided in the first embodiment of the present application includes the following steps S110 to S140.

[0053] Step S110: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles.

[0054] The target object may be a commodity to be displayed, such as furniture, electrical appliances, electronic products, clothing, food, etc. to be displayed, or other objects, such as human bodies, animals, etc., and three-dimensional modeling of human bodies, animals, etc. can be applied to the fields of animation production, film production, advertising production, video synthesis, etc. Those skilled in the art can determine the specific content of the target object according to the specific object of the three-dimensional model created according to actual needs.

[0055] The target image corresponding to the target object may be a photographed image corresponding to the target object, or may be an image in other forms corresponding to the target object, such as a drawn image, a composite image, a video screenshot image, a video frame, etc. corresponding to the target object, but not limited thereto. The target image can display the target object from multiple perspectives, and the present application does not limit the specific type of the target image. Among them, the photographed image corresponding to the target object can clearly and accurately reflect the specific appearance of the target object, and a more accurate three-dimensional model corresponding to the target object can be created based on the photographed image corresponding to the target object.

[0056] Exemplarily, the photographer can use a mobile phone, camera or other device with a shooting function to shoot the target object from multiple angles, generate multiple target images corresponding to the target object, and then send the target images to the electronic device so that the electronic device receives each target image.

[0057] The multiple perspectives may include at least one of the following: multiple display orientations, multiple display scaling ratios, multiple display angles, and multiple local display locations. For example, the target images of the multiple perspectives may include pictures of the target object taken from different angles, images of the target object taken from different distances, etc., but are not limited thereto.

[0058] It is understandable that the multiple target images can display the target object from a relatively complete perspective as much as possible, so that complete information of the target object can be obtained from the multiple images, thereby making the final three-dimensional model more accurate.

[0059] Step S120: Determine the pixel correspondence between two images in an image pair corresponding to the multiple target images and the initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence is used to indicate the pixel position of an object part corresponding to a pixel position in one image of the image pair and the pixel position in the other image.

[0060] The image pair includes two images from the multiple target images.

[0061] The above-mentioned image processing model is used to determine the initial point cloud information of each image in the image pair and the pixel correspondence between the two images in the image pair. Specifically, the information of each target image included in each image pair and the two target images constituting each image pair can be input into the image processing model so that the image processing model can obtain the specific content of each target image and which images constitute the image pair. The image processing model then determines the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images based on the input information.

[0062] The image processing model can be trained through a large number of training samples. The training samples may include groups of sample image pairs. The sample image pairs include two similar sample images. Each group of sample image pairs marks the pixel correspondence between the two images contained therein. Each sample image is marked with corresponding sample point cloud information. Through the training samples, the image processing model can be trained using supervised learning algorithms, unsupervised learning algorithms or other training learning algorithms.

[0063] In one implementation, before step S120, the following step S110a may be further included: Step S110a: determining a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images.

[0064] The image pairs corresponding to the multiple target images may include image pairs consisting of two different images in the multiple target images. In actual applications, there are usually very large differences between two images in the multiple target images, for example, one image is an image of an object taken from the front, and the other image is an image of the object taken from the back, and the two images basically do not have the same part of the object, or the number of the same parts of the object in the two images is very small. Such image pairs are of little help in understanding the structure of the same part of the object at different viewing angles. Therefore, in order to reduce the amount of calculation and for the efficiency and accuracy of three-dimensional modeling, step S110a can be implemented according to the following step S120a.

[0065] Step S120a: According to the similarities between the target images, the multiple target images are grouped into multiple image pairs.

[0066] In this case, the image pair includes two similar images from the multiple target images, wherein the two target images in each group of image pairs are similar images, and the similarity of the two target images may be greater than a certain threshold. The two similar images are combined into an image pair, and the parameters such as the viewing angle and zoom ratio of the two similar images are usually similar, so that the object parts contained in the two images in the image pair are basically the same, that is, each object part contained in one image is also likely to exist in another similar image, so that the posture and appearance of the object at different viewing angles can be reflected through the two images with different viewing angles in the image pair.

[0067] In a specific embodiment, step S120a can be implemented according to the following steps S121 to S123.

[0068] Step S121: extracting image coding features corresponding to each of the target images based on a pre-trained image coding model.

[0069] The image coding model is used to numerically encode the target image to obtain the image coding features corresponding to the target image. The image coding features may include at least one of vector features, matrix features, tensor features, and hash coding features, but are not limited thereto. When the image features are coding features, it is more convenient to subsequently calculate the similarity between different target images.

[0070] The image coding model can be trained by learning algorithms such as supervised learning algorithms, unsupervised learning algorithms, semi-supervised learning algorithms, etc. in related technologies. The image coding model can be trained simultaneously with the image processing model used in the subsequent step S130 to improve the training efficiency and at the same time make the synergy of the two trained models more accurate.

[0071] Step S122: Calculate the similarity between each target image and other target images according to the image coding features.

[0072] The similarity between the image coding features corresponding to each two target images can be calculated. Specifically, the vector distance, matrix distance, matrix inner product and cosine similarity, inter-matrix correlation coefficient, hash coding distance, etc. between the image coding features corresponding to each two target images can be calculated. The specific types of image coding features are different, and the measurement parameters used to represent the similarity between the two target images are also different. Those skilled in the art can flexibly set them according to actual conditions.

[0073] This step is to calculate the similarity between every two different target images.

[0074] Step S123: grouping the multiple target images into multiple groups of image pairs according to the similarity between each target image and other target images.

[0075] Specifically, for each target image, images among other target images whose similarity with the target image is greater than a preset similarity threshold can be used to form image pairs with the target image, or images among other target images whose similarity with the target image is ranked before a preset number can be used to form image pairs with the target image.

[0076] The above similarity threshold and the above first preset number can be flexibly set according to needs, and are not specifically limited in this application.

[0077] In this embodiment, by encoding each target image, the similarity between different target images can be conveniently and accurately calculated through the obtained image encoding features, so that the two images in the composed image pair can be similar images.

[0078] In a specific embodiment, step S123 can be implemented as follows: based on the principle that the total number of image pairs does not exceed the second preset number, for each target image, the other target images that rank the first preset number of similarities with the target image are respectively formed into image pairs. Among them, the second preset number is greater than the first preset number. This embodiment can control the total number of image pairs to be not too large, thereby reducing the time and complexity of the three-dimensional model generation process. The second preset number should not be set too small to better ensure the accuracy of the algorithm, so that the final generated three-dimensional model has higher accuracy, so that the accuracy of the algorithm and the time consumption are balanced.

[0079] In step S123, when the number of composed image pairs exceeds the second preset number, the image pair with the lowest similarity corresponding to each target image can be deleted, and the image pair with the lowest similarity, that is, the similarity of the two images in the group of image pairs is the lowest among the image pairs composed of the target image.

[0080] In other embodiments, step S120a can also be implemented according to the following steps: calculate the color histogram of each target image, use distance comparison methods such as Bhattacharyya distance and chi-square distance to compare the similarity between the color histograms corresponding to different target images, and group the multiple target images into multiple groups of image pairs according to the similarity between the color histograms corresponding to different target images. Specifically, the image pairing can be performed according to the method of performing image pairing according to similarity as described above, which will not be repeated here, wherein the similarity between the color histograms corresponding to the target images can be determined as the similarity between the target images. Those skilled in the art can also calculate the similarity between different target images by other methods, and pair each target image according to the similarity, which is not specifically limited in this application.

[0081] The above pixel correspondence is used to indicate the pixel position of the object part corresponding to the pixel position of one image in the image pair in the other image, that is, the pixel correspondence is used to indicate the pixel points where the same part of the target object is located in the two images of the image pair. For example, the pixel correspondence indicates the pixel points where a corner of a table is located in the two images of the image pair, and there is a pixel correspondence relationship between the pixel points corresponding to the table corner in the two images.

[0082] Point Cloud is used to represent 3D spatial data. It consists of a series of discrete data points with specific locations in a 3D coordinate system. Together, they represent a 3D model of an object or scene. Each point contains its coordinate information in 3D space (usually X, Y, Z values). The initial point cloud information corresponding to the target image is used to represent the target object included in the target image in a three-dimensional form in the form of a point cloud.

[0083] In a specific embodiment, step S120 can be implemented according to the following steps: input the image coding features corresponding to each of the target images and the information of the two images constituting the image pair into a pre-trained image processing model to obtain the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images. The image processing model can be a decoding model. Since the intelligent model processes various coded data, inputting the image coding features into the image processing model can improve the efficiency of the model outputting the initial point cloud information and the pixel correspondence, thereby improving the efficiency of 3D modeling.

[0084] Step S130: Based on the initial point cloud information of the target image and the pixel correspondence, determine the fine point cloud information corresponding to the target image.

[0085] Since the initial point cloud information directly determined by the intelligent model in step S130 may have low accuracy and lack accurate representation in three-dimensional space, and the pixel correspondence can reflect the structural state of the same part of the object in images of two different perspectives, the pixel correspondence reflects the structural state of the object at different perspectives. This step can further optimize and adjust the initial point cloud information from a three-dimensional geometric perspective based on the pixel correspondence to obtain more accurate and fine point cloud information.

[0086] In a specific embodiment, the fine point cloud information corresponding to the target image can be determined in the following manner: based on the above pixel correspondence, the initial point cloud information is subjected to multi-view point cloud registration, specifically, the iterative closest point algorithm (ICP) or other optimization algorithms can be used to perform multi-view point cloud registration, so as to align the point clouds reflecting different perspectives in the image pair, so that the initial point cloud information of the processed target image is aligned at two different perspectives in the image pair to which it belongs; then, based on the pixel correspondence and camera parameters (intrinsic and extrinsic parameters), the position of each point in the processed initial point cloud information is further refined by triangulation and other methods to obtain fine point cloud information. Optionally, the fine point cloud information corresponding to the target image can also be determined in the following manner: the initial point cloud information of the target image and the pixel correspondence are input into a pre-trained fine point cloud determination model to obtain the fine point cloud information corresponding to the target image, and the fine point cloud determination model is used to determine the fine point cloud information corresponding to the initial point cloud information of the image based on the pixel correspondence between the two images.

[0087] In another specific embodiment, step S130 can be implemented according to the following steps S131 to S132.

[0088] Step S131: estimating initial camera intrinsic parameters corresponding to the target image based on the initial point cloud information of the target image.

[0089] Optionally, the initial camera intrinsic parameters corresponding to the target image can be determined by a pre-trained camera intrinsic parameter determination model, and the camera intrinsic parameter determination model is used to predict the camera intrinsic parameters corresponding to the image based on the point cloud information corresponding to the image. The training process of the camera intrinsic parameter determination model can be obtained by training using supervised learning algorithms, unsupervised learning algorithms, etc. in related technologies, and the specific training process will not be described in detail in this application.

[0090] In a specific embodiment, step S131 can be implemented according to the following steps S131a.

[0091] Step S131a: Based on the principle of minimizing the gap between the converted pixel position and the corresponding actual pixel position, the initial camera intrinsic parameters corresponding to the target image are estimated, wherein the converted pixel position is the corresponding pixel position obtained by converting the initial point cloud information corresponding to each pixel point of the target image according to the estimated initial camera intrinsic parameters, and the actual pixel position is the corresponding actual pixel position in the target image.

[0092] The camera intrinsic parameters may include the focal length of the camera and other intrinsic parameters, wherein the focal length is an important parameter for converting pixels in an image into a point cloud. The initial camera intrinsic parameters corresponding to the target image refer to the intrinsic parameters of the camera when the target image is captured.

[0093] In a specific embodiment, the initial point cloud information corresponding to the pixel points of the target image may include: x Dimensional initial point cloud information, y Dimensional initial point cloud information and z The three-dimensional initial point cloud information represents the coordinate information of the three dimensions of the point cloud. Correspondingly, the converted pixel position can be obtained in the following way: x Dimensional initial point cloud information and y The initial point cloud information of dimensions is y Normalization processing is performed on the initial point cloud information of the dimension to obtain the normalized initial point cloud information corresponding to each pixel point of the target image; the normalized initial point cloud information is converted into a conversion pixel position according to the estimated initial camera intrinsic parameters. Normalization processing can make the determined camera intrinsic parameters more accurate.

[0094] Specifically, step S131a can predict the initial camera intrinsic parameters according to the following formula (1).

[0095] (1) Among them, i, j represent the pixel coordinates of the target image, W, H represent the length and width of the target image, represents the nth point cloud prediction result of the target image, k represents the coordinate dimension, k is 1, 2, 3 represents the x, y, z dimensions respectively, f is the conversion function that converts the point cloud to pixel positions, f The function contains the camera internal parameters. It satisfies the minimization requirement on the right side of formula (1) f Function, because f The function contains the camera internal parameters, so, The camera intrinsic parameters in are the estimated initial camera intrinsic parameters.

[0096] Optionally, normalization processing may not be performed during the determination of the conversion pixel position, but the initial point cloud information corresponding to the target image may be directly converted into the conversion pixel position according to the estimated initial camera intrinsic parameters.

[0097] When the difference between the converted pixel position and the corresponding actual pixel position is the smallest, it means that the camera intrinsic parameter based on which the converted pixel position is determined is close to the actual camera intrinsic parameter when shooting the target image. This embodiment can quickly and accurately predict the initial camera intrinsic parameter corresponding to the target image based on the principle of minimizing the difference between the converted pixel position and the corresponding actual pixel position, thereby improving the efficiency and accuracy of three-dimensional modeling. Among them, the process of determining the initial camera intrinsic parameter corresponding to the target image based on the principle of minimizing the difference between the converted pixel position and the corresponding actual pixel position can be gradually optimized by the gradient descent method to obtain the parameter value when minimized.

[0098] Step S132: Determine the fine point cloud information corresponding to the target image based on the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters.

[0099] Specifically, step S132 can be implemented according to the following steps S132a to S132b.

[0100] Step S132a: estimating the initial camera extrinsic parameters corresponding to the target image based on the pixel correspondence relationship and the initial point cloud information corresponding to the target image.

[0101] The initial camera extrinsic parameters corresponding to the target image refer to the extrinsic parameters of the camera when shooting the target image, and the camera extrinsic parameters may include camera pose and zoom factor. Specifically, the pixel correspondence relationship and the initial point cloud information corresponding to the target image may be input into the camera extrinsic parameter determination model through a pre-trained camera extrinsic parameter determination model to obtain the initial camera extrinsic parameters corresponding to the target image.

[0102] Optionally, step S132a can also estimate the initial camera extrinsic parameters corresponding to the target image according to the following steps: based on the principle of minimizing the gap between matching world point cloud information, estimate the initial camera extrinsic parameters corresponding to the target image, the matching world point cloud information is the initial point cloud information of the pixel points corresponding to the two images in the image pair, which is point cloud information after being converted to the world coordinate system according to the estimated initial camera extrinsic parameters.

[0103] When the difference between the initial point cloud information of the pixel points corresponding to the two images in the image pair after being converted to the world coordinate system according to the estimated initial camera extrinsic parameters is relatively small, it means that the estimated camera extrinsic parameters are relatively close to the camera extrinsic parameters of the target image. This embodiment can quickly and accurately predict the initial camera extrinsic parameters corresponding to the target image based on the principle of minimizing the gap between the matching world point cloud information, thereby improving the efficiency and accuracy of three-dimensional modeling.

[0104] Specifically, the initial camera extrinsic parameters corresponding to the target image can be estimated according to the following steps: determine the weight corresponding to each pixel in the target image, the weight is used to represent the importance of the pixel for displaying the target object, and the weights of the pixels corresponding to the two images in the image pair are the same; determine the product of the gap between the matching world point cloud information and the corresponding weight as the first weighted gap of each pair of matching pixels; estimate the initial camera extrinsic parameters corresponding to the target image based on the principle of minimizing the sum of the first weighted gaps corresponding to the image pair. The weights of different pixels are usually different. For example, for an image of a sofa, the weight of the pixel at the edge of the sofa shape is higher than the weight of the pixel in the middle of the sofa, and the weight of the pixel in the blank area of ​​the image is lower than the weight of the pixel with the sofa. The weight of each pixel can be determined by a pre-trained weight determination model. By setting the weight of each pixel, this embodiment can make the pixel of the important part of the reference target object more focused when determining the camera extrinsic parameters, thereby improving the calculation efficiency and accuracy.

[0105] Specifically, the initial camera extrinsic parameters corresponding to the target image may be estimated according to the following formula (2).

[0106] (2) in, , are the camera initial pose and camera initial zoom coefficient corresponding to the estimated target image, that is, P and , P and are the camera pose and scaling factor corresponding to the target image, respectively. n,m ) is an image pair, is the pixel matching information of the image pair, , They are the initial point clouds corresponding to the matching pixels of the two images of the image pair, which are converted to the point clouds of the world coordinate system according to the camera pose and camera zoom factor. q c is the weight.

[0107] Based on the principle of minimizing the gap between the matching world point cloud information, the process of estimating the initial camera extrinsic parameters corresponding to the target image can be gradually optimized by the gradient descent method to obtain the parameter value when it is minimized.

[0108] Step S132b: Determine the fine point cloud information corresponding to the target image based on the initial camera extrinsic parameters, the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters.

[0109] This embodiment adds camera extrinsics corresponding to the target image, which can more accurately and efficiently determine detailed point cloud information, thereby more accurately generating a three-dimensional model corresponding to the target object.

[0110] Specifically, the initial camera extrinsic parameters, the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters may be input into a pre-trained point cloud determination model to obtain fine point cloud information corresponding to the target image.

[0111] In a specific embodiment, step S132b can also be implemented according to the following steps: based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image in the image pair and the converted pixel position corresponding to the matching pixel point, determine the fine point cloud information corresponding to the target image, and the matching pixel point is the matching pixel point in the other image in the image pair.

[0112] The converted pixel position corresponding to the matched pixel point is determined by following steps A to B: Step A: based on the initial camera extrinsic parameters, the pixel correspondence relationship and the initial camera intrinsic parameters, the initial point cloud information corresponding to the matched pixel points is converted into world point cloud information.

[0113] Step B: Convert the world point cloud information corresponding to the matched pixel point to a two-dimensional image position to obtain a converted pixel position corresponding to the matched pixel point.

[0114] When the difference between the actual pixel position of a pixel point in one image in the image pair and the converted pixel position corresponding to the matching pixel point is relatively small, it means that the point cloud used to predict the converted pixel position can well reflect the actual point cloud of the object displayed in the target image. This embodiment can efficiently and accurately determine the point cloud corresponding to the target image by minimizing the difference between the actual pixel position of a pixel point in one image in the image pair and the converted pixel position corresponding to the matching pixel point.

[0115] Specifically, step S132b can be implemented according to the following steps a to c.

[0116] Step a: Determine the weight corresponding to each pixel in the target image, where the weight is used to represent the importance of the pixel for displaying the target object, and the weights of the corresponding pixels in the two images in the image pair are the same.

[0117] Step b: Determine the product of the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point and the corresponding weight as the second weighted difference for each pair of matched pixels.

[0118] Step c: Determine the fine point cloud information corresponding to the target image based on the principle of minimizing the sum of the second weighted differences corresponding to the image pair.

[0119] The specific content of the weight can be referred to above and will not be described in detail here. This embodiment sets the weight of each pixel point, so that when determining the detailed point cloud information, the pixel points of important parts of the reference target object can be more focused, thereby improving the calculation efficiency and accuracy.

[0120] Specifically, step S132b may determine the fine point cloud information corresponding to the target image according to the following formula (3).

[0121] (3) ρ is the loss function, ( n,m ) is an image pair, M n,m is the pixel matching information of the image pair, x n c 、x m c They are the initial point clouds corresponding to the matching pixels of the two images of the image pair, which are converted to the point clouds of the world coordinate system according to the camera pose and camera zoom factor. q c is the weight, π n 、π m is an image n、 The projection function of the three-dimensional point cloud of image m to the two-dimensional projection coordinates, which contains the camera intrinsic parameters corresponding to the image. y n c 、y m c are the actual pixel positions of the matching pixels in the n and m images of the image pair, respectively, and Z * , K * , , are respectively the fine point cloud information corresponding to the estimated target image, the camera internal parameters, the camera pose, and the camera zoom factor, that is, Z, K, P, and , q c is the weight.

[0122] Formula (3) can be used to quickly and accurately determine the fine point cloud information, and formula (3) can also be used to determine the accurate camera intrinsic and extrinsic parameters.

[0123] Based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matching pixel point, the process of determining the fine point cloud information corresponding to the target image can be gradually optimized by the gradient descent method to obtain the parameter value when minimized.

[0124] Step S140: creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images.

[0125] Specifically, the fine point cloud information corresponding to each of the target images can be converted into a world coordinate system to obtain world point cloud information corresponding to each target image, and then the world point cloud information corresponding to each target image can be spliced ​​to obtain dense point cloud information corresponding to the target object, and a three-dimensional model corresponding to the target object can be created based on the dense point cloud information corresponding to the target object.

[0126] In a specific embodiment, step S132b can be implemented in the following steps: based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point, the fine point cloud information corresponding to the target image and the precise camera intrinsic parameter corresponding to the target image are determined. That is, in the process of minimizing the difference between the actual pixel position and the converted pixel position corresponding to the matched pixel point, the fine point cloud information and the precise camera intrinsic parameter are determined at the same time. As described in formula (3), in the process of determining the fine point cloud information by formula (3), the precise camera intrinsic parameter can also be determined.

[0127] Correspondingly, before step S140, the following steps S140a to S140b may also be included.

[0128] Step S140a: Determine a global normal vector corresponding to the target image according to the precise camera intrinsic parameters and the fine point cloud information corresponding to the target image.

[0129] Since step S132b has obtained the depth map corresponding to each target image after global consistency constraint optimization, and camera intrinsics , where the depth map is the fine point cloud information corresponding to the target image. According to the depth map and the camera internal parameters, the global normal vector with global consistency can be easily obtained , where H represents the formula for converting the depth map to the normal vector map.

[0130] Step S140b: Determine the accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image.

[0131] Specifically, the global normal vector may be determined as the accurate normal vector corresponding to the target image, or the global normal vector may be optimized to obtain the accurate normal vector.

[0132] Since the global normal vector is converted from the depth map, the detail accuracy is insufficient. Therefore, in one embodiment, the pre-trained image processing model is also used to determine the predicted normal vector corresponding to each of the image pairs in the process of determining the pixel correspondence and the initial point cloud information. In other words, the image processing model can also output the predicted normal vector corresponding to the image pair. Since the predicted normal vector refers to two images of different perspectives in the image pair at the same time, the detail accuracy is high.

[0133] Since the predicted normal vectors corresponding to the image pairs are determined, assuming that The images contribute to the composition image pairs, then the image has a total of Predicted normal vectors: .

[0134] Step S140b may be specifically implemented according to the following steps: determining the accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image and the corresponding predicted normal vectors.

[0135] This embodiment combines the global normal vector and each predicted normal vector, and the two are combined to obtain an accurate normal vector, so that the obtained accurate normal vector can take into account the requirements of globality and detail accuracy, and has higher accuracy. In addition, the embodiment of the present application determines the normal vector based on two images of different perspectives in the image pair, and the determined normal vector can well avoid perspective discrimination and has more accurate spatial geometric information, so that the determined normal vector is more accurate, and further makes the generated three-dimensional model more accurate.

[0136] For example, the vector with the highest similarity to the global normal vector among the predicted normal vectors corresponding to the target image may be determined as the accurate normal vector corresponding to the target image.

[0137] Optionally, the accurate normal vector corresponding to the target image may be determined according to the following steps 1 to 3.

[0138] Step 1: Calculate the error between each predicted normal vector corresponding to the target image and the corresponding global normal vector.

[0139] The error between the predicted normal vector and the global normal vector is the distance between the predicted normal vector and the global normal vector. The larger the error, the greater the difference between the two vectors.

[0140] Specifically, step 1 may calculate, for each pixel point of the target image, the error between each predicted normal vector corresponding to the target image and the corresponding global normal vector.

[0141] Step 2: excluding the pixel points whose errors with the global normal vector are greater than a preset error from the predicted normal vectors corresponding to the target image, to obtain the excluded predicted normal vectors corresponding to the target image after the excluded pixel points are obtained.

[0142] The predicted normal vector includes the predicted normal vectors of each pixel point of the target image. After excluding some pixels in the predicted normal vector, the predicted normal vector only includes the predicted normal vectors corresponding to some pixels.

[0143] Step 3: Determine the accurate normal vector corresponding to the target image according to the mean of the excluded predicted normal vectors corresponding to the target image.

[0144] The mean value of the predicted normal vectors after elimination can be determined as the accurate normal vector corresponding to each pixel point of the target image.

[0145] In this embodiment, vectors among the predicted normal vectors whose errors with the global normal vector are too large, and the deleted predicted normal vectors are vectors with poor globality, and the remaining predicted normal vectors after exclusion are vectors with relatively high accuracy and relatively good globality. The mean of each predicted normal vector after exclusion is then determined as the accurate normal vector corresponding to the target image. That is, this application optimizes the predicted normal vector through a global consistency optimization strategy, and can obtain a normal vector with better globality and more accurate details.

[0146] Optionally, when a pixel point of the target image is excluded in each of the predicted normal vectors after exclusion, the normal vector of the pixel point in the global normal vector corresponding to the target image can be determined as the accurate normal vector corresponding to the pixel point to ensure that each pixel point can obtain the corresponding accurate normal vector.

[0147] Correspondingly, the above step S140 can be implemented according to the following step S141.

[0148] Step S141: creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images and the accurate normal vector corresponding to each of the target images.

[0149] Since the normal vector is a vector defined on the surface of a three-dimensional object, its direction is perpendicular to the surface and can be used to describe the shape and direction of the surface of the object. In image processing and computer graphics, normal vector information is very important for tasks such as illumination modeling, shadow processing, texture mapping, object surface restoration and 3D reconstruction. This embodiment can create a more accurate three-dimensional model of the target object through accurate normal vectors.

[0150] Specifically, step S141 may create a three-dimensional model corresponding to the target object according to the dense point cloud information and the precise normal vector corresponding to the target object.

[0151] In one embodiment, a 3D model corresponding to a target object can be generated by a pre-trained 3D model generation model. Specifically, dense point cloud information and precise normal vectors corresponding to the target object can be input into the 3D model generation model to obtain a 3D model corresponding to the target object. When the dense point cloud and normal vectors of the object are known, the intelligent model can accurately and quickly generate the corresponding 3D model.

[0152] The following is an illustrative example of the method for creating a three-dimensional model provided by the present application. Figure 3 As shown, the method for creating a three-dimensional model provided in this example includes the following steps: Step 1: Obtain multiple target images corresponding to the target object for which the 3D model is to be created.

[0153] The multiple target images are multiple captured images, and the multiple captured images show the target object from multiple perspectives.

[0154] Step 2: Input multiple target images into a pre-trained image coding model to obtain image coding features corresponding to each of the target images.

[0155] Step 3: According to the similarity between different image coding features, the multiple target images are formed into multiple groups of image pairs.

[0156] Assume that the total number of target images is , according to the similarity between different image encoding features, it is concluded that each target image is similar to other The similarity quantification index between target images is taken for each target image. The top similarity rankings among the target images The target images form an image pair, and the total image pairs, in order to ensure the balance between the accuracy and time consumption of the algorithm, we can control Not exceeding the second preset number .

[0157] Step 4: Input the image encoding features of the image pair sequence into a pre-trained decoding model to obtain the pixel correspondence between the two images in the image pair, the initial point cloud information corresponding to each of the target images, and the predicted normal vector of each target image.

[0158] Step 5: Determine the dense point cloud information and camera parameters of the target object according to the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images.

[0159] Specifically, the dense point cloud information, camera extrinsic parameters and intrinsic parameters of the target object can be determined by the gradient descent method.

[0160] Step 6: Determine the global normal vector corresponding to the target image based on the above-mentioned camera intrinsic parameters and the fine point cloud information corresponding to the target image, and combine the global normal vector and the predicted normal vector to obtain the accurate normal vector corresponding to each target image.

[0161] Step 7: Create a three-dimensional model of the target object based on the accurate normal vector corresponding to each target image and the dense point cloud information of the target object.

[0162] This example is an illustrative description of an embodiment of the present invention. The detailed execution process of each step can be referred to the description above and will not be repeated here.

[0163] The method for creating a three-dimensional model of an object provided in an embodiment of the present application obtains multiple target images corresponding to the target object of the three-dimensional model to be created, and the multiple target images are used to display the target object from multiple perspectives, and then based on a pre-trained image processing model, the pixel correspondence between two images in an image pair corresponding to the multiple target images and the initial point cloud information corresponding to each of the target images are determined, wherein the image pair includes two images of the multiple target images, and the two images in a group of image pairs can display various parts of the object from different perspectives, so that each group of image pairs can better accurately reflect the information of various parts of the object from the perspective of stereo geometry, and the pixel correspondence is used to indicate the pixel position of the object part corresponding to the pixel position of one image in the image pair in the other image, that is, the pixel correspondence can associate and correspond the pixel points belonging to the same object part in the two images, and the pixel correspondence can well reflect the position information of the same object part at different perspectives, so that the pixel correspondence can well reflect the stereo geometric information of the object, and the above-mentioned image processing model is first trained, and the image processing model can determine the pixel correspondence between the two images in each group of the image pairs. The initial point cloud information corresponding to the target image and the pixel correspondence relationship are determined by the pre-trained image processing model, wherein the initial point cloud information reflects the depth image corresponding to each target image, and the initial point cloud information can reflect the geometric information of the object corresponding to a single image. Then, based on the initial point cloud information of the target image and the pixel correspondence relationship, the fine point cloud information corresponding to the target image is determined. Since the pixel correspondence relationship can well reflect the position information of the same object part at different viewing angles, the initial point cloud information is refined through the pixel correspondence relationship, so that the refined point cloud information after the refinement can have higher accuracy in the three-dimensional space, that is, the fine point cloud information can more accurately represent the various parts of the target object in space. Then, according to the fine point cloud information corresponding to each of the target images, it is determined to create a three-dimensional model corresponding to the target object. Since the fine point cloud information can more accurately represent the various parts of the target object in space, the three-dimensional model corresponding to the target object created according to the fine point cloud information corresponding to each of the target images is more accurate in the three-dimensional space and more matched with the actual target object.

[0164] The solution provided by the present application firstly determines the initial point cloud information corresponding to each target image quickly and efficiently through an image processing model. Since the initial point cloud information is obtained based on a single target image, the initial point cloud information may lack an accurate reflection of the target object in three-dimensional space. The present application then adjusts the initial point cloud information through the pixel correspondence corresponding to each group of images that can well reflect the three-dimensional geometric information of the object, and obtains fine point cloud information that can more accurately represent various parts of the target object in space, so that the three-dimensional model created according to the fine point cloud information corresponding to each of the target images is more matched with the actual target object and has higher accuracy.

[0165] It can be seen that the solution provided by the present application can automatically generate a three-dimensional model corresponding to the target object through a computer, without the need for manual operation of the three-dimensional modeling software. Therefore, it is possible to efficiently and quickly create a three-dimensional model of an object based on the image of the object, without the need for the user to have high professional skills in three-dimensional modeling, and the created three-dimensional model is more accurate and more consistent with the target object. In addition, since the image processing model is trained through a large number of training samples, the training process usually includes both good quality images and poor quality images. Therefore, the trained image processing model has a strong prior, which makes the number of pixel matches predicted by the image processing model larger, and can also achieve good prediction and processing for poor image quality such as low texture, blur, low resolution, and poor lighting conditions, and obtain an accurate three-dimensional model. The initial point cloud information of each target image predicted by the pre-trained image processing model itself also has a relatively high accuracy, can contain rich geometric information, and creates conditions for obtaining more accurate fine point cloud information in the future, so that an accurate three-dimensional model can be obtained in the end. Embodiment 2

[0166] The second embodiment of the present application further provides a method for creating a three-dimensional model of an object, the method comprising the following steps S210 to S270.

[0167] Step S210: Acquire multiple target images corresponding to the target object for which the three-dimensional model is to be created, wherein the multiple target images are used to display the target object from multiple viewing angles; Step S220: determining a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Step S230: determining a pixel correspondence relationship between two images in the image pair, initial point cloud information corresponding to each of the target images, and a predicted normal vector corresponding to each of the image pairs based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one of the image pairs; Step S240: determining the fine point cloud information corresponding to the target image and the precise camera intrinsic parameters corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; Step S250: determining a global normal vector corresponding to the target image according to the precise camera intrinsic parameters and the fine point cloud information corresponding to the target image; Step S260: determining an accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image and the corresponding predicted normal vectors; Step S270: creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images and the accurate normal vector corresponding to each of the target images.

[0168] The various execution processes of this embodiment are similar to those of the first embodiment. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the embodiment of the method for creating a three-dimensional model of an object provided in the above-mentioned first embodiment. Embodiment 3

[0169] The third embodiment of the present application also provides a method for creating a three-dimensional model of an object, which is applied to a server and includes the following steps S310 to S330.

[0170] Step S310: acquiring a plurality of target images sent by the client for displaying the target object, wherein the plurality of target images are used to display the target object from multiple viewing angles; Step S320: creating a three-dimensional model corresponding to the target object by using the method for creating a three-dimensional model of an object described in any one of the first embodiment or the second embodiment; Step S330: Send the three-dimensional model to the client, so that the client displays the three-dimensional model.

[0171] This embodiment is an application scenario embodiment of the method for creating a three-dimensional model of an object provided in the above embodiment. Its specific modeling process is basically similar to that of the method embodiment, so the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding instructions of the embodiment of the method for creating a three-dimensional model of an object provided above. Embodiment 4

[0172] The fourth embodiment of the present application also provides a device for creating a three-dimensional model of an object corresponding to the method for creating a three-dimensional model of an object provided in the first embodiment. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For details of the relevant technical features and the effects achieved, please refer to the corresponding description of the method for creating a three-dimensional model of an object provided above. Figure 4As shown, the device for creating a three-dimensional model of an object provided in this embodiment includes: An acquisition unit 410 is used to acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; A pairing unit 420, configured to determine a plurality of image pairs consisting of the plurality of target images, each of the image pairs comprising two different target images; A determination unit 430 is used to determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position corresponding to an object part in another image corresponding to a pixel position in one image of the image pair; based on the initial point cloud information of the target image and the pixel correspondence relationship, determine fine point cloud information corresponding to the target image; The modeling unit 440 is used to create a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images. Embodiment 5

[0173] The third embodiment of the present application also provides an electronic device embodiment corresponding to the method for creating a three-dimensional model of an object provided in the first embodiment. The following description of the electronic device embodiment is only illustrative. The electronic device embodiment is as follows: Please refer to Figure 5 Understand the above electronic devices, Figure 5 The electronic device provided in this embodiment includes: a processor 1001, a memory 1002, a communication bus 1003, and a communication interface 1004; The memory 1002 is used to store computer instructions for data processing. When the computer instructions are read and executed by the processor 1001, the following steps are performed: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; Determine a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one image in the image pair; Determining fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; A three-dimensional model corresponding to the target object is created according to the fine point cloud information corresponding to each of the target images.

[0174] The sixth embodiment of the present application also provides a computer-readable storage medium for implementing the method described in the first embodiment. The computer-readable storage medium embodiment provided in the present application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above method embodiment. The embodiment described below is only illustrative.

[0175] The computer readable storage medium provided in this embodiment stores computer instructions, and when the instructions are executed by a processor, the following steps are implemented: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; Determine a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one image in the image pair; Determining fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; A three-dimensional model corresponding to the target object is created according to the fine point cloud information corresponding to each of the target images.

[0176] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0177] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0178] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0179] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0180] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A method for creating a three-dimensional model of an object, characterized in that: The method comprises: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; Determine a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one image in the image pair; Determining fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; A three-dimensional model corresponding to the target object is created according to the fine point cloud information corresponding to each of the target images.

2. The method for creating a three-dimensional model of an object according to claim 1, characterized in that: The determining, based on the initial point cloud information of the target image and the pixel correspondence, the fine point cloud information corresponding to the target image comprises: estimating initial camera intrinsic parameters corresponding to the target image based on initial point cloud information of the target image; Based on the pixel correspondence, the initial point cloud information corresponding to the target image and the initial camera intrinsic parameters, the fine point cloud information corresponding to the target image is determined.

3. The method for creating a three-dimensional model of an object according to claim 2, characterized in that: The estimating the initial camera intrinsic parameters corresponding to the target image based on the initial point cloud information of the target image includes: Based on the principle of minimizing the gap between the converted pixel position and the corresponding actual pixel position, the initial camera intrinsic parameters corresponding to the target image are estimated. The converted pixel position is the corresponding pixel position obtained by converting the initial point cloud information corresponding to each pixel point of the target image according to the estimated initial camera intrinsic parameters. The actual pixel position is the corresponding actual pixel position in the target image.

4. The method for creating a three-dimensional model of an object according to claim 3, characterized in that: The initial point cloud information corresponding to the pixel points of the target image includes x Dimensional initial point cloud information, y Dimensional initial point cloud information and z Dimensional initial point cloud information; The converted pixel position is obtained as follows: The target image x Dimensional initial point cloud information and y The initial point cloud information of dimensions is z Normalization processing is performed on the initial point cloud information of the dimension to obtain normalized initial point cloud information corresponding to each pixel point of the target image; The normalized initial point cloud information is converted into a conversion pixel position according to the estimated initial camera intrinsic parameters.

5. The method for creating a three-dimensional model of an object according to claim 2, characterized in that: The determining, based on the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameter, of the fine point cloud information corresponding to the target image includes: estimating the initial camera extrinsic parameters corresponding to the target image based on the pixel correspondence relationship and the initial point cloud information corresponding to the target image; Based on the initial camera extrinsic parameters, the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters, the fine point cloud information corresponding to the target image is determined.

6. The method for creating a three-dimensional model of an object according to claim 5, characterized in that: The estimating the initial camera extrinsic parameters corresponding to the target image based on the pixel correspondence relationship and the initial point cloud information corresponding to the target image includes: Based on the principle of minimizing the gap between matching world point cloud information, the initial camera extrinsic parameters corresponding to the target image are estimated. The matching world point cloud information is the point cloud information of the pixel points corresponding to the two images in the image pair after being converted to the world coordinate system according to the estimated initial camera extrinsic parameters.

7. The method for creating a three-dimensional model of an object according to claim 6, characterized in that: The estimating the initial camera extrinsic parameters corresponding to the target image based on the principle of minimizing the gap between the matched world point cloud information includes: Determine a weight corresponding to each pixel in the target image, wherein the weight is used to indicate the importance of the pixel for displaying the target object, and the weights of the corresponding pixels in the two images in the image pair are the same; Determine the product of the gap between the matched world point cloud information and the corresponding weight as the first weighted gap of each pair of matched pixel points; The initial camera extrinsic parameters corresponding to the target image are estimated based on the principle of minimizing the sum of the first weighted differences corresponding to the image pair.

8. The method for creating a three-dimensional model of an object according to claim 5, characterized in that: The determining, based on the initial camera extrinsic parameters, the pixel correspondence, the initial point cloud information corresponding to the target image, and the initial camera intrinsic parameters, of the fine point cloud information corresponding to the target image includes: Determining fine point cloud information corresponding to the target image based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point, wherein the matched pixel point is the matched pixel point in the other image of the image pair; The converted pixel position corresponding to the matched pixel point is determined by: Converting the initial point cloud information corresponding to the matched pixel points into world point cloud information based on the initial camera extrinsic parameters, the pixel correspondence relationship and the initial camera intrinsic parameters; The world point cloud information corresponding to the matched pixel point is converted to a two-dimensional image position to obtain a converted pixel position corresponding to the matched pixel point.

9. The method for creating a three-dimensional model of an object according to claim 8, characterized in that: The method of determining the fine point cloud information corresponding to the target image based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point comprises: Determine a weight corresponding to each pixel in the target image, wherein the weight is used to indicate the importance of the pixel for displaying the target object, and the weights of the corresponding pixels in the two images in the image pair are the same; Determine the product of the difference between the actual pixel position of a pixel point of one image in the image pair and the converted pixel position corresponding to the matched pixel point and the corresponding weight as the second weighted difference for each pair of matched pixels; The fine point cloud information corresponding to the target image is determined based on the principle of minimizing the sum of the second weighted differences corresponding to the image pair.

10. The method for creating a three-dimensional model of an object according to claim 6, characterized in that: The camera external parameters include at least one of a camera posture and a scaling factor, and the camera internal parameters include a camera focal length.

11. The method for creating a three-dimensional model of an object according to any one of claims 1 to 10, characterized in that: The step of determining a plurality of groups of image pairs consisting of the plurality of target images comprises: According to the similarities between the target images, the multiple target images are grouped into multiple groups of image pairs.

12. The method for creating a three-dimensional model of an object according to claim 11, characterized in that: The step of grouping the plurality of target images into a plurality of image pairs according to the similarities between the target images comprises: Extracting image coding features corresponding to each of the target images based on a pre-trained image coding model; Calculating the similarity between each target image and other target images according to the image coding features; The multiple target images are grouped into multiple groups of image pairs according to the similarity between each target image and other target images.

13. The method for creating a three-dimensional model of an object according to claim 12, characterized in that: The step of grouping the plurality of target images into a plurality of image pairs according to the similarity between each target image and other target images comprises: For each target image, images among other target images whose similarity to the target image is ranked before a preset number are respectively formed into image pairs with the target image, or images among other target images whose similarity to the target image is greater than a preset similarity threshold are respectively formed into image pairs with the target image.

14. The method for creating a three-dimensional model of an object according to claim 13, characterized in that: The step of grouping the plurality of target images into a plurality of image pairs according to the similarity between each target image and other target images comprises: Based on the principle that the total number of image pairs does not exceed a second preset number, for each target image, other target images that rank before a preset number of similarities with the target image are respectively formed into image pairs.

15. The method for creating a three-dimensional model of an object according to claim 12, characterized in that: The determining of the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images based on the pre-trained image processing model includes: The image coding features corresponding to each of the target images and the information of the two images constituting the image pair are input into a pre-trained image processing model to obtain the pixel correspondence between the two images in the image pair and the initial point cloud information corresponding to each of the target images.

16. The method for creating a three-dimensional model of an object according to claim 8, characterized in that: The method of determining the fine point cloud information corresponding to the target image based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point comprises: Based on the principle of minimizing the difference between the actual pixel position of a pixel point in one image of the image pair and the converted pixel position corresponding to the matched pixel point, determining the fine point cloud information corresponding to the target image and the precise camera intrinsic parameters corresponding to the target image; The method further comprises: Determine a global normal vector corresponding to the target image according to the precise camera intrinsic parameters and the fine point cloud information corresponding to the target image; Determining an accurate normal vector corresponding to the target image according to a global normal vector corresponding to the target image; The step of creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images comprises: A three-dimensional model corresponding to the target object is created based on the fine point cloud information corresponding to each of the target images and the accurate normal vector corresponding to each of the target images.

17. The method for creating a three-dimensional model of an object according to claim 16, characterized in that: The pre-trained image processing model is further used to determine the predicted normal vector corresponding to each of the image pairs in the process of determining the pixel correspondence relationship and the initial point cloud information; The step of determining the accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image comprises: According to the global normal vector corresponding to the target image and the corresponding predicted normal vectors, the accurate normal vector corresponding to the target image is determined.

18. The method for creating a three-dimensional model of an object according to claim 17, characterized in that: The determining, according to the global normal vector corresponding to the target image and the corresponding predicted normal vectors, the accurate normal vector corresponding to the target image comprises: Calculating the error between each predicted normal vector corresponding to the target image and the corresponding global normal vector; Eliminate pixels whose errors with the global normal vector are greater than a preset error from among the predicted normal vectors corresponding to the target image, and obtain excluded predicted normal vectors corresponding to the target image after excluding the pixels; The accurate normal vector corresponding to the target image is determined according to the mean value of each excluded predicted normal vector corresponding to the target image.

19. The method for creating a three-dimensional model of an object according to claim 18, characterized in that: The method further comprises: When a pixel point of the target image is excluded in each of the excluded predicted normal vectors, the normal vector of the pixel point in the global normal vector corresponding to the target image is determined as the accurate normal vector corresponding to the pixel point.

20. The method for creating a three-dimensional model of an object according to any one of claims 1 to 11, characterized in that: The step of creating a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images comprises: The dense point cloud information corresponding to the target object is determined according to the fine point cloud information corresponding to each of the target images, and a three-dimensional model corresponding to the target object is created according to the dense point cloud information.

21. A method for creating a three-dimensional model of an object, characterized in that: The method comprises: Acquire a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used to display the target object from multiple viewing angles; Determine a plurality of groups of image pairs consisting of the plurality of target images, each group of the image pairs including two different target images; Determine, based on a pre-trained image processing model, a pixel correspondence relationship between two images in the image pair, initial point cloud information corresponding to each of the target images, and a predicted normal vector corresponding to each of the image pairs, wherein the pixel correspondence relationship is used to indicate a pixel position in another image corresponding to an object part corresponding to a pixel position in one of the image pairs; Based on the initial point cloud information of the target image and the pixel correspondence, determining the fine point cloud information corresponding to the target image and the precise camera intrinsic parameters corresponding to the target image; Determine a global normal vector corresponding to the target image according to the precise camera intrinsic parameters and the fine point cloud information corresponding to the target image; Determine the accurate normal vector corresponding to the target image according to the global normal vector corresponding to the target image and the corresponding predicted normal vectors; A three-dimensional model corresponding to the target object is created based on the fine point cloud information corresponding to each of the target images and the accurate normal vector corresponding to each of the target images.

22. A method for creating a three-dimensional model of an object, characterized in that: Applied to the server, the method includes: Acquire multiple target images sent by the client for displaying the target object, wherein the multiple target images are used to display the target object from multiple viewing angles; Creating a three-dimensional model corresponding to the target object by the method for creating a three-dimensional model of an object according to any one of claims 1 to 21; The three-dimensional model is sent to a client so that the client displays the three-dimensional model.

23. A device for creating a three-dimensional model of an object, characterized in that: The device comprises: An acquisition unit, used for acquiring a plurality of target images corresponding to a target object for which a three-dimensional model is to be created, wherein the plurality of target images are used for displaying the target object from a plurality of viewing angles; A pairing unit, used for determining a plurality of image pairs consisting of the plurality of target images, each of the image pairs comprising two different target images; A determination unit, configured to determine a pixel correspondence relationship between two images in the image pair and initial point cloud information corresponding to each of the target images based on a pre-trained image processing model, wherein the pixel correspondence relationship is used to indicate a pixel position corresponding to an object part in another image corresponding to a pixel position in one image of the image pair; and determine fine point cloud information corresponding to the target image based on the initial point cloud information of the target image and the pixel correspondence relationship; A modeling unit is used to create a three-dimensional model corresponding to the target object according to the fine point cloud information corresponding to each of the target images.

24. An electronic device, characterized in that: include: A processor, a memory, and computer program instructions stored on the memory and executable on the processor; When the processor executes the computer program instructions, the method according to any one of claims 1 to 22 is implemented.

25. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in any one of claims 1 to 22.

Images