A VR-based clothing rendering method and system

Abstract

The present invention provides a VR-based clothing rendering method and system. The rendering method includes: obtaining multiple images of clothing taken from different perspectives and performing feature matching and geometric information correction, selecting an image for triangulation to obtain a preliminary restored sparse point cloud structure map; performing pose estimation and beam optimization on the point cloud structure map, sequentially adding images corresponding to other perspectives to the beam-optimized point cloud structure map and performing triangulation and beam optimization to obtain a point cloud clothing map of the clothing and corresponding pose data; generating a three-dimensional Gaussian with each point in the point cloud clothing map as the center, projecting the generated three-dimensional Gaussian onto the image plane using the shooting pose, calculating the light intensity from each point to the projection center based on a volume rendering algorithm to obtain a preliminary rendered image; and performing aliasing artifact removal on the preliminary rendered image to obtain a target clothing rendered image. The VR-based clothing rendering method provided by the present invention is highly efficient and has good rendering quality.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a VR-based clothing rendering method and system. Background Art

[0002] As a cutting-edge human-computer interaction technology, virtual reality (VR) has demonstrated tremendous potential in entertainment, education, healthcare, and other fields. VR utilizes real-life data, generates electronic signals through computer technology, and combines them with various output devices to transform them into phenomena that people can experience.

[0003] In order to provide highly realistic virtual fitting services during clothing design and support multi-angle and multi-scene clothing effect display, VR-based virtual simulation can achieve seamless integration of product design, build an immersive virtual fitting experience, and enhance the immersion and interactivity of user experience; and the construction and rendering of VR scenes are the key to achieving an immersive experience. The current traditional clothing rendering method mainly relies on: relying on laser radar, scanning the surface of objects or scenes and measuring the time and intensity of reflected light to obtain three-dimensional data, and then rendering the obtained three-dimensional data through rendering software. The modeling and rendering processes are separated, the rendering speed is slow and the image quality is not high. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a VR-based clothing rendering method and system to solve the technical problems existing in the prior art.

[0005] The present invention proposes a VR-based clothing rendering method, comprising:

[0006] Acquire several images of clothing taken from different perspectives, perform feature matching and geometric information correction on the images, select one image for triangulation, and obtain a preliminary restored sparse point cloud structure map;

[0007] Perform pose estimation and beam optimization on the initially restored sparse point cloud structure graph, add images from other perspectives to the beam-optimized point cloud structure graph, perform triangulation and beam optimization, and obtain the point cloud clothing graph and corresponding pose data;

[0008] Generate a three-dimensional Gaussian with a point of the point cloud clothing image as the center, project the generated three-dimensional Gaussian onto the image plane using the pose data, calculate the light intensity from each point to the projection center based on the volume rendering algorithm, and obtain a preliminary rendered image;

[0009] Aliasing artifacts are eliminated from the preliminary rendered image to obtain a target clothing rendered image.

[0010] Optionally, the step of sequentially adding images from other perspectives to the beam-optimized point cloud structure image and performing triangulation and beam optimization to obtain a point cloud clothing image of the clothing and corresponding pose data includes:

[0011] Mapping several images into a multimodal semantic space using CLIP to obtain a coding vector corresponding to each image, and calculating the similarity between images corresponding to other perspectives and the initial triangulated image based on the coding vector;

[0012] The image with the highest similarity among the images corresponding to other shooting angles is selected as the conditional input for triangulation. Under the guidance of the initially restored sparse point cloud structure graph, triangulation is performed through the end-to-end generator model to obtain the current point cloud structure graph under the new perspective and perform pose estimation;

[0013] The current point cloud structure graph is used as the input of the fractional distillation sampling algorithm for iteration, and some light clusters in the current point cloud structure graph are selected in batches based on the frequency domain consistency error for beam optimization;

[0014] The remaining images in the images corresponding to other perspectives are triangulated and optimized in turn to obtain the point cloud clothing map of the clothing and the corresponding posture data.

[0015] Optionally, the sampling loss expression of the fractional distillation sampling algorithm is:

[0016]

[0017] Where, represents the multimodal semantic space, Represents the current point cloud structure diagram of the input, represents the model parameters, Indicates the ray cluster corresponding to the current point cloud structure diagram, () indicates the desired solution, represents the time step weight, represents the iterative condition vector, Indicates the image implicit vector corresponding to the current point cloud structure mapped from pixel space to implicit vector space, represents the noise estimate guided by the conditional vector fractional distillation sampling algorithm at time t, represents the noise estimate guided by the fractional distillation sampling algorithm without the conditional vector at time t.

[0018] Optionally, the step of projecting the generated three-dimensional Gaussian onto an image plane using the pose data and calculating the light intensity from each point to the projection center based on a volume rendering algorithm includes:

[0019] Based on the footprint function, the influence range of the voxel corresponding to each point in the point cloud clothing image during projection is determined, and then the overall contribution of each point in the point cloud clothing image to the image is determined;

[0020] Construct a projection ray corresponding to the image plane based on each point of the point cloud clothing image. Calculate the light intensity from the projection ray corresponding to each point to the projection center based on the sputtering diffusion equation of the volume rendering algorithm, thereby determining the color and brightness of each pixel projection.

[0021] The colors and brightness of all pixel projections are superimposed to obtain a preliminary rendered image.

[0022] Optionally, the sputtering diffusion equation is expressed as:

[0023]

[0024] Where, represents the wavelength of light, Indicates passing through the projection center point and plane points The projection ray of Indicates the distance on the ray from the projection center to the k point in the point cloud clothing image. represents the luminous coefficient of point k, is the extinction coefficient at point k, Represents the reconstruction kernel of k points, which is used to reflect the position and shape of individual particles. represents Taylor operation, is the extinction coefficient of point j in the point cloud clothing image, represents the reconstruction kernel of point j.

[0025] The present invention also proposes a VR-based clothing rendering system, comprising:

[0026] The measurement module obtains several images of clothing taken from different perspectives, performs feature matching and geometric information correction on the images, selects one image for triangulation, and obtains a preliminary restored sparse point cloud structure map;

[0027] The optimization module is used to perform pose estimation and beam optimization on the initially restored sparse point cloud structure map, and sequentially add images from other perspectives to the beam-optimized point cloud structure map and perform triangulation and beam optimization to obtain the point cloud clothing map of the clothing and the corresponding pose data;

[0028] a calculation module, configured to generate a three-dimensional Gaussian with a point of the point cloud clothing image as the center, project the generated three-dimensional Gaussian onto an image plane using the pose data, and calculate the light intensity from each point to the projection center based on a volume rendering algorithm to obtain a preliminary rendered image;

[0029] The elimination module is used to eliminate aliasing artifacts from the preliminary rendered image to obtain a target clothing rendering image.

[0030] Optionally, the optimization module includes:

[0031] An encoding unit is used to map several images into a multimodal semantic space through CLIP, obtain a coding vector corresponding to each image, and calculate the similarity between images corresponding to other perspectives and the initial triangulated image based on the coding vector;

[0032] The estimation unit is used to select the image with the highest similarity among the images corresponding to other shooting perspectives as the conditional input for triangulation. Under the guidance of the initially restored sparse point cloud structure graph, the end-to-end generator model performs triangulation to obtain the current point cloud structure graph under the new perspective and perform pose estimation;

[0033] The optimization unit is used to iterate the current point cloud structure map as the input of the fractional distillation sampling algorithm, and select some light clusters in the current point cloud structure map in batches based on the frequency domain consistency error for beam optimization;

[0034] The iterative unit is used to sequentially perform triangulation and beam optimization on the remaining images in the images corresponding to other perspectives to obtain a point cloud clothing image of the clothing and corresponding posture data.

[0035] Optionally, the sampling loss expression of the fractional distillation sampling algorithm is:

[0036]

[0037] Where, represents the multimodal semantic space, Represents the current point cloud structure diagram of the input, represents the model parameters, Indicates the ray cluster corresponding to the current point cloud structure diagram, () indicates the desired solution, represents the time step weight, represents the iterative condition vector, Indicates the image implicit vector corresponding to the current point cloud structure mapped from pixel space to implicit vector space, represents the noise estimate guided by the conditional vector fractional distillation sampling algorithm at time t, represents the noise estimate guided by the fractional distillation sampling algorithm without the conditional vector at time t.

[0038] Optionally, the calculation module includes:

[0039] A determination unit, configured to determine, based on a footprint function, an influence range of a voxel corresponding to each point in the point cloud clothing image when projecting, and further determine an overall contribution of each point in the point cloud clothing image to the image;

[0040] A construction unit is used to construct a projection ray corresponding to the image plane according to each point of the point cloud clothing image, and calculate the light intensity from the projection ray corresponding to each point to the projection center based on the sputtering diffusion equation of the volume rendering algorithm, thereby determining the color and brightness of each pixel projection;

[0041] The superposition unit is used to superimpose the colors and brightness of all pixel projections to obtain a preliminary rendered image.

[0042] Optionally, the sputtering diffusion equation is expressed as:

[0043]

[0044] Where, represents the wavelength of light, Indicates passing through the projection center point and plane points The projection ray of Indicates the distance on the ray from the projection center to the k point in the point cloud clothing image. represents the luminous coefficient of point k, is the extinction coefficient at point k, Represents the reconstruction kernel of k points, which is used to reflect the position and shape of individual particles. represents Taylor operation, is the extinction coefficient of point j in the point cloud clothing image, represents the reconstruction kernel of point j.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows: the VR-based clothing rendering method provided by the present application obtains several images of clothing taken from different perspectives, performs feature matching and geometric information correction on the several images, selects an image for triangulation to obtain a preliminary restored sparse point cloud structure map; performs pose estimation and beam optimization on the preliminary restored sparse point cloud structure map, adds images corresponding to other perspectives to the beam-optimized point cloud structure map in turn and performs triangulation and beam optimization to obtain a point cloud clothing map of the clothing and the corresponding pose data; first, constructs a point cloud clothing map of the clothing at a new perspective based on images at different perspectives; takes each point cloud clothing map as an example, and calculates the pose data of the clothing at a new perspective; A three-dimensional Gaussian is generated as the center of the point, and the generated three-dimensional Gaussian is projected onto the image plane using the shooting posture. The light intensity from each point to the projection center is calculated based on the volume rendering algorithm to obtain a preliminary rendered image; a three-dimensional Gaussian is generated for each point in the clothing point cloud clothing image based on the neural radiation field, and the projection is performed according to the shooting posture. Then, the light intensity from each point to the projection center is calculated based on the volume rendering algorithm to reduce the amount of rendering calculations. Finally, aliasing artifacts are eliminated from the preliminary rendered image to improve the quality of the rendered image and obtain the target clothing rendered image; the VR-based clothing rendering method provided in the present application performs rendering according to the point cloud clothing image, with low calculation amount, high rendering efficiency and rendering quality.

[0046] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of the VR-based clothing rendering method in Example 1 of the present invention;

[0048] Figure 2 This is a structural block diagram of a computer in Embodiment 4 of the present invention.

[0049] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0050] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0052] Example 1

[0053] See also Figure 1 , which shows a VR-based clothing rendering method in a first embodiment of the present invention, and the VR-based clothing rendering method specifically includes steps S10 to S40:

[0054] S10, obtaining a plurality of images of the clothing taken from different perspectives, performing feature matching and geometric information correction on the plurality of images, selecting an image for triangulation, and obtaining a preliminary restored sparse point cloud structure map;

[0055] In the specific implementation, we first obtain clothing images taken from different perspectives. Through multi-perspective shooting and image matching, we can capture the three-dimensional information of the clothing. Then, we use deep neural networks to simulate optical principles for modeling. The multi-layer perceptual radiation mechanism in the deep neural network is used to effectively represent the appearance of geometric figures related to perspective, and learn the color and density information of each point in the scene, so as to reconstruct the three-dimensional scene with high realism. By triangulating the multi-perspective images, we can obtain a preliminary recovery of the sparse point cloud structure map.

[0056] S20, performing pose estimation and beam optimization on the initially restored sparse point cloud structure graph, sequentially adding images from other perspectives to the beam-optimized point cloud structure graph and performing triangulation and beam optimization to obtain a point cloud clothing graph of the clothing and corresponding pose data;

[0057] The step of sequentially adding images from other perspectives to the beam-optimized point cloud structure image and performing triangulation and beam optimization to obtain a point cloud clothing image of the clothing and corresponding posture data includes:

[0058] Mapping several images into a multimodal semantic space using CLIP to obtain a coding vector corresponding to each image, and calculating the similarity between images corresponding to other perspectives and the initial triangulated image based on the coding vector;

[0059] The image with the highest similarity among the images corresponding to other shooting angles is selected as the conditional input for triangulation. Under the guidance of the initially restored sparse point cloud structure graph, triangulation is performed through the end-to-end generator model to obtain the current point cloud structure graph under the new perspective and perform pose estimation;

[0060] The current point cloud structure graph is used as the input of the fractional distillation sampling algorithm for iteration, and some light clusters in the current point cloud structure graph are selected in batches based on the frequency domain consistency error for beam optimization;

[0061] The remaining images in the images corresponding to other perspectives are triangulated and optimized in turn to obtain the point cloud clothing map of the clothing and the corresponding posture data.

[0062] The sampling loss expression of the fractional distillation sampling algorithm is:

[0063]

[0064] Where, represents the multimodal semantic space, Represents the current point cloud structure diagram of the input, represents the model parameters, Indicates the ray cluster corresponding to the current point cloud structure diagram, () indicates the desired solution, represents the time step weight, represents the iterative condition vector, Indicates the image implicit vector corresponding to the current point cloud structure mapped from pixel space to implicit vector space, represents the noise estimate guided by the conditional vector fractional distillation sampling algorithm at time t, represents the noise estimate guided by the fractional distillation sampling algorithm without the conditional vector at time t.

[0065] In this embodiment, images from different perspectives are mapped into a multimodal semantic space and then similarity is calculated. Images from different perspectives are sequentially used as conditional inputs for triangulation based on the degree of similarity. Guided by the initially restored sparse point cloud structure graph, triangulation is performed using an end-to-end generator model to obtain a current point cloud structure graph at the combined new perspective and perform pose estimation. The current point cloud structure graph is then subjected to batch beam optimization. For example, assuming a set of images from several perspectives is: A = (A1, A2, A3...), the point cloud structure graph corresponding to A1 is first obtained, and then A2 is used as the conditional input for a second triangulation. Guided by the point cloud structure graph corresponding to A1, a point cloud structure graph B1 is generated. The perspective of B1 is different from both A1 and A2, and it is a point cloud structure graph at a new perspective. The pose of B1 can be estimated using the COLMAP software. Then, A3 is selected as input, and guided by the point cloud structure graph corresponding to B1, a point cloud structure graph B2 is generated. This process continues in this manner, resulting in a final synthesized point cloud garment graph at the new perspective.

[0066] S30, generating a three-dimensional Gaussian with a point of the point cloud clothing image as the center, projecting the generated three-dimensional Gaussian onto the image plane using the pose data, and calculating the light intensity from each point to the projection center based on a volume rendering algorithm to obtain a preliminary rendered image;

[0067] Starting from the obtained point cloud structure diagram, the clothing is reconstructed and rendered based on 3D Gaussian sputtering rendering. The three-dimensional Gaussian can describe and express data in three-dimensional space, improving the richness of the data description; the parameters of the three-dimensional Gaussian can be used to obtain the exact position of the point and attribute information such as the shape and transparency of the point; for example, the coordinates of the three-dimensional Gaussian can describe the position of the point, the covariance matrix can describe the shape and direction, and the transparency can be used for rendering.

[0068] The step of projecting the generated three-dimensional Gaussian onto an image plane using the pose data and calculating the light intensity from each point to the projection center based on a volume rendering algorithm comprises:

[0069] Based on the footprint function, the influence range of the voxel corresponding to each point in the point cloud clothing image during projection is determined, and then the overall contribution of each point in the point cloud clothing image to the image is determined;

[0070] Construct a projection ray corresponding to the image plane based on each point of the point cloud clothing image. Calculate the light intensity from the projection ray corresponding to each point to the projection center based on the sputtering diffusion equation of the volume rendering algorithm, thereby determining the color and brightness of each pixel projection.

[0071] The colors and brightness of all pixel projections are superimposed to obtain a preliminary rendered image.

[0072] The expression of the sputtering diffusion equation is:

[0073]

[0074] Where, represents the wavelength of light, Indicates passing through the projection center point and plane points The projection ray of Indicates the distance on the ray from the projection center to the k point in the point cloud clothing image. represents the luminous coefficient of point k, is the extinction coefficient at point k, Represents the reconstruction kernel of k points, which is used to reflect the position and shape of individual particles. represents Taylor operation, is the extinction coefficient of point j in the point cloud clothing image, represents the reconstruction kernel of point j.

[0075] In this embodiment, the image plane is covered with the projected three-dimensional Gaussian footprint, and the projection in the 2D plane is linearly integrated based on volume reconstruction. By projecting the three-dimensional Gaussian corresponding to each point in the point cloud clothing image onto the 2D image, during volume integration rendering, only two-dimensional convolution calculation is required for each sampling point along the projection ray, which greatly improves the rendering efficiency compared to conventional object rendering that requires three-dimensional convolution.

[0076] S40, performing aliasing artifact removal on the preliminary rendered image to obtain a target clothing rendered image.

[0077] In a specific implementation, when the rendered image or a part of the image is sampled to a discrete raster grid, aliasing will occur during the rendering process due to the difference between the low sampling frequency and the high signal frequency, causing visual artifacts and affecting the quality of the rendered image. Therefore, the aliasing artifacts need to be eliminated. In a specific implementation, the artifact elimination can be performed based on a low-pass filter. The specific approach is: the light intensity of each pixel point is evenly distributed from the surrounding area to the pixel point itself to accurately approximate a single pixel. Compared with conventional low-pass filters, accurate approximation of a single pixel helps to reduce computational complexity, while removing high-frequency details in the rendered image, effectively reducing the aliasing effect, and improving the quality of the rendered image.

[0078] In summary, the VR-based clothing rendering method provided by the present application obtains several images of clothing taken from different perspectives, performs feature matching and geometric information correction on the several images, selects an image for triangulation to obtain a preliminary restored sparse point cloud structure map; performs pose estimation and beam optimization on the preliminary restored sparse point cloud structure map, adds images corresponding to other perspectives to the beam-optimized point cloud structure map in turn and performs triangulation and beam optimization to obtain a point cloud clothing map of the clothing and the corresponding pose data; first, a point cloud clothing map of the clothing at a new perspective is constructed based on images at different perspectives; a triangulation is generated with each point of the point cloud clothing map as the center Dimensional Gaussian, uses the shooting posture to project the generated three-dimensional Gaussian to the image plane, calculates the light intensity from each point to the projection center based on the volume rendering algorithm, and obtains a preliminary rendered image; generates a three-dimensional Gaussian for each point in the clothing point cloud clothing image based on the neural radiation field, projects it according to the shooting posture, and then calculates the light intensity from each point to the projection center based on the volume rendering algorithm, reducing the amount of rendering calculation, and finally eliminates aliasing artifacts on the preliminary rendered image to improve the quality of the rendered image and obtain the target clothing rendering image; the VR-based clothing rendering method provided in this application performs rendering according to the point cloud clothing image, with low calculation amount, high rendering efficiency and rendering quality.

[0079] Example 2

[0080] This embodiment provides a VR-based clothing rendering system, including:

[0081] The measurement module obtains several images of clothing taken from different perspectives, performs feature matching and geometric information correction on the images, selects one image for triangulation, and obtains a preliminary restored sparse point cloud structure map;

[0082] The optimization module is used to perform pose estimation and beam optimization on the initially restored sparse point cloud structure map, and sequentially add images from other perspectives to the beam-optimized point cloud structure map and perform triangulation and beam optimization to obtain the point cloud clothing map of the clothing and the corresponding pose data;

[0083] a calculation module, configured to generate a three-dimensional Gaussian with a point of the point cloud clothing image as the center, project the generated three-dimensional Gaussian onto an image plane using the pose data, and calculate the light intensity from each point to the projection center based on a volume rendering algorithm to obtain a preliminary rendered image;

[0084] The elimination module is used to eliminate aliasing artifacts from the preliminary rendered image to obtain a target clothing rendering image.

[0085] Optionally, the optimization module includes:

[0086] An encoding unit is used to map several images into a multimodal semantic space through CLIP, obtain a coding vector corresponding to each image, and calculate the similarity between images corresponding to other perspectives and the initial triangulated image based on the coding vector;

[0087] The estimation unit is used to select the image with the highest similarity among the images corresponding to other shooting perspectives as the conditional input for triangulation. Under the guidance of the initially restored sparse point cloud structure graph, the end-to-end generator model performs triangulation to obtain the current point cloud structure graph under the new perspective and perform pose estimation;

[0088] The optimization unit is used to iterate the current point cloud structure map as the input of the fractional distillation sampling algorithm, and select some light clusters in the current point cloud structure map in batches based on the frequency domain consistency error for beam optimization;

[0089] The iterative unit is used to sequentially perform triangulation and beam optimization on the remaining images in the images corresponding to other perspectives to obtain a point cloud clothing image of the clothing and corresponding posture data.

[0090] Optionally, the sampling loss expression of the fractional distillation sampling algorithm is:

[0091]

[0092] Where, represents the multimodal semantic space, Represents the current point cloud structure diagram of the input, represents the model parameters, Indicates the ray cluster corresponding to the current point cloud structure diagram, () indicates the desired solution, represents the time step weight, represents the iterative condition vector, Indicates the image implicit vector corresponding to the current point cloud structure mapped from pixel space to implicit vector space, represents the noise estimate guided by the conditional vector fractional distillation sampling algorithm at time t, represents the noise estimate guided by the fractional distillation sampling algorithm without the conditional vector at time t.

[0093] Optionally, the calculation module includes:

[0094] A determination unit, configured to determine, based on a footprint function, an influence range of a voxel corresponding to each point in the point cloud clothing image when projecting, and further determine an overall contribution of each point in the point cloud clothing image to the image;

[0095] A construction unit is used to construct a projection ray corresponding to the image plane according to each point of the point cloud clothing image, and calculate the light intensity from the projection ray corresponding to each point to the projection center based on the sputtering diffusion equation of the volume rendering algorithm, thereby determining the color and brightness of each pixel projection;

[0096] The superposition unit is used to superimpose the colors and brightness of all pixel projections to obtain a preliminary rendered image.

[0097] Optionally, the sputtering diffusion equation is expressed as:

[0098]

[0099] Where, represents the wavelength of light, Indicates passing through the projection center point and plane points The projection ray of Indicates the distance on the ray from the projection center to the k point in the point cloud clothing image. represents the luminous coefficient of point k, is the extinction coefficient at point k, Represents the reconstruction kernel of k points, which is used to reflect the position and shape of individual particles. represents Taylor operation, is the extinction coefficient of point j in the point cloud clothing image, represents the reconstruction kernel of point j.

[0100] Example 3

[0101] This embodiment provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the VR-based clothing rendering method as described above.

[0102] Example 4

[0103] The present invention also provides a computer, see Figure 2 , shown is a computer in an embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored in the memory 10 and executable on the processor 20. When the processor 20 executes the computer program 30, the above-mentioned VR-based clothing rendering method is implemented.

[0104] The memory 10 includes at least one type of storage medium, including flash memory, a hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of a computer, such as the computer's hard disk. In other embodiments, the memory 10 may also be an external storage device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the memory 10 may include both an internal storage unit of the computer and an external storage device. The memory 10 can be used not only to store application software installed in the computer and various types of data, but also to temporarily store data that has been output or is about to be output.

[0105] Among them, in some embodiments, the processor 20 can be an electronic control unit (Electronic Control Unit, abbreviated as ECU, also known as a vehicle computer), a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, used to run the program code stored in the memory 10 or process data, such as executing access restriction programs.

[0106] It should be pointed out that Figure 2 The structure shown does not constitute a limitation of the computer. In other embodiments, the computer may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0107] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0108] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting, or processing it in another suitable manner as necessary, and then storing it in a computer memory.

[0109] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0110] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0111] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A VR-based clothing rendering method, characterized in that: Acquire several images of clothing taken from different perspectives, perform feature matching and geometric information correction on the images, select one image for triangulation, and obtain a preliminary restored sparse point cloud structure map; Perform pose estimation and beam optimization on the initially restored sparse point cloud structure graph, add images from other perspectives to the beam-optimized point cloud structure graph, perform triangulation and beam optimization, and obtain the point cloud clothing graph and corresponding pose data; Generate a three-dimensional Gaussian with a point of the point cloud clothing image as the center, project the generated three-dimensional Gaussian onto the image plane using the pose data, calculate the light intensity from each point to the projection center based on the volume rendering algorithm, and obtain a preliminary rendered image; Eliminating aliasing artifacts on the preliminary rendered image to obtain a target clothing rendered image; The step of projecting the generated three-dimensional Gaussian onto an image plane using the pose data and calculating the light intensity from each point to the projection center based on a volume rendering algorithm comprises: Based on the footprint function, the influence range of the voxel corresponding to each point in the point cloud clothing image during projection is determined, and then the overall contribution of each point in the point cloud clothing image to the image is determined; Construct a projection ray corresponding to the image plane based on each point of the point cloud clothing image. Calculate the light intensity from the projection ray corresponding to each point to the projection center based on the sputtering diffusion equation of the volume rendering algorithm, thereby determining the color and brightness of each pixel projection. Superimpose the color and brightness of all pixel projections to obtain a preliminary rendered image; The expression of the sputtering diffusion equation is: Where, represents the wavelength of light, Indicates passing through the projection center point and plane points The projection ray of Indicates the distance on the ray from the projection center to the k point in the point cloud clothing image. represents the luminous coefficient of point k, is the extinction coefficient at point k, represents the reconstruction kernel of k points, which is used to reflect the position and shape of individual particles. represents Taylor operation, is the extinction coefficient of point j in the point cloud clothing image, represents the reconstruction kernel of point j.

2. The VR-based clothing rendering method according to claim 1, characterized in that: The step of sequentially adding images from other perspectives to the beam-optimized point cloud structure image and performing triangulation and beam optimization to obtain a point cloud clothing image of the clothing and corresponding posture data includes: Mapping several images into a multimodal semantic space using CLIP to obtain a coding vector corresponding to each image, and calculating the similarity between images corresponding to other perspectives and the initial triangulated image based on the coding vector; The image with the highest similarity among the images corresponding to other shooting angles is selected as the conditional input for triangulation. Under the guidance of the initially restored sparse point cloud structure graph, triangulation is performed through the end-to-end generator model to obtain the current point cloud structure graph under the new perspective and perform pose estimation; The current point cloud structure graph is used as the input of the fractional distillation sampling algorithm for iteration, and some light clusters in the current point cloud structure graph are selected in batches based on the frequency domain consistency error for beam optimization; The remaining images in the images corresponding to other perspectives are triangulated and optimized in turn to obtain the point cloud clothing map of the clothing and the corresponding posture data.

3. The VR-based clothing rendering method according to claim 2, characterized in that: The sampling loss expression of the fractional distillation sampling algorithm is: Where, represents the multimodal semantic space, Represents the current point cloud structure diagram of the input, represents the model parameters, Indicates the ray cluster corresponding to the current point cloud structure diagram, () indicates the desired solution, represents the time step weight, represents the conditional vector of the iteration, Indicates the image implicit vector corresponding to the current point cloud structure mapped from pixel space to implicit vector space, represents the noise estimate guided by the conditional vector fractional distillation sampling algorithm at time t, represents the noise estimate guided by the fractional distillation sampling algorithm without the conditional vector at time t.

4. A VR-based clothing rendering system, characterized in that: include: The measurement module obtains several images of clothing taken from different perspectives, performs feature matching and geometric information correction on the images, selects one image for triangulation, and obtains a preliminary restored sparse point cloud structure map; The optimization module is used to perform pose estimation and beam optimization on the initially restored sparse point cloud structure map, and sequentially add images from other perspectives to the beam-optimized point cloud structure map and perform triangulation and beam optimization to obtain the point cloud clothing map of the clothing and the corresponding pose data; a calculation module, configured to generate a three-dimensional Gaussian with a point of the point cloud clothing image as the center, project the generated three-dimensional Gaussian onto an image plane using the pose data, and calculate the light intensity from each point to the projection center based on a volume rendering algorithm to obtain a preliminary rendered image; an elimination module, configured to eliminate aliasing artifacts from the preliminary rendered image to obtain a target clothing rendered image; The calculation module includes: A determination unit, configured to determine, based on a footprint function, an influence range of a voxel corresponding to each point in the point cloud clothing image when projecting, and further determine an overall contribution of each point in the point cloud clothing image to the image; A construction unit is used to construct a projection ray corresponding to the image plane according to each point of the point cloud clothing image, and calculate the light intensity from the projection ray corresponding to each point to the projection center based on the sputtering diffusion equation of the volume rendering algorithm, thereby determining the color and brightness of each pixel projection; An overlay unit, used to overlay the colors and brightness of all pixel projections to obtain a preliminary rendered image; The expression of the sputtering diffusion equation is: Where, represents the wavelength of light, Indicates passing through the projection center point and plane points The projection ray of Indicates the distance on the ray from the projection center to the k point in the point cloud clothing image. represents the luminous coefficient of point k, is the extinction coefficient at point k, represents the reconstruction kernel of k points, which is used to reflect the position and shape of individual particles. represents Taylor operation, is the extinction coefficient of point j in the point cloud clothing image, represents the reconstruction kernel of point j.

5. The VR-based clothing rendering system according to claim 4, characterized in that: The optimization module includes: An encoding unit is used to map several images into a multimodal semantic space through CLIP, obtain a coding vector corresponding to each image, and calculate the similarity between images corresponding to other perspectives and the initial triangulated image based on the coding vector; The estimation unit is used to select the image with the highest similarity among the images corresponding to other shooting perspectives as the conditional input for triangulation. Under the guidance of the initially restored sparse point cloud structure graph, the end-to-end generator model performs triangulation to obtain the current point cloud structure graph under the new perspective and perform pose estimation; The optimization unit is used to iterate the current point cloud structure map as the input of the fractional distillation sampling algorithm, and select some light clusters in the current point cloud structure map in batches based on the frequency domain consistency error for beam optimization; The iterative unit is used to sequentially perform triangulation and beam optimization on the remaining images in the images corresponding to other perspectives to obtain a point cloud clothing image of the clothing and corresponding posture data.

6. The VR-based clothing rendering system according to claim 5, characterized in that: The sampling loss expression of the fractional distillation sampling algorithm is: Where, represents the multimodal semantic space, Represents the current point cloud structure diagram of the input, represents the model parameters, Indicates the ray cluster corresponding to the current point cloud structure diagram, () indicates the desired solution, represents the time step weight, represents the conditional vector of the iteration, Indicates the image implicit vector corresponding to the current point cloud structure mapped from pixel space to implicit vector space, represents the noise estimate guided by the conditional vector fractional distillation sampling algorithm at time t, represents the noise estimate guided by the fractional distillation sampling algorithm without the conditional vector at time t.

Images