The invention discloses a text-guided 3D knitted
product model generation and editing method and device, and relates to the field of
computer vision, and the method comprises the steps: S1, employing a 3D
diffusion model to enable a knitted product prompt text to generate an
initial point cloud; s2, optimizing the
initial point cloud and initializing the
initial point cloud into a 3D
Gaussian; s3, performing projection rendering on the initial 3D
Gaussian to obtain a multi-view
image sequence, and performing semantic segmentation on the multi-view
image sequence based on an editing prompt text to obtain a
mask sequence; s4, acquiring a 3D
Gaussian with a semantic tag based on the
mask sequence; s5, projecting the 3D Gaussian with the semantic tag to a 2D plane to obtain a rendered image; inputting the rendered image and the editing prompt text into a 2D
diffusion model, and outputting a loss gradient; and utilizing the loss gradient to guide the 3D Gaussian to carry out iteration, wherein the 3D Gaussian obtained after iteration is finished is the final 3D knitted
product model. According to the method, the 3D Gaussian is introduced, the semantic tag is added for the 3D Gaussian by using the editing prompt text, and the generation efficiency, the local semantic control and the high-precision editing are considered.