A method for engraving based on an artificial intelligence large model

Abstract

The application provides a kind of carving method based on artificial intelligence big model, it is related to handicraft carving technical field, including the following steps: from client access, the client includes consumer end and designer end;After selecting the material of carving product in the client, select the corresponding label or text for input and send to the server;The server will identify the label or text transmitted, then AIGC drawing is carried out;The drawing drawn by AIGC is fed back to the client, and the result can be regenerated when it is not satisfied;After obtaining the satisfactory result, the designer can generate 3D model by one key;After generating the model, the model is transmitted to the carving software, and the carving production can be entered, and completed.The beneficial aspect of the present application is that not only the loss of material can be avoided, but also a wide range of design scheme can be provided, and finally a visual design draft is formed, which greatly enhances the initial design, directly reduces a large amount of work and thus liberates the time of the designer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of handicraft carving, and in particular to a carving method based on an artificial intelligence large model. BACKGROUND

[0002] At present, the production of carving design types usually needs a large amount of pre-carving design work before the carving starts, along with the different needs of customers, material conditions and the like. After the initial step is completed by the carving designer, the design is delivered to the customer for confirmation. After a plurality of rounds of requirement changes and design adjustments such as "changing a material, changing a form, requiring a front view, requiring a mountain, requiring water", confirmation enters the carving link.

[0003] The current operation mode is to first perform graphic modeling on the design drawing, and then input into the software of the carving machine for machine carving. In this process, the initial design usually takes 3 days, and in the case of passing through a design, it takes 17 days from design to modeling to carving completion to be delivered to the consumer. However, this operation, on the one hand, is low in efficiency, and on the other hand, the design and the finished product are also affected by the personal cognition of the designer, thereby resulting in high customization cost. Finally, due to the difference in the expected effect of the finished product by the consumer, the return rate is high, thereby forming a vicious cycle.

[0004] At present, the market is roughly "designer hand-drawn line drawing", "two-dimensional design using drawing tool software", then "three-dimensional software modeling", and finally generating a knife path through another software to a carving machine for production. Mainly using JDPaint, ARTAM, RhinoGold 6 and the like to design and corresponding carving machine software to produce.

[0005] In the current carving design production process, the design part needs a lot of time to design when contacting the consumer. The design needs inspiration sources due to the individual level of the carver, the number of works, the process level and the characteristics of the material. The designer has very little practical experience in designing materials and some special features, which builds the threshold of the carver. Therefore, the processing of special materials and special features becomes the dividing line between ordinary carvers and master carvers. The consumer can only wait for the design scheme given by the carver within 3 days, and decide the final line drawing after communicating with the scheme. This also requires the consumer to understand the abstraction of the line drawing, which increases the threshold of the consumer, and the 3-day time period is no longer suitable for the current fast pace. The linear flow from line drawing to modeling to knife path and production has been used for a long time. Due to the small number of carvers and the even smaller number of masters, the industry has the phenomenon of slow and abstract output due to the scheduling of design.

[0006] Therefore, the application discloses a carving method based on an artificial intelligence large model, which can be packaged as an application or as a technical support core of other software, the Internet, etc., mainly realizes design and production in each link related to production of the carving design type, provides design assistance for designers, provides design and reference for consumers, and provides production for factories or manufacturers, so that the design time can be shortened in all aspects after use, the design time cost and labor cost are reduced, a plurality of schemes can be provided for consumers or designers to select, the model can be continuously learned, retrained, and iterated, and the model is enhanced; the generated effect picture is converted into a 3D model through interaction or automatically, a tool path is generated and provided to a carving machine, and carving production is performed. SUMMARY

[0007] The application overcomes the defects in the prior art and provides a carving method based on an artificial intelligence large model, which adopts an AIGC mode to make a carving designer more brave when facing high-end materials, model training from high-end materials provides a feasible design scheme, and the design scheme is put into production after being fully evaluated by the designer, wherein material loss avoidance is also included; the carving method can also provide a design draft scheme with a wide audience when facing mass production type design, greatly reduces the initial design cycle, and a scheme of about 10 seconds can be replaced at any time, and a pattern texture is superimposed on a material to finally form a visual design draft, which greatly enhances the initial design. The generated design draft can directly generate a 3D model and a tool path for direct production, thereby directly reducing a large amount of work and liberating the time of a designer.

[0008] The artificial intelligence large model is trained according to the design content of the industry, so that the AI can directly generate an effect picture and a 3D file for carving production according to the requirements of a user, thereby improving the diversity of design and the efficiency of the whole design and production process.

[0009] To solve the above technical problems, the application is implemented by the following technical scheme:

[0010] A carving method based on an artificial intelligence large model, comprising the following steps:

[0011] S1, accessing from a client, wherein the client comprises a consumer end and a designer end;

[0012] S2, selecting a material of a carving product in the client, then selecting and inputting a corresponding label or text and sending to a server;

[0013] S3, the server identifies the transmitted label or text, and then performs AIGC drawing;

[0014] S4, the graph drawn by the AIGC is fed back to the client, and the result can be regenerated when it is not satisfactory; after getting a satisfactory result, the designer can generate a 3D model with one key;

[0015] S5, after generating the model, the model is transmitted to the engraving software to enter the engraving production and complete.

[0016] Further, the server uses the AIGC method to reorganize the process, and then trains the AI model structure. Then, the words in the traditional design method, classification, are trained one by one, so that the AI fully understands these words; the words of the product material of engraving are also trained by the above method; finally, this technology is released as an application or packaged as a technical service to other Internet and software carriers.

[0017] Further, the model training includes data preparation and training process.

[0018] Further, the data preparation includes data collection and data screening, data preprocessing and data labeling, wherein,

[0019] Data collection and data screening refer to collecting about 150,000 high-quality image data through various ways, removing pictures that do not meet the requirements, and finally screening out about 15,000 high-quality image data for use;

[0020] Data preprocessing refers to preprocessing images, including but not limited to image cropping, scaling, color correction, background removal, normalization, etc. to facilitate input into the model; in addition, we also convert the image to Tensor format for model training;

[0021] Data labeling refers to using the common sense Q&A visual model to make a preliminary annotation of the data to be trained, and then performing secondary annotation and data verification by artificial means to build a high-quality data set.

[0022] Further, the training process includes the following:

[0023] S1, install the training script;

[0024] S2, configure the pre-training model, VAE model, load image data and labeled data, and regularization data;

[0025] S3, set the hyperparameters of model training, including learning rate scheduling, Optimizer optimizer, sampler batch size, data set training rounds, training precision, latent space learning rate, adding xformers cross attention mechanism, adding minimum signal-to-noise ratio, adding original noise offset, etc.

[0026] Further, the screening principle of high-quality image data is as follows:

[0027] a. To clearly reflect the texture and other detail features of jade articles;

[0028] b. On the basis of retaining the detail features of jade articles, other aspects are as various as possible; different angles, styles and the like should ensure the overall neatness of the picture, and the image should not have too much noise, blurring, unclear angle, ghosting and the like problems;

[0029] c. The image should ensure that the resolution is above 512*512 after being cropped to the main body of the jade article;

[0030] d. The image needs to be accurately classified according to the type of jade and the shape of the jade article to facilitate subsequent training.

[0031] Compared with the prior art, the beneficial effects of the present application are:

[0032] 1. The technical scheme provided by the present application greatly reduces the time of the initial design scheme, can provide a design scheme for a designer, can also be used for a consumer to customize a design scheme, supports text input for semantic recognition, and will generate a new effect drawing for the consumer or the designer through AI, and the design can be repeated until satisfaction, so that the final draft can be determined on the same day.

[0033] 2. In terms of shortening the whole process time, the modeling time is also saved, so that the whole process is shortened from more than 10 days to 2 days.

[0034] 3. The acceleration of the process also increases the consumer experience, shortens the transaction time and improves the transaction conversion rate.

[0035] 4. The visible effect drawing can not only be used for production, but also can be used as a basis for judgment when disputes arise.

[0036] 5. The continuously trained model will become more and more powerful, so that more personalized design requirements can also be met, and this part of the demand that was originally placed will be met.

[0037] 6. After generating the effect drawing, the designer can generate a producible 3D model through the function provided by the present patent; the whole process of 3D modeling is omitted, thereby greatly improving the work efficiency of the designer;

[0038] 7. The generated 3D model can be directly imported into the self-developed algorithm to automatically generate a tool path and enter the engraving production stage. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the present application, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application, in which:

[0040] Figure 1 is a flowchart of the carving method based on the artificial intelligence large model according to the present application;

[0041] Figure 2 is a flowchart of modern hand-made jade design and machining according to the present application;

[0042] Figure 3 is a reference diagram of the Stable Diffusion model structure according to the present application. DETAILED DESCRIPTION

[0043] The preferred embodiments of the present application are described below in conjunction with the accompanying drawings, and it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0044] As shown in Figure 1 , the present application claims a carving method based on an artificial intelligence large model, comprising the following steps:

[0045] S1, accessing from a client, the client including a consumer end and a designer end;

[0046] S2, after selecting the material of the carving product in the client, selecting the corresponding label or text for input and sending to the server;

[0047] S3, the server will identify the transmitted label or text, and then perform AIGC drawing;

[0048] S4, the AIGC drawing is fed back to the client, and when the result is not satisfactory, it can be regenerated; after obtaining a satisfactory result, the designer can generate a 3D model with one key;

[0049] S5, after generating the model, the model is transmitted to the carving software, and the carving production is started.

[0050] In the current production of carving design, a large amount of pre-carving design work is usually required before carving starts due to different customer needs, material conditions, etc. After the initial step is completed by the carving designer, it is delivered to the customer for confirmation; after several rounds of demand changes, design adjustments such as "change the material, change the form, want the front, want the mountain, want the water", etc., confirmation enters the carving link, the design drawing is first modeled, and then transmitted to the software of the carving machine for machine carving; this process is quite time-consuming.

[0051] Based on this, the inventors propose a carving method based on an artificial intelligence large model, which can be packaged as an application or as a technical support core of other software, the Internet, etc., mainly realizing the design and production parts in each link related to the production of carving design, providing design assistance for designers, providing design and reference for consumers, and providing production for factories or manufacturers. After use, it can fully shorten the design time, reduce the design time cost and labor cost, while providing multiple solutions for consumers or designers to switch and select, and can continuously learn, retrain, and continuously iterate the model to enhance the model. The generated rendering is converted to a 3D model through interaction or automatically, and the tool path is generated to the carving machine for carving production.

[0052] Specifically, the server uses the AIGC method to reorganize the process, and then trains the AI model structure. The words that appear in the traditional design method, such as "zodiac properties, front, mountains, and water", are trained one by one to allow the AI to fully understand these words. The words of the carving product material are also trained using the above method. Finally, this technology is released as an application or packaged as a technical service to other Internet and software carriers.

[0053] AIGC refers to (Artificial Intelligence Generated Content), which generally refers to the process of automatically generating content using artificial intelligence technology.

[0054] As shown in Figure 2 The AIGC method refers to the analysis and reorganization of modern handcrafted jade design and machining processes. It uses generative artificial intelligence to intervene in the design and production process. The 3D scanning uses open source technology (not limited to equipment, all 3D scanning tools on the market can be used) to scan the jade material into a 3D model, including the basic dimensions, surface color, volume, and other parameters of the jade material. Then select the appropriate surface screenshot (can be taken by 3D scanning model, or directly taken by mobile phone or scanner), then the AI picture generation model redraws the picture based on the input jade rough picture and prompt words to generate a jade effect picture.

[0055] There is also a reverse method, that is, input a picture (such as a child's portrait or cartoon image) to generate a desired jade effect picture.

[0056] Among them, 3D scanning uses open source technology (not limited to equipment, all 3D scanning tools on the market can be used) to scan the jade material into a 3D model, including the basic dimensions, surface color, volume, and other parameters of the jade material.

[0057] Then, the AI picture to 3D model is a picture generation model based on Diffusion Model.

[0058] The AI picture to 3D model has two kinds, one is also a picture generation model based on Diffusion Model, and the other is a 3D model generated by a single picture. Through a multi-view cross-domain attention mechanism, high-quality surfaces are extracted from multi-view two-dimensional representations, and a geometric perception normal fusion algorithm is combined to construct high-quality three-dimensional models.

[0059] Our model is based on a certain already fine-tuned stable diffusion base model, Figure 3 The Stable Diffusion model structure reference diagram is as follows, and the training steps and methods are as follows:

[0060] Premise: The general stable diffusion model includes three parts of the model: CLIP text encoder, an image automatic encoder decoder (or VAE), and a Unet structure model.

[0061] The CLIP model (Contrastive Language-Image Pre-Training) is a multi-modal pre-training neural network. This model can convert a variable-length text into a fixed-dimensional vector, which is used to embed the prompt text and then input it into the U-Net (the next model). This model is based on

[0062] The openai / clip-vit-large-patch14 model.

[0063] The U-net structure: As its network structure includes a stack of ResNet convolution matrices and Cross-Attention matrices. A simple explanation of Unet is that they learn the data distribution by adding noise step by step, and in turn generate new samples by removing noise step by step. The model we use contains about 3.5 billion parameters.

[0064] The VAE (Variational Auto Encoder) model is responsible for encoding images from the image space (pixel space encode) to the latent space and finally decoding them back.

[0065] Among them, the model training includes data preparation and the training process.

[0066] Data preparation includes data collection and data screening, data preprocessing and data labeling, wherein,

[0067] Data collection and data screening refer to collecting about 150,000 high-quality image data through various ways, eliminating pictures that do not meet the requirements, and finally screening out about 15,000 high-quality image data for use;

[0068] Data preprocessing refers to preprocessing the image, including but not limited to picture cropping, scaling, color correction, background removal, normalization, etc. to facilitate input into the model. In addition, we also convert the image into Tensor format for model training;

[0069] Data labeling refers to using the common sense visual model to make a preliminary annotation of the data to be trained, and then performing secondary annotation and data verification by artificial means to build a high-quality data set.

[0070] Further, the training process includes the following:

[0071] S1, install the training script (Kohya Trainer, open source, search

kohya

kohya ss github

[0072] S2, configure the pre-trained model, VAE model, load the image data and annotation data, and regularize the data;

[0073] S3, set the hyperparameters of the model training, including: learning rate scheduling (usually cosine), Optimizer optimizer (open source, AdamW), sampler (select Euler A) batch size, data set training rounds, training precision, latent space learning rate, add xformers cross attention mechanism, add minimum signal-to-noise ratio, add original noise offset and other parameters.

[0074] In this embodiment, the selection principles of high-quality image data are as follows:

[0075] a. To clearly reflect the texture and other detail features of jade articles;

[0076] b. On the basis of preserving the detail features of jade articles, other aspects should be as various as possible; different angles, styles, etc. should ensure the overall neatness of the picture, and the image should not have too much noise, blurring, unclear angles, ghosting and other problems;

[0077] c. To ensure that the image still has a resolution of more than 512*512 after being cropped to the main body of the jade article;

[0078] d. The image needs to be accurately classified according to the type of jade and the shape of the jade article to facilitate subsequent training.

[0079] The carving method based on the artificial intelligence large model can greatly reduce the time of the initial design scheme, can provide a design scheme for a designer, can also be used for a consumer to customize a design scheme, supports text input to perform semantic recognition, and then generates a new effect picture through AI and provides the consumer or the designer, and can repeatedly design until satisfaction, so that a final draft can be determined on the same day; the consumption experience of the consumer is increased, the transaction time is shortened, and the transaction conversion rate is improved; the visible effect picture can be used for production and can be used as a basis for judgment when a dispute occurs; in addition, the model that is continuously trained will become more and more powerful, so that more personalized design requirements can also be met, and the part of the requirements that is originally placed will be met; after the effect picture is generated, the designer can generate a 3D model that can be produced through the function provided by the patent.

[0080] Finally, it should be noted that: the above is only the preferred embodiment of the present application, and is not used to limit the present application, although the present application is described in detail with reference to the embodiments, for those skilled in the art, the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced, but any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A carving method based on a large-scale artificial intelligence model, characterized in that, Includes the following steps: S1. Access from a client, which includes a consumer client and a designer client; S2. After selecting the material of the engraved product in the client, select the corresponding label or text to enter and send it to the server; S3. The server will identify the transmitted tags or text and then perform AIGC drawing. The drawings generated by S4 and AIGC are fed back to the client. If the result is not satisfactory, the drawing is regenerated. Once a satisfactory result is obtained, the designer can generate a 3D model with one click. S5. After generating the model, transfer it to the carving software to begin carving production and complete the process. The server uses AIGC to re-organize the process, then trains the AI ​​with a structural model, and then trains the AI ​​to fully understand the vocabulary that appears in traditional design methods—classifications; the vocabulary for carving product materials is trained in the same way; finally, this technology is released as an application or packaged as a technical service to other Internet and software carriers. The model training includes data preparation and the training process; The data preparation includes data collection and filtering, data preprocessing, and data labeling.

2. The carving method based on a large artificial intelligence model according to claim 1, characterized in that, The data collection and data filtering refers to: collecting 150,000 high-quality image data through various methods, removing images that do not meet the requirements, and finally filtering out 15,000 high-quality image data for use.

3. The carving method based on a large artificial intelligence model according to claim 2, characterized in that, The data preprocessing refers to: preprocessing the images, including but not limited to image cropping, scaling, color correction, background removal, and normalization, so that they can be input into the model; in addition, we also convert the images into Tensor format for model training.

4. The carving method based on a large artificial intelligence model according to claim 3, characterized in that, The data annotation refers to: using the Tongyi Qianwen visual model to initially annotate these training materials, and then manually verifying the annotated data to construct a high-quality dataset.

5. The carving method based on a large artificial intelligence model according to claim 4, characterized in that, The training process includes the following: S1. Install the training script; S2. Configure the pre-trained model, VAE model, load image data and labeled data, and regularization data; S3. Set the hyperparameters for model training. Hyperparameter settings include: learning rate scheduling, optimizer, sampler batch size, dataset training epochs, training accuracy, latent space learning rate, adding xformers cross-attention mechanism, adding minimum signal-to-noise ratio, and adding raw noise offset parameter.

6. The carving method based on a large artificial intelligence model according to claim 5, characterized in that, The selection principles for high-quality image data are as follows: a. It should clearly show the detailed features of the jade's texture; b. While preserving the detailed features of the jade artifact, different angles and styles should ensure the overall cleanliness of the image. The image should not have excessive noise, be blurry, have unclear angles, or have ghosting issues. c. Ensure that the image still has a resolution of 512*512 or higher after being cropped to the main body of the jade artifact; d. Images need to be accurately classified according to the type of jade and the shape of the jade artifact to facilitate subsequent training.

Images