Diffusion model watermarking method and apparatus
Patent Information
- Application Number
- HK42026126366
- Authority / Receiving Office
- HK · HK
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-07-20
- Publication Date
- 2026-09-11
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202410979812.9 (22) Application Date 2024.07.19 (71) Applicant Huawei Cloud Computing Technology Co., Ltd. Address 550025, Guizhou Province, Guiyang City, Gui'an New District, Qianzhong Avenue, Xinggong Road, Huawei Cloud Data Center (72) Inventors Chang Heng, Liu Li, Li Likun (74) Patent Agency Beijing Zhongbo Shida Patent & Trademark Agency Co., Ltd. 11274 Patent Attorney Zhang Na (51) Int.Cl. G06T 1 / 00 (2006.01) G06N 3 / 0455 (2023.01) G06N 3 / 084 (2023.01) G06N 3 / 096 (2023.01) (54) Invention Title: A Method and Apparatus for Watermark Embedding a Diffusion Model (57) Abstract: A method and apparatus for watermark embedding a diffusion model are disclosed, relating to the field of artificial intelligence technology. A computing device acquires a first diffusion model and trains the first diffusion model according to the encoder in the teacher model to obtain an intermediate diffusion model. The intermediate diffusion model is then trained according to the decoder in the teacher model to obtain a second diffusion model. The intermediate diffusion model can generate a 3D model with a first watermark, and the second diffusion model can generate a 3D model with a second watermark. The matching degree between the second watermark and the target watermark is greater than that between the first watermark and the target watermark. By segmenting and adjusting the first diffusion model through the encoder and decoder, the first diffusion model learns the features of the stage of generating a 3D model with a target watermark. The resulting second diffusion model can retain the model parameters of the generated 3D model as much as possible while updating the model parameters of the generated watermark, thereby improving the usability of the second diffusion model. Claims (3 pages), Description (19 pages), Drawings (7 pages), CN 121366071 A, 2026.01.20, CN 1 21 36 60 71 A. 1. A diffusion model watermark embedding method, characterized in that the method comprises: obtaining a first diffusion model, the first diffusion model being used to generate a three-dimensional model; training the first diffusion model according to an encoder in a teacher model to obtain an intermediate diffusion model; the encoder being used to generate a three-dimensional model with a target watermark, the intermediate diffusion model being capable of generating a three-dimensional model with a first watermark; training the intermediate diffusion model according to a decoder in the teacher model to obtain a second diffusion model; the decoder being used to extract a watermark from the three-dimensional model, the second diffusion model being capable of generating a three-dimensional model with a second watermark, the matching degree between the second watermark and the target watermark being greater than the matching degree between the first watermark and the target watermark.2. The method according to claim 1, characterized in that, before training the first diffusion model according to the encoder in the teacher model to obtain the intermediate diffusion model, the method further includes: providing a user interface, the user interface including a first control component; and acquiring the target watermark in response to a user triggering operation on the first control component. 3. The method according to claim 1 or 2, characterized in that acquiring the first diffusion model includes: providing a user interface, the user interface including a second control component; and acquiring the first diffusion model in response to a user triggering operation on the second control component. 4. The method according to claim 1 or 2, characterized in that acquiring the first diffusion model includes: acquiring the first diffusion model from a plurality of reference diffusion models, each of the plurality of reference diffusion models being used to generate a three-dimensional model. 5. The method according to any one of claims 1 to 4, characterized in that the teacher model is trained using a training dataset and the target watermark, the training dataset including different types of three-dimensional models, the types including one or more of the following: transportation vehicles, office equipment, and buildings. 6. The method according to any one of claims 1 to 5, characterized in that, training the first diffusion model according to the encoder in the teacher model to obtain an intermediate diffusion model includes: inputting a first 3D model in the fine-tuning dataset into the encoder and the first diffusion model, and outputting a first reconstructed 3D model and a second reconstructed 3D model; the first 3D model is any one of a plurality of 3D models included in the fine-tuning dataset; updating the model parameters of the first diffusion model according to the loss between the first reconstructed 3D model and the second reconstructed 3D model to obtain the intermediate diffusion model. 7. The method according to any one of claims 1 to 6, characterized in that, training the intermediate diffusion model according to the decoder in the teacher model to obtain a second diffusion model includes: inputting the second 3D model in the fine-tuning dataset into the intermediate diffusion model to obtain a third reconstructed 3D model; the third reconstructed 3D model includes the first watermark, and the second 3D model is any one of a plurality of 3D models included in the fine-tuning dataset; inputting the third reconstructed 3D model into the decoder to obtain the first watermark; updating the model parameters of the intermediate diffusion model according to the loss between the first watermark and the target watermark to obtain the second diffusion model. 8. The method according to any one of claims 1 to 7, characterized in that the encoder comprises a local feature extraction network and a global feature extraction network. 9. The method according to claim 8, characterized in that the local feature extraction network comprises a graph convolutional neural network; and / or,The global feature extraction network includes the graph convolutional neural network. 10. A diffusion model watermark embedding device, characterized in that the device comprises: a first acquisition module, configured to acquire a first diffusion model, the first diffusion model being used to generate a three-dimensional model; a first training module, configured to train the first diffusion model according to an encoder in a teacher model to obtain an intermediate diffusion model; the encoder being used to generate a three-dimensional model with a target watermark, the intermediate diffusion model being capable of generating a three-dimensional model with a first watermark; a second training module, configured to train the intermediate diffusion model according to a decoder in the teacher model to obtain a second diffusion model; the decoder being used to extract a watermark from the three-dimensional model, the second diffusion model being capable of generating a three-dimensional model with a second watermark, the matching degree between the second watermark and the target watermark being greater than the matching degree between the first watermark and the target watermark. 11. The device according to claim 10, characterized in that the device further comprises: a second acquisition module; the second acquisition module being configured to provide a user interface, the user interface including a first control component, and in response to a user's trigger operation on the first control component, acquiring the target watermark. 12. The apparatus according to claim 10 or 11, wherein the first acquisition module is specifically configured to provide a user interface, the user interface including a second control component, and acquire the first diffusion model in response to a user triggering operation on the second control component. 13. The apparatus according to claim 10 or 11, wherein the first acquisition module is specifically configured to acquire the first diffusion model from a plurality of reference diffusion models, each of the plurality of reference diffusion models being used to generate a three-dimensional model. 14. The apparatus according to any one of claims 10 to 13, wherein the teacher model is obtained by training using a training dataset and the target watermark, the training dataset including different types of three-dimensional models, the types including one or more of the following: transportation vehicles, office equipment, and buildings. 15. The apparatus according to any one of claims 10 to 14, wherein the first training module is specifically configured to input a first three-dimensional model from a fine-tuning dataset into an encoder and a first diffusion model, output a first reconstructed three-dimensional model and a second reconstructed three-dimensional model, and update the model parameters of the first diffusion model according to the loss between the first reconstructed three-dimensional model and the second reconstructed three-dimensional model to obtain the intermediate diffusion model, wherein the first three-dimensional model is any one of the plurality of three-dimensional models included in the fine-tuning dataset. 16. The apparatus according to any one of claims 10 to 15, characterized in that the second training module is specifically used to input the second three-dimensional model in the fine-tuning dataset into the intermediate diffusion model to obtain the third reconstructed three-dimensional model, andThe third reconstructed 3D model is input into the decoder to obtain the first watermark. Based on the loss between the first watermark and the target watermark, the model parameters of the intermediate diffusion model are updated to obtain the second diffusion model. The third reconstructed 3D model includes the first watermark, and the second 3D model is any one of the multiple 3D models included in the fine-tuning dataset. 17. The apparatus according to any one of claims 10 to 16, wherein the encoder includes a local feature extraction network and a global feature extraction network. 18. The apparatus according to claim 17, wherein the local feature extraction network includes a graph convolutional neural network; and / or, the global feature extraction network includes the graph convolutional neural network. 19. A computing device cluster, comprising at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform the method according to any one of claims 1 to 9. 20. A computer-readable storage medium, characterized in that the storage medium stores a computer program or instructions, which, when executed by a computing device, implement the method of any one of claims 1 to 9. 21. A computer program product, comprising a computer program or instructions, characterized in that, when executed by a computing device, the computer program or instructions implement the method of any one of claims 1 to 9. Claims 3 / 3 Page 4 CN 121366071 A Diffusion Model Watermarking Embedding Method and Apparatus Technical Field
[0001] This application relates to the field of artificial intelligence technology, and more particularly to a diffusion model watermarking embedding method and apparatus. Background Art
[0002] Model watermarking refers to the technique of embedding specific markers or information into a neural network model in order to identify the owner of the model or prove the source of the model. By fine-tuning the model parameters of the diffusion model used to generate the image through a watermark extraction network (decoder), the image generated by the diffusion model can be processed by the watermark extraction network to obtain the corresponding watermark.
[0003] However, using a watermark extraction network to fine-tune the parameters of a diffusion model can lead to a decrease in the image quality generated by the diffusion model and a reduction in the usability of the fine-tuned diffusion model because a large number of model parameters in the diffusion model are adjusted to generate the watermark. Summary of the Invention
[0004] This application provides a diffusion model watermark embedding method and apparatus to solve the problem of fine-tuning a watermark extraction network.The parameters of the diffusion model may degrade the image quality generated by the diffusion model and reduce the usability of the fine-tuned diffusion model because a large number of model parameters in the diffusion model are adjusted to generate watermarks.
[0005] This application adopts the following solution.
[0006] In a first aspect, this application provides a diffusion model watermark embedding method. The method can be executed by a computing device, a chip (such as a processor) in the computing device, or a cluster of computing devices composed of multiple computing devices. The following description takes the execution of the diffusion model watermark embedding method by a computing device as an example. The diffusion model watermark embedding method includes: the computing device obtains a first diffusion model, trains the first diffusion model according to the encoder in the teacher model to obtain an intermediate diffusion model, and trains the intermediate diffusion model according to the decoder in the teacher model to obtain a second diffusion model. The first diffusion model is used to generate a 3D model, the encoder is used to generate a 3D model with a target watermark, the decoder is used to extract the watermark from the 3D model, the intermediate diffusion model can generate a 3D model with a first watermark, and the second diffusion model can generate a 3D model with a second watermark. The matching degree between the second watermark and the target watermark is greater than that between the first watermark and the target watermark.
[0007] In this application, the computing device adjusts the first diffusion model in segments through the encoder and decoder in the teacher model, so that the first diffusion model learns the features of different stages, such as the stage of generating a 3D model with a target watermark and the stage of extracting the watermark from the 3D model. The resulting second diffusion model can retain the model parameters of the generated 3D model as much as possible, and can also update the model parameters of the generated watermark, thereby improving the usability, i.e., robustness, of the second diffusion model.
[0008] In one possible case, the above-mentioned computing device is a computing device provided by a cloud service provider, that is, the above-mentioned diffusion model watermark embedding method can be applied to a cloud computing platform and executed by one or more computing devices included in the cloud computing platform.
[0009] In one possible example, the above-described diffusion model watermark embedding method can be a service on a cloud computing platform. Users can call this service to embed a watermark into the first diffusion model to obtain a second diffusion model.
[0010] In one possible scenario, the above-described computing device can be a computing device located on the user side. Specification 1 / 19 pages 5 CN 121366071 A
[0011] In one possible scenario, the above-described target watermark can be a predefined watermark by the computing device. This predefined watermark can be randomly generated by the computing device, or it can be generated by the computing device based on user information.
[0012] For example, the computing device uses the identity document (ID) from the user information as the target watermark.
[0013] In one possible scenario, the above-described first diffusion model is a pre-trained diffusion model.
[0014] In one possible scenario, the above-mentioned three-dimensional model is represented by any of the following: point cloud, polygon mesh, voxel mesh.
[0015] In one possible implementation, before the computing device trains the first diffusion model according to the encoder in the teacher model to obtain the intermediate diffusion model, the above-mentioned diffusion model watermark embedding method further includes: the computing device providing a user interface, the user interface including a first control component, and acquiring a target watermark in response to the user's trigger operation on the first control component.
[0016] In this application, the computing device acquires the target watermark determined by the user by responding to the user's trigger operation on the first control component on the display interface, which not only achieves visualization but also meets the user's customization needs for the target and improves the user experience.
[0017] In one possible example, the computing device displays a user interface at its connected front end, which may be a monitor or a terminal located on the user side, which can access the user interface provided by the computing device through the application programming interface (API) provided by the computing device.
[0018] In one possible scenario, the target watermark includes one or more of the following: image, text, audio, video, bitstream.
[0019] In one possible example, the watermark in the 3D model is represented as a bitstream. In other words, after obtaining a target watermark in the form of an image, text, audio, or video, the computing device converts the target watermark in the form of an image, text, audio, or video into a target watermark in the form of a bitstream, and then embeds the target watermark in the form of a bitstream into the first diffusion model to obtain the second diffusion model.
[0020] Regarding the method by which the computing device obtains the first diffusion model, two possible implementations are provided below.
[0021] In one possible implementation, the computing device obtains the first diffusion model by: the computing device providing a user interface, the user interface including a second control component, and obtaining the first diffusion model in response to a user's trigger operation on the second control component.
[0022] In one possible example, the computing device displays the user interface at its connected front end.
[0023] In this application, the computing device obtains a user-defined first diffusion model by responding to a user's trigger operation on a second control component on the display interface. This achieves visualization while also enabling watermark injection into the user-specified first diffusion model, allowing for targeted processing of the user-specified first diffusion model and making the resulting second diffusion model easier to trace.
[0024] In another possible implementation, the computing device obtains the first diffusion model by: obtaining the first diffusion model from multiple reference diffusion models, each of which is used to generate a three-dimensional model.
[0025] In this application, when the user does not provide a first diffusion model, the computing device can obtain the first diffusion model from multiple reference diffusion models, and then perform watermark embedding processing on the first diffusion model to obtain the user's own second diffusion model. When others use the user's second diffusion model, the source can be traced based on the watermark in the three-dimensional model generated by the second diffusion model, reducing the difficulty of obtaining evidence of others using the second diffusion model. Specification 2 / 19 pages 6 CN 121366071 A
[0026] In one possible example, before the computing device executes the diffusion model watermark embedding method provided in this application, multiple reference diffusion models have been deployed in the computing device. Therefore, the computing device can select one of the multiple reference diffusion models as the first diffusion model.
[0027] In one possible case, the above-mentioned reference diffusion model is a pre-trained diffusion model.
[0028] The above selection method can be: random selection, selection according to the user's instructions, or selection according to the labels determined by the user. Each of the multiple reference diffusion models has one or more labels.
[0029] In one possible scenario, the differences between the above-mentioned multiple reference diffusion models may lie in different application scenarios and differences in diffusion model structures.
[0030] For example, the first reference diffusion model among the multiple reference diffusion models is used to generate 3D models of office supplies, and the second reference diffusion model is used to generate 3D models of transportation vehicles.
[0031] Compared with the network structure of the fourth reference diffusion model, the network structure of the third reference diffusion model among the multiple reference diffusion models adds convolutional layers, pooling processing layers, feature extraction network layers, etc.
[0032] In one possible implementation, the teacher model is trained using a training dataset and a target watermark. The training dataset includes different types of 3D models, including one or more of the following: transportation vehicles, office supplies, and buildings.
[0033] In this application, the computing device trains the teacher model using a training dataset including different types of 3D models and a target watermark, which can improve the robustness of the encoder in the teacher model in generating 3D models with target watermarks. Then, the encoder in the teacher model is used to train the first diffusion model, which can improve the robustness of the finally obtained second diffusion model.
[0034] In one possible example, the above types may also include: animals, plants, clothing, etc.
[0035] In one possible example, the computing device uses the training dataset and the target watermark to jointly train the encoder and decoder in the teacher model.
[0036] In one possible implementation, the computing device trains the first diffusion model based on the encoder in the teacher model to obtain an intermediate diffusion model, including: the computing device inputs the first 3D model from the fine-tuning dataset into the encoder and...The first diffusion model outputs a corresponding first reconstructed 3D model and a second reconstructed 3D model, and updates the model parameters of the first diffusion model based on the loss between the first reconstructed 3D model and the second reconstructed 3D model to obtain an intermediate diffusion model. The first 3D model is any one of the multiple 3D models included in the fine-tuning dataset.
[0037] In this application, the computing device updates the model parameters of the first diffusion model based on the loss between the first reconstructed 3D model and the second reconstructed 3D model, enabling the intermediate diffusion model to learn the ability to generate a 3D model with a target watermark. This continuously increases the matching degree between the first reconstructed 3D model and the second reconstructed 3D model generated by the first diffusion model, improving the quality of the 3D model generated by the intermediate model, and thus enhancing the robustness of the final second diffusion model.
[0038] In one possible example, the first 3D model has a watermark.
[0039] In another possible example, the first 3D model does not have a watermark.
[0040] In one possible case, the 3D model in the fine-tuning dataset is a discrete point cloud, i.e., it does not have a fixed shape.
[0041] In one possible implementation, the computing device trains the intermediate diffusion model based on the decoder in the teacher model to obtain a second diffusion model, including: the computing device inputs the second three-dimensional model in the fine-tuning dataset into the intermediate diffusion model to obtain a third reconstructed three-dimensional model, and inputs the third reconstructed three-dimensional model into the decoder to obtain a first watermark, and then updates the model parameters of the intermediate diffusion model according to the loss between the first watermark and the target watermark, to obtain the second diffusion model. The third reconstructed three-dimensional model includes the first watermark, and the second three-dimensional model is any one of the multiple three-dimensional models included in the fine-tuning dataset.
[0042] In this application, the computing device optimizes the model parameters of the intermediate diffusion model based on the loss between the first watermark and the target watermark, so that the watermark in the three-dimensional model generated by the final second diffusion model has a higher matching degree with the target watermark, thereby improving the quality of the watermark in the three-dimensional model generated by the second diffusion model.
[0043] In one possible implementation, the encoder includes a local feature extraction network and a global feature extraction network.
[0044] In one possible example, the local feature extraction network includes a graph convolutional neural network; and / or, the global feature extraction network includes a graph convolutional neural network.
[0045] In one possible implementation, if a 3D model is generated using the second diffusion model, the watermark in the 3D model can be extracted using the decoder in the aforementioned teacher model, thereby enabling rapid tracing of the second diffusion model's origin, i.e., identifying the owner of the second diffusion model, reducing the difficulty of obtaining evidence of others using the aforementioned second diffusion model.
[0046] Secondly, this application provides a diffusion model watermark embedding device. This diffusion model watermark embedding device is applied to a computer system (such as a computer cluster) or to a computing device that supports the computer system in implementing the diffusion model watermark embedding method. The diffusion model watermark embedding device includes modules for executing the diffusion model watermark embedding method in the first aspect or any optional implementation of the first aspect. For example, the diffusion model watermark embedding device includes: a first acquisition module, a first training module, and a second training module.
[0047] The first acquisition module is used to acquire a first diffusion model, which is used to generate a three-dimensional model.
[0048] The first training module is used to train the first diffusion model according to the encoder in the teacher model to obtain an intermediate diffusion model; the encoder is used to generate a three-dimensional model with a target watermark, and the intermediate diffusion model can generate a three-dimensional model with the first watermark.
[0049] The second training module is used to train the intermediate diffusion model according to the decoder in the teacher model to obtain the second diffusion model; the decoder is used to extract the watermark in the three-dimensional model, and the second diffusion model can generate a three-dimensional model with the second watermark. The matching degree between the second watermark and the target watermark is greater than the matching degree between the first watermark and the target watermark.
[0050] More detailed implementation details of the diffusion model watermark embedding device can be found in the description of any implementation of the first aspect above, as well as the content of the following specific embodiments, which will not be repeated here.
[0051] In a third aspect, this application provides a chip, including: a processor and a power supply circuit; the power supply circuit is used to supply power to the processor, and the processor is used to execute the method in the first aspect above or any possible implementation of the first aspect; and / or, the processor is used to execute the method in the second aspect above or any possible implementation of the second aspect.
[0052] In a fourth aspect, this application provides a computing device. The computing device includes a memory and a processor. The memory stores computer instructions. When the processor executes the computer instructions, it implements the method in the first aspect or any possible implementation of the first aspect. And / or, when the processor executes the computer instructions, it implements the method in the second aspect or any possible implementation of the second aspect.
[0053] In a fifth aspect, this application provides a computing device cluster. The computing device cluster includes at least one computing device, which includes a memory and a processor. The memory stores computer instructions. When the processor executes the computer instructions, it implements the method in the first aspect or any possible implementation of the first aspect.
[0054] In a sixth aspect, this application provides a computer-readable storage medium storing a computer program or instructions. When the computer program or instructions are executed by a processing device, they implement the method in the first aspect or any possible implementation of the first aspect.The method in one of the possible implementations; and / or, when the computer program or instructions are executed by the processing device, the method in any one of the possible implementations of the second aspect or the second aspect described above.
[0055] In a seventh aspect, this application provides a computer program product, which includes a computer program or instructions, which, when executed by the processing device, implement the method in the first aspect or any one of the possible implementations of the first aspect; and / or, when the computer program or instructions are executed by the processing device, implement the method in the second aspect or any one of the possible implementations of the second aspect.
[0056] The beneficial effects of the second to seventh aspects can be referred to the first aspect or any one of the possible implementations of the first aspect, and will not be repeated here. Based on the implementations provided in the above aspects, this application can also make further combinations to provide more implementations.
[0057] Figure 1 is a schematic diagram of the architecture of a computer system provided in this application;
[0058] Figure 2 is a schematic diagram of the structure of a chip provided in this application;
[0059] Figure 3 is a flowchart of a diffusion model watermark embedding method provided in this application;
[0060] Figure 4 is a schematic diagram of a user interface provided in this application;
[0061] Figure 5 is a schematic diagram of the model structure of a teacher model provided in this application;
[0062] Figure 6 is a flowchart of a teacher model training method provided in this application;
[0063] Figure 7 is a schematic diagram of a user interface provided in this application;
[0064] Figure 8a is a flowchart of a distillation learning method provided in this application;
[0065] Figure 8b is a flowchart of a diffusion model verification method provided in this application;
[0066] Figure 9 is a schematic diagram of the structure of a diffusion model watermark embedding device provided in this application;
[0067] Figure 10 is a schematic diagram of the structure of a diffusion model watermark embedding device provided in this application;
[0068] Figure 11 is a schematic diagram of the structure of a computing device cluster provided in this application;
[0069] Figure 12 is a schematic diagram of the connection between computing devices provided in this application. Detailed Implementation
[0070] To address the problem that when using a watermark extraction network to fine-tune the parameters of a diffusion model, many model parameters in the fine-tuned diffusion model are adjusted to generate watermarks, resulting in a deterioration in the image quality generated by the fine-tuned diffusion model and a decrease in the usability of the fine-tuned diffusion model, this application provides a diffusion model watermark embedding method. The computing device performs segmented adjustments to the first diffusion model through the encoder and decoder in the teacher model, enabling the first diffusion model to learn features at different stages, such as the encoder generating features of the 3D model stage with the target watermark, and the decoder extracting the watermark level from the 3D model.The characteristics of the segment, and thus the second diffusion model can not only retain the model parameters of the generated three-dimensional model as much as possible, but also update the model parameters of the generated watermark, thereby improving the usability, i.e., robustness of the second diffusion model.
[0071] For ease of understanding, the technical terms involved in this application will be introduced first.
[0072] Watermark: The content that reflects the source of the diffusion model. The watermark may include, but is not limited to, one or more of the following: images, text, audio, video, bitstream. The watermark may be the content specified by the owner / provider / generator / producer of the diffusion model.
[0073] Diffusion model: A generative model that learns the distribution of underlying data by adding noise and iterative denoising. In this application, the diffusion model can generate a three-dimensional model based on text.
[0074] Model parameters: The parameters that can be learned and adjusted inside the model, used to describe the relationship between data features and target variables.
[0075] The following, in conjunction with the accompanying drawings, will exemplarily illustrate the scenarios in which the embodiments of this application can be applied.
[0076] Figure 1 is a schematic diagram of the architecture of a computer system provided in this application. The computer system 100 includes a computing device 110. The computing device 110 may be an electronic device with computing capabilities or a virtual device with computing capabilities. When the computing device 110 is an electronic device with computing capabilities, the computing device 110 may be a server, a personal computer, a tablet computer, etc. When the computing device 110 is a virtual device with computing capabilities, the computing device 110 may be a virtual machine, a container, etc.
[0077] In one possible example, the user may input a first diffusion model and a target watermark into the computing device 110, and the computing device 110 may train a teacher model using training data and the target watermark, and then use the trained teacher model to fine-tune the first diffusion model, and the computing device 110 may output a second diffusion model (i.e., the fine-tuned first diffusion model).
[0078] In another possible example, a user can input a target watermark into the computing device 110, which then trains a teacher model using training data and the target watermark. The trained teacher model is then used to fine-tune a first diffusion model among multiple reference diffusion models, and the computing device 110 outputs a second diffusion model.
[0079] The computing device 110 includes a communication interface 114, a processor 111, and a memory 112. The communication interface 114 is used to communicate with devices located outside the computing device 110. For example, the computing device 110 receives a first diffusion model or a target watermark through the communication interface 114. After processing the teacher model (e.g., training the teacher model) using the received target watermark, the computing device 110 outputs the processing result (e.g., a second diffusion model) through the communication interface 114. This communication interface 114 can be an input / output interface.Input / output (I / O) interface.
[0080] The processor 111 is the computing core and control core of the computing device 110. It may include: a central processing unit (CPU), a specific integrated circuit, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. In practical applications, the computing device 110 may also include multiple processors. The processor 111 may include one or more processor cores. An operating system and other software programs are installed in the processor 111, so that the processor 111 can access the memory 112 and various peripheral component interconnect express (PCIe) devices.
[0081] The processor 111 is connected to the memory 112 through a double data rate (DDR) bus or other types of buses. The memory 112 is the main memory of the computing device 110. Memory 112 is typically used to store various running software in the operating system, received input data, and output results obtained from processing the input data. To improve the access speed of the processor 111, memory 112 needs to have a high access speed. In traditional computer devices, dynamic random access memory (DRAM) is typically used as memory 112. Besides DRAM, memory 112 can also be other random access memories, such as static random access memory (SRAM). Alternatively, memory 112 can also be read-only memory (ROM). For example, read-only memory can be programmable read-only memory (PROM) or erasable programmable read-only memory (EPROM). This embodiment does not specify the number of memories 112.The type is limited.
[0082] Optionally, in order to persistently store the data, the computer system 100 is also provided with a data storage system 113. The data storage system 113 may be located outside the computing device 110 (as shown in Figure 1) and exchange data with the computing device 110 through a network. Optionally, the data storage system 113 may also be located inside the host, such as the data storage system 113 exchanging data with the processor 111 through the bus 116. In this case, the data storage system 113 is manifested as a hard disk.
[0083] Optionally, the computer system 100 may also include a client device 120. The user can input the first diffusion model or target watermark to the computing device 110 through the client device 120, and the computing device 110 can send the processing result (such as the second diffusion model) to the user through the client device 120. The client device 120 may be a terminal device, including but not limited to personal computers, servers, mobile phones, tablets, or smart cars.
[0084] Optionally, the computer system 100 may further include an acceleration device 115. The acceleration device 115 is used to perform the watermark embedding task of the first diffusion model. The processor 111 sends the received watermark embedding task to the acceleration device 115, and the acceleration device 115 sends the processing result (such as the second diffusion model) to the processor 111 after completing the watermark embedding task according to the training data. As shown in FIG1, the acceleration device 115 can be directly inserted into the card slot on the motherboard of the computing device 110 and exchange data with the processor 111 through the bus 116. It should be noted that the bus 116 in FIG1 can also be replaced by a bus of the acceleration device 115 using the Compute Express Link (CXL), Universal Serial Bus (USB) protocol or other protocols for data transmission.
[0085] In addition, the above-mentioned acceleration device 115 may not be directly inserted into the card slot on the motherboard of the computing device 110, but may be located in the acceleration device. For example, the acceleration device may be a device independent of the computing device 110, such as an acceleration card. At this time, the computing device 110 can connect to the acceleration device 115 via a wired network such as a network cable, or via a wireless network such as a Wi-Fi hotspot or Bluetooth. If the acceleration device 115 is used to complete the watermark embedding task of the first diffusion model, for example, by training the teacher model using training data and then fine-tuning the first diffusion model using the teacher model, the acceleration device 115 can be implemented by one or more chips. These chips may include a CPU, a graphics processing unit (GPU), and a neural network processor.Any one of the following: NPU, Tensor Processing Unit (TPU), FPGA, and ASIC. Among them, GPU, also known as display core, visual processor, or display chip, is a microprocessor specifically designed for image processing on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones). NPU simulates human neurons and synapses at the circuit level and directly processes a large number of neurons and synapses using a deep learning instruction set, with one instruction completing the processing of a group of neurons. ASIC is suitable for integrated circuit products with a single purpose.
[0086] For example, the processor 111 in FIG1 can be implemented by a chip, as shown in FIG2. FIG2 is a schematic diagram of the structure of a chip provided in this application. For example, the chip 200 includes a core 201, a CPU 202, a system buffer 203, a DDR 204, and an input / output (I / O) device 205.
[0087] Wherein, the CPU 202 is used to accept AI tasks (such as the watermark embedding task of the first diffusion model, etc.) and call the core 201 to execute the task. In the case where chip 200 has multiple cores 201, CPU 202 is also used to undertake scheduling tasks. For example, CPU 202 can be implemented by an advanced RISC machine (ARM) processor, which is small in size, low in power consumption, and uses a 32-bit reduced instruction set, making addressing simple and flexible. Of course, in some embodiments, CPU 202 can also be implemented by other processors.
[0088] Core 201 is used to provide the computing power required in the watermark embedding task of the first diffusion model. In an optional case, core 201 includes a load / store unit (LSU), a cube computing unit, a scalar computing unit, a vector computing unit, and a buffer. Among them, the LSU is used to load the data to be processed and store the processed data, and can also be used for reading and writing data between different buffers within the core, as well as performing some format conversion operations. The cube computing unit is used to provide the core computing power for matrix multiplication. A scalar computation unit is a single-instruction single-data (SISD) processor. As stated on page 7 / 19 of this specification (CN 121366071 A type processor), it processes only one data item (usually an integer or floating-point number) at a time. A vector computation unit, also known as an array processor, is a processor capable of directly manipulating an array or vector for computation. The number of buffers may be one or more; for example, this buffer primarily refers to the level 1 cache.L1 buffer is used to temporarily store some data that the core 201 needs to use repeatedly, thereby reducing read and write operations from the bus. In addition, the implementation of certain data format conversion functions also requires the source data to be located in the buffer. In this embodiment, since the buffer is located in the core, the distance between the cube computing unit in the core and the storage area where the data is located is shortened, reducing the access of the cube computing unit to DDR204, thereby reducing the data access latency and the core data processing latency.
[0089] The system buffer 203 mainly refers to the level 2 buffer (L1 buffer or L2 cache), which is used to temporarily store the input data (such as training data), intermediate results or final results (such as the second diffusion model) that have passed through the chip.
[0090] DDR204 is an off-chip memory, which can also be replaced by high bandwidth memory (HBM) or other off-chip memory. DDR204 is located between the chip and the external memory, overcoming the access speed limitation when reading and writing shared computing resources.
[0091] The I / O device 205 included in the chip 200 refers to the hardware that performs data transmission, and can also be understood as a device that interfaces with the I / O interface. Common I / O devices include network cards, printers, keyboards, mice, etc. All external storage can also be used as I / O devices, such as hard disks, floppy disks, optical disks, etc.
[0092] The core 201, CPU 202, system buffer 203, DDR 204, and I / O device 205 are connected by a bus. The bus may include a path for transmitting information between the above components (such as CPU 202 and system buffer 203). In addition to the data bus, the bus may also include a power bus, control bus, and status signal bus, etc. However, for clarity, the bus may be a PCIe bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. For example, core 201 can access these I / O devices 205 via the PCIe bus. Core 201 is connected to system buffer 203 via the DDR bus. Here, different system buffers 203 may use different data buses to communicate with core 201; therefore, the DDR bus can also be replaced with other types of data buses. This application embodiment does not limit the bus type.
[0093] For example, when the CPU 202 loads the data to be processed by the AI task (such as the first diffusion model) into the DDR 204, the LSU in the core 201 reads (loads) the data from the DDR 204, trains the first diffusion model, and obtains the processing result (such as the second diffusion model). After obtaining the processing result, the LSU loads (stores) the processing result into the DDR 204, and the network interface card sends the processing result to the client device 120, or to the data storage system 113 for persistent storage.
[0094] It is worth noting that the acceleration device 115 shown in FIG1 can also be implemented by the chip 200 shown in FIG2, and this application does not limit it in this way.
[0095] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the computing device. In other embodiments, the computing device and the chip may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components. The components illustrated can be implemented in hardware, software, or a combination of software and hardware.
[0096] In the embodiments of this application, a teacher model may be deployed in the computing device or the processor (or chip) in the computing device, or other neural network models or algorithm models with three-dimensional model generation functions may be deployed, such as a reference diffusion model. This application does not limit this. In this case, the computing device receives the first diffusion model specification 8 / 19 pages 12 CN 121366071 A and the target watermark input by the user to the computing device, and then uses the target watermark and training data to train the encoder and decoder in the teacher model, thereby using the encoder and decoder in the teacher model to perform segmented fine-tuning of the first diffusion model to obtain the second diffusion model.
[0097] The implementation of the diffusion model watermark embedding method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0098] Figure 3 is a schematic flowchart of a diffusion model watermark embedding method provided in this application. The diffusion model watermark embedding method can be executed by a computing device or a chip or processor in the computing device. The computing device may be the computing device 110, client device 120 or acceleration device 115 shown in Figure 1. The chip or processor in the computing device can be the chip shown in Figure 2, etc. The hardware implementation of the computing device can be referred to the description in Figure 1 above, and the implementation of the chip or processor can be referred to the description in Figure 2 above, and will not be repeated here. In some optional examples, the diffusion model watermark embedding method can also be executed by other computing devices or computer clusters. The hardware implementation of the computing device can be referred to the descriptions in Figures 1 and 2 above, and will not be repeated here. The computer cluster includes multiple computing devices.
[0099] In one possible scenario, the above-mentioned computing device 110 can be one of N computing devices managed by a cloud computing platform.One or more, where N is an integer greater than or equal to 2.
[0100] In one possible case, the computing device 110 may be a terminal, server, etc.
[0101] Here, the example of the diffusion model watermark embedding method provided in this embodiment being executed by the computing device 110 is used for illustration. As shown in FIG3, the diffusion model watermark embedding method provided in this embodiment includes the following steps S310-S330.
[0102] S310, the computing device 110 obtains a first diffusion model.
[0103] Wherein, the first diffusion model is used to generate a three-dimensional model.
[0104] In one possible case, the first diffusion model is a pre-trained diffusion model. That is, the first diffusion model can generate a corresponding three-dimensional model according to the prompt word.
[0105] In one possible example, the prompt word is airplane, and the first diffusion model can generate a three-dimensional model of airplane according to the aforementioned prompt word.
[0106] In one possible example, the prompt word is train, and the first diffusion model can generate a three-dimensional model of train according to the aforementioned prompt word.
[0107] It is worth noting that the first diffusion model being a pre-trained diffusion model is only a possible scenario provided in this embodiment and should not be construed as a limitation of this application. In other embodiments of this application, the first diffusion model may also be an untrained diffusion model.
[0108] In one possible scenario, the above-mentioned three-dimensional model may be represented by any of the following: point cloud, polygon mesh, voxel mesh.
[0109] Regarding the acquisition of the content of the first diffusion model by the computing device 110, two possible implementation methods are provided below.
[0110] In the first possible implementation method, the computing device 110 provides a user interface including a second control component, and acquires the first diffusion model in response to the user's trigger operation on the second control component.
[0111] In one possible example, the client device 120 accesses the computing device 110 through the API provided by the computing device 110 to display the user interface on the terminal. Then, the computing device 110 acquires the user's trigger operation on the second control component on the user interface through various input devices (keyboard, mouse, touch screen, etc.) connected to the client device 120.
[0112] As shown in Figure 4, Figure 4 is a schematic diagram of the user interface provided in this application. This user interface schematic diagram includes a second control component for uploading the first diffusion model and a control component for exporting the second diffusion model.
[0113] For the specific implementation of the trigger operation, three possible examples are provided below.
[0114] Example 1: The trigger operation can be a user confirmation of the second control component via the keyboard, such as the user pressing the Enter key on page 9 / 19 of the specification (CN 121366071 A).
[0115] Example 2: The trigger operation can be a user clicking the second control component via the mouse.
[0116] For example, when a user drags a file containing the first diffusion model to a designated location on the user interface, the terminal sends the aforementioned file containing the first diffusion model to the computing device 110, and the computing device 110 directly obtains the first diffusion model.
[0117] As another example, after the user clicks the second control component on the user interface, a floating window is displayed on the user interface provided by the computing device 110. This floating window includes the storage path of the first diffusion model (such as the storage path of the first diffusion model 1, the first diffusion model 2, the first diffusion model 3, ..., the first diffusion model n). After the user selects a specified first diffusion model (such as the first diffusion model 1) on the floating window, the terminal sends the aforementioned storage path of the first diffusion model 1 to the computing device 110 by clicking the "OK" control component with the mouse, and the computing device 110 obtains the first diffusion model 1 from the storage path of the first diffusion model 1. This storage path indicates the storage location of the first diffusion model.
[0118] Example 3: This triggering operation can be an input operation by the user through the keyboard in the second control component, such as inputting the corresponding storage path.
[0119] The above examples are only optional content provided in this embodiment and should not be construed as limiting this application. In other embodiments of this application, the triggering operation can also be a remote operation, voice control, knuckle tapping, etc.
[0120] In a second possible implementation, the computing device 110 obtains a first diffusion model from a plurality of reference diffusion models, each of which is used to generate a three-dimensional model.
[0121] In one possible case, the plurality of reference diffusion models are all pre-trained diffusion models.
[0122] The difference between the plurality of reference diffusion models is that they have different usage scenarios and different model structures.
[0123] In one possible example, the first reference diffusion model in the plurality of reference diffusion models is used to generate a three-dimensional model of office supplies, and the second reference diffusion model is used to generate a three-dimensional model of transportation vehicles. In other words, different types of training sets are used when training the reference diffusion models.
[0124] Compared to the network structure of the fourth reference diffusion model, the network structure of the third reference diffusion model among the multiple reference diffusion models adds convolutional layers, pooling layers, feature extraction network layers, etc.
[0125] Regarding the acquisition of the content of the first diffusion model from the multiple reference diffusion models by the computing device 110, three possible examples are provided below.
[0126] Example 1: The computing device 110 randomly determines the first diffusion model from the multiple reference diffusion models.
[0127] Example 2: The computing device 110 determines the first diffusion model from the multiple reference diffusion models according to the user's instructions.
[0128] For example, the user interface provided by the computing device 110 includes a control component that includes information about the multiple reference diffusion models.In response to the user's trigger operation on the aforementioned control component, a first diffusion model among multiple reference diffusion models is obtained.
[0129] Example 3: The computing device 110 determines the first diffusion model from multiple reference diffusion models based on the tags determined by the user.
[0130] Each of the multiple reference diffusion models has one or more tags.
[0131] The computing device 110 determines at least one reference diffusion model from multiple reference diffusion models that satisfies the tags determined by the user, wherein the tags of the at least one reference diffusion model match the tags determined by the user. If the at least one reference diffusion model has only one reference diffusion model, the computing device 110 uses that reference diffusion model as the first diffusion model. If the at least one reference diffusion model has multiple reference diffusion models, the computing device 110 determines the content of the first diffusion model, which can be referred to the descriptions in Examples 1 and 2 above, and will not be repeated here.
[0132] S320: The computing device 110 trains the first diffusion model based on the encoder in the teacher model to obtain the diffusion model 14 CN 121366071 A on pages 10 / 19 of the intermediate specification.
[0133] The encoder described above is used to generate a 3D model with a target watermark, and the intermediate diffusion model can generate a 3D model with a first watermark.
[0134] In one possible implementation, the teacher model described above is trained using a training dataset and a target watermark. The training dataset includes different types of 3D models, including one or more of the following types: transportation vehicles, office supplies, and buildings.
[0135] In one possible case, the above types may also include: animals, plants, clothing, toys, and daily necessities.
[0136] For details of this implementation, please refer to the description in Figure 6 below, which will not be repeated here.
[0137] In one possible implementation, the computing device 110 trains the first diffusion model according to the encoder in the teacher model to obtain an intermediate diffusion model, including:
[0138] The computing device 110 transfers the ability of the encoder to generate a 3D model with a target watermark to the first diffusion model through model distillation, thereby adjusting the model parameters of the first diffusion model to obtain an intermediate diffusion model (the first diffusion model after model parameter adjustment).
[0139] Model distillation, also known as knowledge distillation, refers to fixing the model parameters of a pre-trained teacher model and, based on a predetermined loss, causing the first diffusion model to gradually approach the performance characteristics (generating a 3D model with the target watermark) of the teacher model during training. After training reaches a preset number of rounds or the loss converges, an intermediate diffusion model is obtained.
[0140] The aforementioned loss refers to the difference between the encoder in the teacher model and the different output results obtained by the first diffusion model for the same input.
[0141] For details of this implementation, please refer to Figure 8a below, which will not be elaborated here.
[0142] S330, the computing device 110 trains the intermediate diffusion model according to the decoder in the teacher model to obtain the second diffusion model.
[0143] The decoder is used to extract the watermark in the three-dimensional model. The second diffusion model can generate a three-dimensional model with the second watermark. The matching degree between the second watermark and the target watermark is greater than the matching degree between the first watermark and the target watermark.
[0144] In one possible implementation, the computing device 110 trains the intermediate diffusion model according to the decoder in the teacher model to obtain the second diffusion model, including:
[0145] The computing device 110 uses the decoder to extract the watermark from the output result of the intermediate diffusion model to obtain the first watermark. Then, based on the loss between the first watermark and the target watermark, the parameters of the intermediate diffusion model are updated to obtain the second diffusion model.
[0146] For details of this implementation, please refer to Figure 8a below, which will not be elaborated here.
[0147] In one possible embodiment, this application provides a network structure of a teacher model. First, as shown in Figure 5, Figure 5 is a schematic diagram of the model structure of a teacher model provided in this application.
[0148] The teacher model 500 shown in Figure 5 includes an encoder 510 and a decoder 520.
[0149] The encoder 510 includes: a local feature extraction network 511, a global feature extraction network 512, a convolutional layer 513, a convolutional layer 514, a pooling layer 515, and a multi-layer perceptron (MLP) 516.
[0150] The decoder 520 includes: a global feature extraction network 521, a pooling layer, a convolutional layer 522, a pooling layer 523, and an MLP 524.
[0151] Furthermore, a convolutional layer 530 and a convolutional layer 540 are respectively provided before the encoder 510 and the decoder 520.
[0152] In one possible case, the local feature extraction network 511 includes graph convolutional networks (GCN).
[0153] In one possible scenario, the global feature extraction network 512 includes a GCN. Specification 11 / 19 pages 15 CN 121366071 A
[0154] In one possible scenario, the global feature extraction network 521 includes a GCN.
[0155] In one possible scenario, the pooling layer 515 is a max-pooling layer.
[0156] In one possible scenario, the pooling layer 523 is a max-pooling layer.
[0157] It is worth noting that in FIG. 5, fl represents the vector obtained after processing by the convolutional layer 513 in the encoder 510, represents the vector obtained after processing by the pooling layer 515 in the encoder 510, and represents the vector obtained after processing by the pooling layer 523 in the decoder 520.The processed vector, m represents the target watermark, and represents the predicted watermark.
[0158] The above convolutional layers 513, 514, 530, and 540 may have the same or different sizes (size of the convolutional kernel), which is not limited in this application.
[0159] In one possible embodiment, based on the model structure of the teacher model shown in FIG5, a training method for the teacher model is provided below. As shown in FIG6, FIG6 is a flowchart of the training method for the teacher model provided in this application. The method includes the following steps S610-S630.
[0160] S610, the computing device 110 acquires the training dataset and the target watermark.
[0161] The training dataset includes different types of three-dimensional models.
[0162] In one possible case, the three-dimensional models in the training dataset do not have watermarks.
[0163] Regarding the content of the training dataset acquired by the computing device 110, three possible implementation methods are provided below.
[0164] In a first possible implementation, the data storage system 113 stores a training dataset, and the computing device 110 retrieves the training dataset from the data storage system 113.
[0165] In a second possible implementation, the memory 112 stores a training dataset, and the computing device 110 retrieves the training dataset from the memory 112.
[0166] In a third possible implementation, the computing device 110 retrieves the training dataset sent by the client device 120.
[0167] Regarding the content of the target watermark retrieved by the computing device 110, three possible implementations are provided below.
[0168] In a first possible implementation, the computing device 110 uses a preset watermark as the target watermark.
[0169] In one possible case, the user interface provided by the computing device 110 includes the preset target watermark.
[0170] In a second possible implementation, the computing device 110 uses a randomly generated watermark as the target watermark.
[0171] In one possible case, the user interface provided by the computing device 110 includes the randomly generated target watermark.
[0172] In a third possible implementation, the computing device 110 acquires the target watermark sent by the client device 120.
[0173] In one possible scenario, as shown in FIG7, FIG7 is a schematic diagram of the user interface provided in this application. The user interface shown in FIG7 adds a first control component for receiving the target watermark input by the user to the user interface shown in FIG4. The computing device 110 acquires the target watermark sent by the client device 120 in response to the user's trigger operation on the first control component.
[0174] In other words, the user interface provided by the computing device 110 includes a first control component and a second control component.
[0175] In one possible example, the above-mentioned user interface also includes a control component for specifying the watermark. If the user triggers the control component, the user interface can receive the target watermark.If the user selects "Yes" in the control component, the user needs to input the target watermark in the first control component; if the user selects "No" in the control component, the computing device 110 obtains the target watermark using the first possible implementation or the second possible implementation.
[0176] For example, if the client device 120 sends user information to the computing device 110 when accessing the computing device 110, the computing device 110 uses the ID in the user information as the target watermark, and the user interface provided by the computing device 110 includes the target watermark.
[0177] It is worth noting that the target watermark obtained by the computing device 110 can be in the form of: image, text, audio, video, bitstream.
[0178] S620, the computing device 110 inputs the training dataset and the target watermark into the teacher model, the encoder outputs a 3D model with the watermark, and the decoder outputs the predicted watermark;
[0179] The computing device 110 divides the data in the training dataset into multiple batches of data, and then inputs the multiple batches of data into the teacher model sequentially for training.
[0180] In one possible case, before inputting the target watermark into the teacher model, the computing device 110 converts the target watermark from a representation of: image, text, audio, and video into a representation of: bitstream.
[0181] In one possible example, the computing device 110 can encode images, text, audio, and video to obtain the corresponding bitstream.
[0182] The following description uses the input of one batch of data from multiple batches of data into the teacher model for training as an example. This batch of data includes a 3D model A.
[0183] The computing device 110 inputs the 3D model A into the teacher model 500. The computing device 110 extracts feature A from the 3D model through convolutional layer 530, and then inputs the feature into local feature extraction network 511 and global feature extraction network 512 respectively to obtain the corresponding features B and C. The computing device 110 processes feature B using convolutional layer 513 to obtain fl, and processes feature B using convolutional layer 514 to obtain feature D. Feature D is processed by pooling layer 515 to obtain the feature f after being fused with the target watermark. The computing device 110 inputs fl and the feature f after being fused with the target watermark into MLP to output the 3D model with watermark.
[0184] The computing device 110 further inputs the 3D model with watermark into convolutional layer 540 to extract feature E of the 3D model with watermark. Feature E is processed by global feature extraction network 521 to obtain feature F. The computing device 110 inputs feature F into convolutional layer 522 to obtain feature G, and inputs feature G into pooling layer 523 to obtain the predicted watermark. The computing device 110 inputs the predicted watermark into MLP.
[0185] It is worth noting that the target watermark input to the teacher model is represented as a bitstream, such as 01001011.
[0186] It is worth noting that in the actual training process, each batch of data will include a large number of three-dimensional models.
[0187] S630, the computing device 110 updates the encoder's model parameters according to the loss A between the three-dimensional model in the training dataset and the corresponding three-dimensional model with watermark, and updates the encoder's model parameters and the decoder's model parameters according to the loss B between the predicted watermark and the target watermark, thereby obtaining the trained teacher model.
[0188] The trained teacher model is the teacher model shown in Figure 3.
[0189] In one possible case, the above loss A can be calculated by Euclidean distance, chamfer distance (CD), mean squared error, cross-entropy loss function, etc., which is not limited in this application.
[0190] The following is an example of the computing device 110 using Euclidean distance to calculate the loss between the three-dimensional model in the training dataset and the corresponding three-dimensional model with watermark.
[0191] The 3D model in the training dataset and the 3D model with watermark are represented as point clouds. The 3D model in the training dataset includes multiple point clouds 1, and the 3D model with watermark includes multiple point clouds 2. The computing device 110 uses Euclidean distance to determine the distance between point cloud 1 and point cloud 2, and then determines the above-mentioned loss A based on the distance between multiple point clouds 1 and multiple point clouds 2. The set of point clouds 1 and point clouds 2 with determined distances has the following correspondence: the position of point cloud 1 in the 3D model in the above-mentioned training dataset corresponds to the position of point cloud 2 in the above-mentioned 3D model with watermark.
[0192] For example, the position of point cloud 1 in the 3D model in the above-mentioned training dataset is: the outermost position of the upper right corner; the position of point cloud 2 in the above-mentioned 3D model with watermark is: the outermost position of the upper right corner.
[0193] In one possible example, the reciprocal of the sum of the distances between multiple point clouds 1 and multiple point clouds 2 is the loss A. Instruction manual, pages 13 / 19, CN 121366071 A
[0194] In one possible example, the computing device 110 calculates the distance between multiple point clouds 1 and multiple point clouds 2 according to a preset formula to obtain a loss A.
[0195] In one possible case, the above-mentioned loss B can be calculated by Hamming distance, edit distance, cross-entropy loss function, etc., which is not limited in this application.
[0196] The computing device 100 can use the loss A (Lrecon) between the 3D model A and the 3D model with watermark to update the parameters of the encoder 510, and use the loss B (Lw) between the predicted watermark and the target watermark to update the parameters of the teacher model (encoder and decoder).
[0197] The above content is only one round of training in the multi-round training of the teacher model by the computing device 110. The content of each round of training in the multi-round training can be referred to the above description, and will not be repeated here. The computing device 110 performs multiple rounds of training on the teacher model until the above Lrecon and / or Lw converge, the data in the training dataset is used up, or the set number of training rounds is reached, thereby obtaining the trained teacher model.
[0198] Regarding the above content of obtaining the intermediate diffusion model in S320 and obtaining the second diffusion model in S330, the following provides a possible embodiment. As shown in Figure 8a, Figure 8a is a schematic flowchart of the distillation learning method provided in this application. The content shown in Figure 8a may include the following steps S810-S850.
[0199] S810, the computing device 110 inputs the first three-dimensional model in the fine-tuning dataset into the encoder and the first diffusion model, and outputs the first reconstructed three-dimensional model and the second reconstructed three-dimensional model.
[0200] Wherein, the first three-dimensional model is any one of the multiple three-dimensional models included in the fine-tuning dataset.
[0201] In one possible scenario, the 3D model in the fine-tuning dataset does not have a watermark. The fine-tuning dataset in this scenario can refer to the content of the training dataset described above, and will not be repeated here.
[0202] In another possible scenario, the 3D model in the fine-tuning dataset has a watermark. The watermark can be any bitstream content, such as 1010111, which is not limited in this application.
[0203] In one possible implementation, the computing device 110 inputs the first 3D model in the fine-tuning dataset into the encoder and the first diffusion model, and outputs a first reconstructed 3D model and a second reconstructed 3D model, including: the computing device 110 inputs the first 3D model into the encoder, outputs the first reconstructed 3D model, and inputs the first 3D model into the first diffusion model, and outputs the second reconstructed 3D model.
[0204] The first reconstructed 3D model and the second reconstructed 3D model described above have watermarks.
[0205] S820, the computing device 110 updates the model parameters of the first diffusion model according to the loss between the first reconstructed 3D model and the second reconstructed 3D model to obtain an intermediate diffusion model.
[0206] In one possible implementation, the computing device 110 updates the model parameters of the first diffusion model based on the loss between the first reconstructed 3D model and the second reconstructed 3D model to obtain an intermediate diffusion model. This includes: the computing device 110 calculates the loss (ldenoising) between the first reconstructed 3D model and the second reconstructed 3D model, and then uses this loss to backpropagate the first diffusion model to update its model parameters, thereby obtaining the intermediate diffusion model.
[0207] Ldenoising can be calculated using Euclidean distance, Chiffon distance, mean squared error, cross-entropy loss function, etc., and is not limited thereto in this application.
[0208] It is worth noting that the above description only uses one three-dimensional model (the first three-dimensional model) as an example. In the actual training process, the computing device 110 will use the fine-tuning dataset for multiple rounds of training. In each round of training, the computing device 110 inputs a batch of data into the encoder and the first diffusion model, outputs multiple sets of reconstructed three-dimensional models, and then updates the model parameters of the first diffusion model according to the loss between the multiple sets of reconstructed three-dimensional models to obtain the intermediate diffusion model.
[0209] The above-mentioned batch of data includes at least one three-dimensional model. Each time a batch of data is input into the encoder and the first diffusion model, the computing device 110 will update the model parameters of the first diffusion model once. That is, the model parameters of the first diffusion model will be updated once for each round of training until the loss between the reconstructed three-dimensional models (such as the first reconstructed three-dimensional model and the second reconstructed three-dimensional model) converges, the set number of training rounds is reached, or all three-dimensional models in the fine-tuning dataset are used up, and the intermediate diffusion model is obtained.
[0210] S830, the computing device 110 inputs the second 3D model in the fine-tuning dataset into the intermediate diffusion model to obtain a third reconstructed 3D model.
[0211] Wherein, the third reconstructed 3D model includes a first watermark, and the second 3D model is any one of the multiple 3D models included in the fine-tuning dataset.
[0212] In one possible example, the computing device 110 inputs a batch of data in the fine-tuning dataset into the intermediate diffusion model to obtain multiple third reconstructed 3D models. A batch of data in the fine-tuning dataset includes multiple 3D models, and the multiple 3D models include a second 3D model.
[0213] For example, the fine-tuning dataset includes multiple batches of data, and each batch of data includes at least one 3D model.
[0214] S840, the computing device 110 inputs the third reconstructed 3D model into the decoder in the teacher model to obtain a first watermark.
[0215] The computing device 110 extracts the watermark in the third reconstructed 3D model through the decoder in the teacher model to obtain the first watermark.
[0216] S850, the computing device 110 updates the model parameters of the intermediate diffusion model based on the loss between the first watermark and the target watermark to obtain the second diffusion model.
[0217] In one possible implementation, the computing device 110 updates the model parameters of the intermediate diffusion model based on the loss between the first watermark and the target watermark to obtain the second diffusion model, including: the computing device 110 calculates the loss (Lwm) between the first watermark and the target watermark, and then uses this loss to backpropagate the intermediate diffusion model to update the model parameters of the intermediate diffusion model, thereby obtaining the second diffusion model.
[0218] Lwm can be calculated by Hamming distance, edit distance, cross-entropy loss function, etc., and is not limited in this application.
[0219] It is worth noting that the above description only uses one three-dimensional model (the second three-dimensional model) as an example. In the actual training process, the computing device 110 will use the fine-tuning dataset for multiple rounds of training. In each round of training, the computing device 110 inputs a batch of data into the intermediate diffusion model, outputs multiple reconstructed three-dimensional models, and then uses the decoder to extract the watermarks from the multiple reconstructed three-dimensional models. Using the loss between the multiple watermarks and the target watermark, the model parameters of the intermediate diffusion model are updated to obtain the second diffusion model.
[0220] The above batch of data includes at least one three-dimensional model. Each batch of data is processed by the intermediate diffusion model and the decoder. The computing device 110 will update the model parameters of the intermediate diffusion model once. That is, the model parameters of the intermediate diffusion model will be updated once for each round of training until the loss between the first watermark and the target watermark converges, the set number of training rounds is reached, or all three-dimensional models in the fine-tuning dataset are used up, to obtain the second diffusion model.
[0221] The above-mentioned second diffusion model can generate a three-dimensional model with a second watermark. Since the model parameters of the intermediate diffusion model were adjusted according to the loss between the first watermark and the target watermark during the training process, the matching degree between the second watermark and the target watermark is greater than that between the first watermark and the target watermark. This achieves that the second diffusion model can retain the model parameters of the generated three-dimensional model as much as possible, and can also update the model parameters of the generated watermark, thereby improving the usability, i.e., robustness, of the second diffusion model. Specification 15 / 19 pages 19 CN 121366071 A
[0222] In one possible case, the matching degree between the second watermark and the target watermark is 100%, i.e., the second watermark is the target watermark.
[0223] In one possible case, the matching degree between the second watermark and the target watermark is within a set range. For example, greater than or equal to a first value, less than 1. The first value can be 70%. It is worth noting that this application does not limit the first value; it can be greater than 70% or less than 70%.
[0224] In one possible embodiment, the computing device 110 can verify whether the model has a watermark. As shown in FIG8b, FIG8b is a flowchart of the diffusion model verification method provided in this application. There are two diffusion models 1 and 2 suspected of misappropriating the second diffusion model, and it is necessary to perform watermark judgment on diffusion models 1 and 2.
[0225] In FIG8b, the user inputs prompt words (such as airplane, train) into diffusion models 1 and 2 on the client device 120. Diffusion model 1 outputs three-dimensional model 1, and diffusion model 2 outputs three-dimensional model 2. Since the watermark carried in three-dimensional model 1 and three-dimensional model 2 is implicit, the user can use the client device 120 to verify whether the three-dimensional model has a watermark.1. The 3D model 2 is sent to the computing device 110. The computing device 110 inputs the 3D model 1 and 3D model 2 into the decoder in the teacher model for watermark extraction, and inputs watermark 1 and watermark 2 respectively. If the matching degree of watermark 1 with the target watermark of the user's second diffusion model reaches a threshold, such as 70%, it can be determined that the diffusion model 1 to be judged is consistent with the above-mentioned second diffusion model, that is, others have stolen the second diffusion model. If the matching degree of watermark 2 with the target watermark of the user's second diffusion model does not reach the threshold, it can be determined that the diffusion model 2 to be judged is inconsistent with the above-mentioned second diffusion model.
[0226] It is understood that in order to realize the functions in the above embodiments, the computing device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily realize that, in conjunction with the units and method steps of the various examples described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.
[0227] The diffusion model watermarking embedding method provided according to the embodiments of this application has been described in detail above with reference to Figures 1 to 8b. The diffusion model watermarking embedding device provided according to the embodiments of this application will be described below with reference to Figure 9.
[0228] Figure 9 is a schematic diagram of the structure of a diffusion model watermarking embedding device provided by this application. The diffusion model watermarking embedding device can be used to implement the function of the computing device in the above-described diffusion model watermarking embedding method embodiments, and thus can also achieve the beneficial effects of the above-described method embodiments. In this embodiment, the diffusion model watermarking embedding device can be any device shown in Figure 1, such as computing device 110, client device 120 and acceleration device 115, or computing device shown in subsequent embodiments, or it can be a module (such as a chip) applied to the device.
[0229] As shown in Figure 9, the diffusion model watermarking embedding device 900 includes a first acquisition module 910, a first training module 920 and a second training module 930. The diffusion model watermarking embedding device 900 is used to implement the function of the computing device 110 in the method embodiments corresponding to Figures 1 to 8b. In one possible example, the diffusion model watermark embedding device 900 implements the above-described diffusion model watermark embedding method through the following specific processes:
[0230] A first acquisition module 910 is used to acquire a first diffusion model, which is used to generate a three-dimensional model.
[0231] A first training module 920 is used to train the first diffusion model according to the encoder in the teacher model to obtain an intermediate diffusion model. The encoder is used to generate a three-dimensional model with the target watermark, and the intermediate diffusion model can generate a three-dimensional model with the first watermark.
[0232] A second training module 930 is used to train the intermediate diffusion model according to the decoder in the teacher model to obtain...The second diffusion model. The decoder is used to extract the watermark in the three-dimensional model. The second diffusion model can generate a three-dimensional model with the second watermark (pages 16 / 19 of the specification, 20 CN 121366071 A). The matching degree between the second watermark and the target watermark is greater than that between the first watermark and the target watermark.
[0233] To further realize the functions of the method embodiments shown in Figures 1 to 8b above, this application also provides a diffusion model watermark embedding device, as shown in Figure 10. Figure 10 is a schematic diagram of the structure of a diffusion model watermark embedding device provided by this application. The diffusion model watermark embedding device 900 further includes: a second acquisition module 940.
[0234] The second acquisition module 940 is used to provide a user interface, the user interface includes a first control component, and in response to the user's trigger operation on the first control component, acquires the target watermark.
[0235] For more functions of the first acquisition module 910, the first training module 920, the second training module 930, and the second acquisition module 940, please refer to the description of the diffusion model watermark embedding method above, which will not be repeated here.
[0236] The diffusion model watermark embedding device 900 of this application embodiment can be implemented by software modules. The diffusion model watermark embedding device 900 according to the embodiment of this application can correspond to the execution of the diffusion model watermark embedding method described in the embodiment of this application, and the above and other operations and / or functions of each module in the diffusion model watermark embedding device 900 are respectively to implement the method flow in the foregoing figures. For the sake of brevity, they will not be described again here.
[0237] It is worth noting that if the diffusion model watermark embedding device 900 is implemented by software modules, for example, the software module can be provided to users through a cloud service subscription mode, and users can choose different subscription levels according to their needs; for example, the software module can also provide enterprise-level customized services with professional domain customization, interface personalization and extended functions according to the needs of users or enterprises.
[0238] The diffusion model watermark embedding device 900 of this application embodiment can also be implemented by hardware, such as computing devices. The specific implementation method of computing devices can be referred to the description in FIG1, which will not be described again here.
[0239] Furthermore, when the diffusion model watermark embedding device 900 is implemented through a diffusion model watermark embedding system, the diffusion model watermark embedding system may include the computing device 110 provided in FIG. 1 and a first diffusion model generating device. The first diffusion model generating device and the computing device communicate via a wired or wireless connection. The first diffusion model generating device is used to generate a first diffusion model, and the computing device 110 is used to train the first diffusion model generated by the first diffusion model generating device to obtain a second diffusion model. The computing device may be used to execute the diffusion model watermark embedding method provided in the foregoing embodiments.
[0240] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, ROM, PROM, EPROM, electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in computing device 110. Of course, the processor and storage medium can also exist as discrete components in network devices or terminal devices.
[0241] This application embodiment also provides a computing device cluster. The computing device cluster includes at least one computing device, which can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a desktop computer, a laptop computer, or a smartphone, etc.
[0242] As shown in FIG11, FIG11 is a schematic diagram of a computing device cluster provided in this application. The computing device cluster includes at least one computing device 110. The memory 112 of one or more computing devices 110 in the computing device cluster may store the same instructions for executing the diffusion model watermarking embedding method. Specification 17 / 19 pages 21 CN 121366071 A
[0243] In some possible implementations, the memory 112 of one or more computing devices 110 in the computing device cluster may also store partial instructions for executing the diffusion model watermarking embedding method. In other words, a combination of one or more computing devices 110 can jointly execute the instructions for executing the diffusion model watermarking embedding method.
[0244] It should be noted that the memory 112 of different computing devices 110 in the computing device cluster may store different instructions, which are respectively used to execute partial functions of the diffusion model watermarking embedding method. That is, the instructions stored in the memory 112 of different computing devices 110 can implement the functions of one or more modules among the first acquisition module 910, the first training module 920, and the second training module 930.
[0245] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 12 shows one possible implementation. As shown in Figure 12...Figure 12 is a schematic diagram of the connection between computing devices provided in this application. Two computing devices 110A and 110B are connected through a network. Specifically, they are connected to the network through the communication interface in each computing device. In this possible implementation, the memory 112 in computing device 110A stores instructions for executing the function of the first acquisition module 910. At the same time, the memory 112 in computing device 110B stores instructions for executing the functions of the first training module 920 and the second training module 930.
[0246] It should be understood that the function of computing device 110A shown in Figure 12 can also be performed by multiple computing devices 110. Similarly, the function of computing device 110B can also be performed by multiple computing devices 110.
[0247] This application embodiment also provides a computer program product containing instructions. The computer program product can be a software or program product containing instructions that can run on a computing device or be stored in any available medium. When the computer program product is run on at least one computing device, it causes at least one computing device to perform the above-described diffusion model watermark embedding method.
[0248] This application embodiment also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform a diffusion model watermark embedding method.
[0249] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in this application embodiment are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any usable medium that a computer can access, or a data storage device such as a server or data center that integrates one or more usable media. The usable medium may be a magnetic medium, such as a floppy disk or hard disk.Magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid-state drive (SSD).
[0250] The above description is only a specific embodiment of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application. These modifications or substitutions should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims. Instruction Manual 19 / 19 Page 23 CN 121366071 A Figure 1 Figure 2 Instruction Manual Appendix 1 / 7 Page 24 CN 121366071 A Figure 3 Figure 4 Figure 5 Instruction Manual Appendix 2 / 7 Page 25 CN 121366071 A Figure 6 Figure 7 Instruction Manual Appendix 3 / 7 Page 26 CN 121366071 A Figure 8a Instruction Manual Appendix 4 / 7 Page 27 CN 121366071 A Figure 8b Figure 9 Instruction Manual Appendix 5 / 7 Page 28 CN 121366071 A Figure 10 Figure 11 Instruction Manual Appendix 6 / 7 Page 29 CN 121366071 A Figure 12 Instruction Manual Appendix 7 / 7 Page 30 CN 121366071 A Disclosed are a diffusion model watermarking method and apparatus, relating to the technical field of artificial intelligence. A computing device acquires a first diffusion model, trains the first diffusion model on the basis of an encoder in a teacher model {0 obtain an intermediate diffusion model, and trains the intermediate diffusion model on the basis of a decoder in the teacher model to obtain a second diffusionmodel. The intermediate diffusion model can generate a three-dimensional model having a first watermark, the second diffusion model can generate a three-dimensional model having a second watermark, and the matching degree between the second watermark and a target watermark is greater than the matching degree between the first watermark and the target watermark. The first diffusion model is segmentally adjusted by means of the encoder and the decoder, so that the first diffusion model can learns the features during the stage of generating the three-dimensional model having the target watermark; thus, the obtained second diffusion model can not only preserve model parameters of the generated three- dimensional model as much as possible, but also update the model parameters of the generated watermark, thereby improving the availability of the second diffusion model. Abstract
Claims
1. A diffusion model watermark embedding method, characterized in that, The method includes: Obtain a first diffusion model, which is used to generate a three-dimensional model; The first diffusion model is trained using the encoder in the teacher model to obtain an intermediate diffusion model; the encoder is used to generate a 3D model with the target watermark, and the intermediate diffusion model can generate a 3D model with the first watermark. The intermediate diffusion model is trained using the decoder in the teacher model to obtain a second diffusion model; the decoder is used to extract the watermark from the 3D model, and the second diffusion model can generate a 3D model with the second watermark. The matching degree between the second watermark and the target watermark is greater than the matching degree between the first watermark and the target watermark.
2. The method according to claim 1, characterized in that, Before training the first diffusion model based on the encoder in the teacher model to obtain the intermediate diffusion model, the method further includes: A user interface is provided, the user interface including a first control component; In response to a user's trigger operation on the first control component, the target watermark is acquired.
3. The method according to claim 1 or 2, characterized in that, The process of obtaining the first diffusion model includes: A user interface is provided, the user interface including a second control component; In response to a user's triggering operation on the second control component, the first diffusion model is obtained.
4. The method according to claim 1 or 2, characterized in that, The process of obtaining the first diffusion model includes: The first diffusion model is obtained from a plurality of reference diffusion models, each of which is used to generate a three-dimensional model.
5. The method according to any one of claims 1 to 4, characterized in that, The teacher model is trained using a training dataset and the target watermark. The training dataset includes different types of 3D models, including one or more of the following: vehicles, office equipment, and buildings.
6. The method according to any one of claims 1 to 5, characterized in that, The step of training the first diffusion model based on the encoder in the teacher model to obtain the intermediate diffusion model includes: The first 3D model in the fine-tuning dataset is input into the encoder and the first diffusion model, and the output is a first reconstructed 3D model and a second reconstructed 3D model; the first 3D model is any one of the multiple 3D models included in the fine-tuning dataset; Based on the loss between the first reconstructed 3D model and the second reconstructed 3D model, the model parameters of the first diffusion model are updated to obtain the intermediate diffusion model.
7. The method according to any one of claims 1 to 6, characterized in that, The step of training the intermediate diffusion model based on the decoder in the teacher model to obtain the second diffusion model includes: The second 3D model in the fine-tuned dataset is input into the intermediate diffusion model to obtain the third reconstructed 3D model; the third reconstructed 3D model includes the first watermark, and the second 3D model is any one of the multiple 3D models included in the fine-tuned dataset; The third reconstructed 3D model is input into the decoder to obtain the first watermark; Based on the loss between the first watermark and the target watermark, the model parameters of the intermediate diffusion model are updated to obtain the second diffusion model.
8. The method according to any one of claims 1 to 7, characterized in that, The encoder includes a local feature extraction network and a global feature extraction network.
9. The method according to claim 8, characterized in that, The local feature extraction network includes a graph convolutional neural network; and / or, The global feature extraction network includes the graph convolutional neural network.
10. A diffusion model watermark embedding device, characterized in that, The device includes: The first acquisition module is used to acquire a first diffusion model, which is used to generate a three-dimensional model. The first training module is used to train the first diffusion model based on the encoder in the teacher model to obtain an intermediate diffusion model; the encoder is used to generate a 3D model with a target watermark, and the intermediate diffusion model can generate a 3D model with a first watermark. The second training module is used to train the intermediate diffusion model based on the decoder in the teacher model to obtain a second diffusion model; the decoder is used to extract the watermark from the three-dimensional model, and the second diffusion model can generate a three-dimensional model with the second watermark, wherein the matching degree between the second watermark and the target watermark is greater than the matching degree between the first watermark and the target watermark.
11. The apparatus according to claim 10, characterized in that, The device further includes: a second acquisition module; The second acquisition module is used to provide a user interface, the user interface including a first control component, and to acquire the target watermark in response to a user's trigger operation on the first control component.
12. The apparatus according to claim 10 or 11, characterized in that, The first acquisition module is specifically used to provide a user interface, the user interface including a second control component, and to acquire the first diffusion model in response to a user's trigger operation on the second control component.
13. The apparatus according to claim 10 or 11, characterized in that, The first acquisition module is specifically used to acquire the first diffusion model from a plurality of reference diffusion models, wherein each of the plurality of reference diffusion models is used to generate a three-dimensional model.
14. The apparatus according to any one of claims 10 to 13, characterized in that, The teacher model is trained using a training dataset and the target watermark. The training dataset includes different types of 3D models, including one or more of the following: vehicles, office equipment, and buildings.
15. The apparatus according to any one of claims 10 to 14, characterized in that, The first training module is specifically used to input the first 3D model in the fine-tuning dataset into the encoder and the first diffusion model, output the first reconstructed 3D model and the second reconstructed 3D model, and update the model parameters of the first diffusion model according to the loss between the first reconstructed 3D model and the second reconstructed 3D model to obtain the intermediate diffusion model. The first 3D model is any one of the multiple 3D models included in the fine-tuning dataset.
16. The apparatus according to any one of claims 10 to 15, characterized in that, The second training module is specifically used to input the second 3D model in the fine-tuning dataset into the intermediate diffusion model to obtain the third reconstructed 3D model, and input the third reconstructed 3D model into the decoder to obtain the first watermark. Based on the loss between the first watermark and the target watermark, the model parameters of the intermediate diffusion model are updated to obtain the second diffusion model. The third reconstructed 3D model includes the first watermark, and the second 3D model is any one of the multiple 3D models included in the fine-tuning dataset.
17. The apparatus according to any one of claims 10 to 16, characterized in that, The encoder includes a local feature extraction network and a global feature extraction network.
18. The apparatus according to claim 17, characterized in that, The local feature extraction network includes a graph convolutional neural network; and / or, the global feature extraction network includes the graph convolutional neural network.
19. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 9.
20. A computer-readable storage medium, characterized in that, The storage medium stores a computer program or instructions, which, when executed by a computing device, implement the method of any one of claims 1 to 9.
21. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a computing device, the method of any one of claims 1 to 9 is implemented.
