Method, system and equipment for generating attached drawings in patent field
By determining multiple target prompt words and modifying the first edition of the drawing generation model, the problem that the existing literary drawing technology cannot generate accurate patent drawings is solved, and efficient and accurate drawing generation is achieved, which is suitable for the patent field.
Patent Information
- Application Number
- CN202510042211.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-10
- Publication Date
- 2025-04-25
AI Technical Summary
The existing literary and biographical drawing technology cannot accurately generate the accompanying drawings required in the patent field, resulting in inaccurate images and cannot be applied to the patent field.
By determining the first target prompt word and the second target prompt word, the first page attached drawing is generated using the preset drawing generation model, and the first page attached drawing is modified through the third target prompt word and the target modification area to generate the final attached drawing.
It realizes the rapid and efficient generation of accurate drawings that can be applied to the patent field, improves work efficiency, and avoids the common illusion phenomena in literary drawing technology.
Smart Images
Figure CN119887990A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, system and device for generating drawings in the patent field. Background Art
[0002] With the development of data processing technology, using text to directly generate the required images can greatly improve work efficiency, and its application is becoming more and more extensive. However, the existing text-to-image method is generally based on the direct transfer principle to generate images, and "hallucination" phenomena often occur, resulting in inaccurate images generated by text. This is obviously not feasible in some fields that have strict image requirements, such as patent drawings. Summary of the invention
[0003] In view of this, the purpose of the present invention is to provide a method, system and device for generating drawings for the patent field, so as to overcome the problem that the current text image technology cannot be directly applied to the patent field.
[0004] To achieve the above objectives, the present invention adopts the following technical solutions:
[0005] In a first aspect, the present application embodiment provides a method for generating drawings in the patent field, including:
[0006] Determine a first target prompt word and a second target prompt word, and generate a preliminary version of the drawing based on the first target prompt word and the second target prompt word through a preset drawing generation model; wherein the first target prompt word is used to determine the components in the drawing to be generated, and the second target prompt word is used to determine the connection information of each component in the drawing to be generated;
[0007] Determine a third target prompt word and a target modification area, and modify the preliminary version of the drawing through a preset drawing generation model to obtain a final drawing, wherein the third target prompt word is used to determine how to modify the target modification area.
[0008] Furthermore, in some embodiments of the present application, the generation process of the drawing generation model includes:
[0009] Generate a component data set based on a first prompt word and a component image, wherein the first prompt word and the component image have a one-to-one correspondence;
[0010] Generate a component connection data set based on a second prompt word and a component connection image; wherein the second prompt word and the component connection image have a one-to-one correspondence;
[0011] The component data set and the component connection data set are used as the full data set of the Stable Diffusion model to train and generate a basic model.
[0012] Furthermore, in some embodiments of the present application, the generation process of the drawing generation model further includes:
[0013] Randomly extracting images and corresponding prompt words from the component connection data set to generate connection image-text pairs; and masking the connection parts in the component connection image using a mask to obtain a masked image with a mask;
[0014] Produce a third prompt word corresponding to the mask image, obtain a mask image-text pair consisting of the mask image and the third prompt word corresponding to the mask image, so as to construct a control data set;
[0015] The basic model is diffusely modified based on the control data set to obtain the drawing generation model.
[0016] Further, in some embodiments of the present application, the component data set and the component connection data set are used as the full data set of the Stable Diffusion model to train and generate a basic model, including:
[0017] When the component dataset and the component connection dataset are input into the stable diffusion model for training, the encoder and the decoder in the VAE network module in the stable diffusion are frozen.
[0018] Furthermore, in some embodiments of the present application, the training and generating a basic model by using the component dataset and the component connection dataset as the full dataset of the Stable Diffusion model also includes:
[0019] Based on the implicit vectors of the component dataset and the component connection dataset training process, the text encoding layer and UNet in the StableDiffusion model are constrained to train and generate the basic model.
[0020] Furthermore, in some embodiments of the present application, the diffusion modification of the basic model based on the control data set to obtain the drawing generation model includes:
[0021] Start the training of the encoder and decoder in the VAE module in the base model, and freeze the trainable parameters in the UNet;
[0022] Inputting the mask image into the base model to obtain a first parameter output by an encoder of the VAE in the base model;
[0023] Inputting the unmasked image corresponding to the masked image into the base model to obtain a second parameter output by the encoder of the VAE in the base model;
[0024] Input the first parameter into a preset ControlNet to obtain a third parameter;
[0025] Inputting the second parameter into the UNet in the basic model to obtain a fourth parameter;
[0026] The mean square error of the third parameter and the fourth parameter is used as a loss function to train the ControlNet layer.
[0027] Furthermore, in some embodiments of the present application, the first prompt words include professional fields, graphic element names, graphic element sizes, graphic element directions and graphic element positions; the second prompt words include assembly methods, connection methods and quantities of components.
[0028] Furthermore, in some embodiments of the present application, the first target prompt word corresponds to the first prompt word, and the second target prompt word corresponds to the second prompt word.
[0029] In a second aspect, the present application embodiment provides a system for generating drawings in the patent field, including:
[0030] A first determination module is used to determine a first target prompt word and a second target prompt word, and generate a preliminary version of the drawing based on the first target prompt word and the second target prompt word through a preset drawing generation model; wherein the first target prompt word is used to determine the components in the drawing to be generated, and the second target prompt word is used to determine the connection information of each component in the drawing to be generated;
[0031] The second determination module is used to determine the third target prompt word and the target modification area, and to modify the preliminary version of the drawing through a preset drawing generation model to obtain a final drawing, wherein the third target prompt word is used to determine how to modify the target modification area.
[0032] In a third aspect, an embodiment of the present application provides a device for generating drawings in the patent field, including a processor and a memory, wherein the processor is connected to the memory:
[0033] Wherein, the processor is used to call and execute the program stored in the memory;
[0034] The memory is used to store the program, and the program is at least used to execute the above-mentioned method for generating drawings for the patent field.
[0035] The present invention relates to the field of data processing technology, and in particular to a method, system and device for generating drawings in the patent field, the method comprising determining a first target prompt word and a second target prompt word, and generating a preliminary version of drawings based on the first target prompt word and the second target prompt word through a preset drawing generation model; wherein the first target prompt word is used to determine the components in the drawings to be generated, and the second target prompt word is used to determine the connection information of the components in the drawings to be generated; determining a third target prompt word and a target modification area, and modifying the preliminary version of the drawings through the preset drawing generation model to obtain a final drawing, wherein the third target prompt word is used to determine how to modify the target modification area. In this way, drawings that can be applied to the patent field can be generated quickly and efficiently, thereby greatly improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 It is a flowchart of a method for generating drawings for the patent field provided by an embodiment of the present invention.
[0038] Figure 2 It is a flow chart of generating a drawing generation model in a drawing generation method for the patent field provided in an embodiment of the present invention.
[0039] Figure 3 It is a structural diagram of a system for generating drawings for patent fields provided by an embodiment of the present invention.
[0040] Figure 4 It is a structural schematic diagram of a device for generating illustrations for the patent field provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0041] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.
[0042] Figure 1 This is a flow chart of a method for generating drawings for patent fields provided by an embodiment of the present invention. Figure 1 , this embodiment may include the following steps:
[0043] S101, determining a first target prompt word and a second target prompt word.
[0044] S102: Generate a preliminary version of the drawing based on the first target prompt word and the second target prompt word by using a preset drawing generation model.
[0045] The first target prompt word is used to determine the components in the drawing to be generated, and the second target prompt word is used to determine the connection information of each component in the drawing to be generated.
[0046] Specifically, the present application constructs a model for generating drawings in advance. In actual application, it is only necessary to input the first target prompt word and the second target prompt word into the model, and the first version of the drawings can be obtained through the output of the model. The first target prompt word and the second target prompt word can be identified and extracted from relevant files.
[0047] S103: Determine a third target prompt word and a target modification area.
[0048] S104, modifying the preliminary drawings through a preset drawing generation model to obtain a final drawing.
[0049] The third target prompt word is used to determine how to modify the target modification area.
[0050] Specifically, after obtaining the preliminary version of the drawing, the third target prompt word and the target modification area are determined, wherein the target modification area can be determined by masking the area to be modified in the generated preliminary version of the drawing, and then the preliminary version image with the mask and the third target prompt word for determining how to modify the target modification area are input into the drawing generation model, the preliminary version image is modified by the drawing generation model, and the final drawing that can be directly applied to the patent document is output by the drawing generation model.
[0051] The method for generating drawings for the patent field provided by the present application generates a preliminary version of drawings by determining a first target prompt word and a second target prompt word, and by using a preset drawing generation model, based on the first target prompt word and the second target prompt word; wherein the first target prompt word is used to determine the components in the drawings to be generated, and the second target prompt word is used to determine the connection information of each component in the drawings to be generated; the third target prompt word and the target modification area are determined, and the preliminary version of drawings are modified by using the preset drawing generation model to obtain the final drawings, wherein the third target prompt word is used to determine how to modify the target modification area. In this way, drawings that can be applied to the patent field can be generated quickly and efficiently, thereby greatly improving work efficiency.
[0052] Figure 2This is a flow chart of generating a model for generating drawings in a method for generating drawings for patent fields provided by an embodiment of the present invention. Figure 2 In the embodiment of the present application, the process of generating the drawing generation model may include the following steps:
[0053] S201: Generate a component data set based on a first prompt word and a component image.
[0054] Specifically, a large number of component images and first prompt words, such as millions of them, are obtained, wherein the component images and the first prompt words are in one-to-one correspondence, and a component data set is obtained. The first prompt words may include professional fields, primitive names, primitive sizes, primitive directions, and primitive positions.
[0055] S202: Generate a component connection data set based on the second prompt word and the component connection image.
[0056] Specifically, a large number of component connection images and second prompt words, such as tens of millions, are obtained, wherein the component connection images and the second prompt words are also one-to-one corresponding data, and a component connection data set is obtained. The second prompt word report may include the assembly method, connection method, and number of components of the component images.
[0057] S203: Use the component data set and the component connection data set as the full data set of the Stable Diffusion model to train and generate a basic model.
[0058] Specifically, the component dataset and the component connection dataset are used as the full dataset of the Stable Diffusion model for training to obtain a basic model that can generate a preliminary version of the image.
[0059] S204 , generating a mask image based on the component connection data set, and making a third prompt word corresponding to the mask image to construct a control data set.
[0060] Specifically, images and corresponding prompt words are randomly extracted from the component connection data set to generate connection image-text pairs; the connection parts in the component connection image are masked using a mask to obtain a masked image with a mask; a third prompt word corresponding to the mask image is produced to obtain a masked image-text pair consisting of the mask image and the third prompt word corresponding to the mask image, so as to construct a control data set.
[0061] In practical applications, some images and corresponding prompt words are randomly extracted from the component connection data set, and recorded as connection image-text pairs. Then the connection parts in the extracted images are masked using a mask to obtain a masked image, i.e., a masked image. Then, a third prompt word is generated for the masked image, and several masked image-text pairs consisting of a masked image and a third prompt word corresponding to each masked image are obtained to obtain a control data set.
[0062] In some embodiments of the present application, the mask image, the third prompt word corresponding to the mask image, and the second prompt word of the image corresponding to the mask image in the component connection data set can also be used to form a mask image-text pair for subsequent training, thereby improving the accuracy of subsequent model training.
[0063] S205 , performing diffusion modification on the basic model based on the control data set to obtain a model for generating the attached figure.
[0064] Specifically, the control data set is directly input into the basic model, the diffusion model is adjusted, and the corresponding conditional diffusion model is obtained.
[0065] It should be noted that, because the model for generating the attached figures in this application is mainly divided into two parts, namely, obtaining the basic model and adjusting the basic model to obtain the final model, the entire training process can be achieved by freezing part of the training, as follows:
[0066] First, for the process of obtaining the basic model based on the component dataset and the component connection dataset, when the component dataset and the component connection dataset are input into the stable diffusion model for training, the encoder and decoder in the VAE network module in stable diffusion can be frozen to maintain the information extraction capability and training weight of the VAE module in general image training.
[0067] In addition, in this process, the implicit vectors of the component dataset and the component connection dataset training process can be used to constrain the text encoding layer and UNet, so as to train the corresponding basic model.
[0068] On this basis, based on the basic model and control data set obtained above, the final model is trained. Specifically, the control data set is input into the basic model, the encoder and decoder training in the VAE module in the basic model is started, and the trainable parameters in UNet are frozen, so as to adjust the basic model and obtain the conditional diffusion model, which is the final figure generation model.
[0069] In addition, in the process of obtaining the final illustration generation model based on the basic model, it also includes inputting the mask image into the basic model to obtain the first parameter P1 of the encoder output of the VAE in the basic model; inputting the unmasked picture corresponding to the mask image into the basic model to obtain the second parameter P2 of the encoder output of the VAE in the basic model; then inputting the first parameter P1 into the preset ControlNet for processing, and outputting the third parameter P3; and inputting the second parameter P2 into the UNet in the basic model for processing, and outputting the fourth parameter P4; finally, the mean square error of the third parameter P3 and the fourth parameter P4 is used as the loss function of the ControlNet layer in the illustration generation model (that is, the conditional diffusion model is obtained by training the ControlNet layer based on the above-mentioned basic model), so as to obtain the conditional diffusion model, that is, the final illustration generation model.
[0070] It can be understood that other training parts in this application can be understood by referring to the stable diffusion in the prior art and the solution of obtaining the conditional diffusion model through ControlNet, which will not be repeated here.
[0071] On this basis, in actual applications, it is only necessary to determine the first target prompt word and the second target prompt word based on the relevant documents, and input the first target prompt word and the second target prompt word into the drawing generation model. The drawing generation model automatically matches the first prompt word corresponding to the first target prompt word, thereby determining the component image, and automatically matches the second prompt word corresponding to the second target prompt word, thereby determining the connection relationship between the components and other information, thereby obtaining the preliminary version of the drawing. At this time, if the preliminary version of the drawing does not meet the requirements, the relevant personnel can determine the third target prompt word, and mask the preliminary version of the drawing, and input the third target prompt word and the masked preliminary version of the drawing into the drawing generation model. The drawing generation model can automatically modify the preliminary version of the drawing, thereby generating the final drawing.
[0072] The method for generating drawings for the patent field provided in the present application can efficiently generate drawings for the patent field based on the trained drawing generation model, and can also make fine modifications to the generated drawings by means of masks and prompt words, thereby avoiding the problem of hallucinations that are easy to occur when drawing text, resulting in the image being unusable, and can greatly improve the efficiency of image generation in the patent writing process.
[0073] Based on the same inventive concept, the present invention also provides a system for generating drawings in the patent field, which is used to implement the above method embodiments. Figure 3 is a schematic diagram of the structure of a patent field-oriented drawing generation system provided by an embodiment of the present invention, such as Figure 3As shown, the system includes: a first determination module 11 and a second determination module 12, wherein the first determination module 11 is used to determine a first target prompt word and a second target prompt word, and generate a preliminary version of the drawing based on the first target prompt word and the second target prompt word through a preset drawing generation model; the first target prompt word is used to determine the components in the drawing to be generated, and the second target prompt word is used to determine the connection information of each component in the drawing to be generated; the second determination module 12 is used to determine a third target prompt word and a target modification area, and modify the preliminary version of the drawing through a preset drawing generation model to obtain a final drawing, wherein the third target prompt word is used to determine how to modify the target modification area.
[0074] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0075] The present invention also provides a device for generating drawings in the patent field, which is used to implement the above method embodiments. Figure 4 is a schematic diagram of the structure of a device for generating drawings for the patent field provided by an embodiment of the present invention, such as Figure 4 As shown, the patent field-oriented drawing generation device of this embodiment includes a processor 21 and a memory 22, and the processor 21 is connected to the memory 22. The processor 21 is used to call and execute the program stored in the memory 22; the memory 22 is used to store the program, which is at least used to execute the patent field-oriented drawing generation method in the above embodiment.
[0076] The specific implementation scheme of the device for generating illustrations for the patent field provided in the embodiment of the present application can refer to the implementation scheme of the method for generating illustrations for the patent field in any of the above embodiments, which will not be repeated here.
[0077] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.
[0078] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.
[0079] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.
[0080] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0081] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0082] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0083] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.
[0084] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0085] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for generating drawings for patent fields, characterized in that: include: Determine a first target prompt word and a second target prompt word, and generate a preliminary version of the drawing based on the first target prompt word and the second target prompt word through a preset drawing generation model; wherein the first target prompt word is used to determine the components in the drawing to be generated, and the second target prompt word is used to determine the connection information of each component in the drawing to be generated; Determine a third target prompt word and a target modification area, and modify the preliminary version of the drawing through a preset drawing generation model to obtain a final drawing, wherein the third target prompt word is used to determine how to modify the target modification area.
2. The method for generating drawings for patent fields according to claim 1, characterized in that: The generation process of the drawing generation model includes: Generate a component data set based on a first prompt word and a component image, wherein the first prompt word and the component image have a one-to-one correspondence; Generate a component connection data set based on a second prompt word and a component connection image; wherein the second prompt word and the component connection image have a one-to-one correspondence; The component data set and the component connection data set are used as the full data set of the Stable Diffusion model to train and generate a basic model.
3. The method for generating drawings for patent fields according to claim 2, characterized in that: The generation process of the drawing generation model further includes: Randomly extracting images and corresponding prompt words from the component connection data set to generate connection image-text pairs; and masking the connection parts in the component connection image using a mask to obtain a masked image with a mask; Produce a third prompt word corresponding to the mask image, obtain a mask image-text pair consisting of the mask image and the third prompt word corresponding to the mask image, so as to construct a control data set; The basic model is diffusely modified based on the control data set to obtain the drawing generation model.
4. The method for generating drawings for patent fields according to claim 3, characterized in that: The method of using the component data set and the component connection data set as the full data set of the Stable Diffusion model to train and generate a basic model includes: When the component dataset and the component connection dataset are input into the stable diffusion model for training, the encoder and the decoder in the VAE network module in the stable diffusion are frozen.
5. The method for generating drawings for patent fields according to claim 4, characterized in that: The training and generating a basic model by using the component data set and the component connection data set as the full data set of the Stable Diffusion model further includes: Based on the implicit vectors of the component dataset and the component connection dataset training process, the text encoding layer and UNet in the StableDiffusion model are constrained to train and generate the basic model.
6. The method for generating drawings for patent fields according to claim 5, characterized in that: The step of performing diffusion modification on the basic model based on the control data set to obtain the drawing generation model comprises: Start the training of the encoder and decoder in the VAE module in the base model, and freeze the trainable parameters in the UNet; Inputting the mask image into the base model to obtain a first parameter output by an encoder of the VAE in the base model; Inputting the unmasked image corresponding to the masked image into the base model to obtain a second parameter output by the encoder of the VAE in the base model; Input the first parameter into a preset ControlNet to obtain a third parameter; Inputting the second parameter into the UNet in the basic model to obtain a fourth parameter; The mean square error of the third parameter and the fourth parameter is used as a loss function to train the ControlNet layer.
7. The method for generating drawings for patent fields according to claim 2, characterized in that: The first prompt words include professional fields, graphic element names, graphic element sizes, graphic element directions and graphic element positions; the second prompt words include assembly methods, connection methods and quantities of components.
8. The method for generating drawings for patent fields according to claim 2, characterized in that: The first target prompt word corresponds to the first prompt word, and the second target prompt word corresponds to the second prompt word.
9. A system for generating drawings for patent fields, characterized in that: include: A first determination module is used to determine a first target prompt word and a second target prompt word, and generate a preliminary version of the drawing based on the first target prompt word and the second target prompt word through a preset drawing generation model; wherein the first target prompt word is used to determine the components in the drawing to be generated, and the second target prompt word is used to determine the connection information of each component in the drawing to be generated; The second determination module is used to determine the third target prompt word and the target modification area, and to modify the preliminary version of the drawing through a preset drawing generation model to obtain a final drawing, wherein the third target prompt word is used to determine how to modify the target modification area.
10. A device for generating drawings for patent fields, characterized in that: The invention comprises a processor and a memory, wherein the processor is connected to the memory: Wherein, the processor is used to call and execute the program stored in the memory; The memory is used to store the program, and the program is at least used to execute the method for generating drawings for the patent field as described in any one of claims 1-7.