Drawing method and device and drawing equipment

Abstract

The invention provides a plotting method, a plotting device and plotting equipment. The method comprises the steps of obtaining a first picture; wherein the first picture comprises character information; determining a corresponding vector diagram template according to the semantics of the text information; generating a line draft graph corresponding to the first picture according to the vector diagram template; wherein the line draft graph comprises contour lines of scenery corresponding to semantics of the character information; and inputting the semantics of the character information and the draft graph into a multi-modal image generation model to generate a first scene picture corresponding to the first picture. By the adoption of the method, according to the character information in the first picture made by the user, the artificial intelligence algorithm can be used for achieving staged drawing from characters and draft graphs to the scene effect picture, and it is guaranteed that the picture of the needed scenery is accurately generated through the character information.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a drawing method, device and drawing equipment. Background Art

[0002] Artificial Intelligence (AI) generated content is a general term for the production, manipulation and modification of data or media through artificial intelligence algorithms. With the implementation of a large number of applications, AI-generated content such as pictures, text, audio and even video has gradually entered people's daily lives. Using artificial intelligence algorithms to accurately generate pictures of required scenes through keywords is an important research direction of artificial intelligence generative drawing. Summary of the invention

[0003] The technical solution of the present invention aims to provide a drawing method, device and drawing equipment for accurately generating pictures of required scenes through keywords using artificial intelligence algorithms.

[0004] One embodiment of the present invention provides a drawing method, which includes:

[0005] Acquire a first picture; wherein the first picture includes text information;

[0006] Determine a corresponding vector diagram template according to the semantics of the text information;

[0007] Generate a line drawing graphic corresponding to the first picture according to the vector graphics template; wherein the line drawing graphic includes the contour lines of the scene corresponding to the semantics of the text information;

[0008] The semantics of the text information and the line drawing graphic are input into a multimodal image generation model to generate a first scene image corresponding to the first image.

[0009] Optionally, the mapping method further comprises:

[0010] Identify the coordinate position of the text information in the first picture;

[0011] Wherein, the vector diagram template includes a plurality of line arrays; after determining the corresponding vector diagram template, the method further includes:

[0012] Determine the dot matrix coordinates corresponding to the plurality of line arrays according to the coordinate positions; the coordinate positions are located within the coverage area of ​​the dot matrix coordinates;

[0013] The step of generating a line drawing corresponding to the first picture according to the vector graphics template includes:

[0014] Generate a line drawing corresponding to the first picture according to the dot coordinates corresponding to the plurality of line arrays in the vector graphics template.

[0015] Optionally, the mapping method further comprises:

[0016] According to the semantics of the text information, when it is determined that the text information includes a first target text whose text type is an adjective or a verb, the dot matrix coordinates corresponding to the multiple line arrays are adjusted according to the first target text.

[0017] Optionally, the mapping method further comprises:

[0018] Outputting label options corresponding to the text information according to the semantics of the text information;

[0019] Obtaining the label attribute corresponding to the text information selected on the label option;

[0020] According to the semantics of the text information, determining the corresponding vector diagram template includes:

[0021] A corresponding vector graphics template is determined according to the semantics of the text information and the corresponding tag attributes.

[0022] Optionally, the drawing method, wherein, according to the semantics of the text information, outputting label options corresponding to the text information, includes:

[0023] According to the semantics of the text information, a label option corresponding to a second target text whose text type is a noun in the text information is output.

[0024] Optionally, the mapping method further comprises:

[0025] Displaying the line drawing graphic on the screen according to a first input operation of the user;

[0026] Acquire a first adjustment operation for adjusting the position, size and / or shape of the line drawing graphic, and obtain the adjusted line drawing graphic;

[0027] The semantics of the text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a second scene image corresponding to the first image.

[0028] Optionally, the mapping method further comprises:

[0029] Displaying the first picture on the screen according to a second input operation of the user;

[0030] Acquire a second adjustment operation for adjusting the position, size and / or shape of the text information, and obtain the coordinate position of the adjusted text information in the first picture;

[0031] Generate the adjusted line drawing graphic according to the adjusted coordinate position and the corresponding vector diagram template;

[0032] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a third scene image corresponding to the first image.

[0033] Optionally, the mapping method further comprises:

[0034] According to a third input operation of the user, a label option corresponding to the text information is displayed on the screen;

[0035] Obtaining a selection operation on the tag option, and obtaining the tag attribute selected by the selection operation;

[0036] Generate the adjusted line drawing graphic according to the selected label attribute and the corresponding vector graphic template;

[0037] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a fourth scene image corresponding to the first image.

[0038] Optionally, in the drawing method, the first picture further includes auxiliary lines for drawing composition;

[0039] Wherein, the method further comprises:

[0040] According to the auxiliary composition lines, the contour lines of different scenes in the line drawing and / or the interval positions of the contour lines of different scenes are determined.

[0041] Optionally, in the drawing method, the vector image template includes a border line drawing template of a scene corresponding to the semantics of the text information and a skeleton line drawing template of a scene corresponding to the semantics of the text information.

[0042] One embodiment of the present invention further provides a drawing device, which includes:

[0043] A first acquisition module is used to acquire a first picture, wherein the first picture includes text information;

[0044] A determination module, used to determine a corresponding vector diagram template according to the semantics of the text information;

[0045] A first generating module is used to generate a line drawing graphic corresponding to the first picture according to the vector graphic template; wherein the line drawing graphic includes the contour lines of the scene corresponding to the semantics of the text information;

[0046] The second generation module is used to input the semantics of the text information and the line drawing graphics into a multimodal image generation model to generate a first scene image corresponding to the first image.

[0047] One embodiment of the present invention further provides a drawing device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the drawing method described in any one of the above.

[0048] At least one of the above technical solutions in the specific embodiment of the present invention has the following beneficial effects:

[0049] By adopting the drawing method described in this embodiment, semantic recognition is performed based on the text information (keywords) in the first picture created by the user, and the corresponding vector graphics model is determined based on the semantics of the recognized text information, and the corresponding line drawing graphics are generated. The drawing process can use artificial intelligence algorithms to realize phased drawing from text, line drawing graphics to scene renderings, ensuring that the pictures of the required scenes are accurately generated through text information. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of a flow chart of a drawing method according to an embodiment of the present invention;

[0051] Figure 2 One implementation of the display format of the first picture;

[0052] Figure 3 It is a schematic diagram of the structure of the drawing device described in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0054] One embodiment of the present invention provides a drawing method, such as Figure 1 As shown, the method includes:

[0055] S110, obtaining a first picture; wherein the first picture includes text information;

[0056] S120, determining a corresponding vector diagram template according to the semantics of the text information;

[0057] S130, generating a line drawing graphic corresponding to the first picture according to the vector graphic template; wherein the line drawing graphic includes the contour lines of the scene corresponding to the semantics of the text information;

[0058] S140, inputting the semantics of the text information and the line drawing graphic into a multimodal image generation model to generate a first scene image corresponding to the first image.

[0059] By adopting the drawing method described in this embodiment, semantic recognition is performed based on the text information (keywords) in the first picture created by the user, and the corresponding vector graphics model is determined based on the semantics of the recognized text information, and the corresponding line draft graphics are generated. This drawing process can realize the phased drawing from text, line draft graphics to scene renderings, ensuring that the picture of the required scene is accurately generated through the text information.

[0060] It should be noted that, in the embodiment of the present invention, the text information in the first picture is the keywords for the scene to be presented in the picture; for example, when it is necessary to generate a picture including scenes such as windows, beds and paintings, the user can draw on the touch screen with a stylus and write the keywords or key words such as "window", "bed" and "painting" in the corresponding positions.

[0061] Optionally, in step S110, a first picture is obtained. In one implementation mode, optionally, a first picture directly created by a user on a touch screen may be obtained.

[0062] Using this implementation mode, the user inputs text information on the touch screen, and the text information is distributed at a specific position on the touch screen. Finally, the picture of at least one text information distributed at a specific position displayed is formed as a first picture, that is, the text information in the first picture is distributed at a specific position, and the specific position indicates the distribution position and / or distribution area of ​​the scene indicated by the text information when generating the scene picture corresponding to the text information.

[0063] In another implementation, the first image may be obtained by retrieving a file in a database. Optionally, after the first image is obtained, the obtained first image may be output and displayed on a screen.

[0064] In this implementation, similarly, the text information is distributed at specific positions in the first image, and the specific positions indicate the distribution positions and / or distribution areas of the scenes indicated by the text information when the scene image corresponding to the text information is generated.

[0065] Optionally, the multiple text information in the first picture may have different sizes, and the text size ratio is used to indicate the size ratio of the corresponding scene that needs to be presented.

[0066] Optionally, in addition to the text information, the first picture also includes auxiliary composition lines around the text information, which are used as outline lines of different scenes and / or as spacing lines between different scenes.

[0067] For example, the first picture created by the user on the touch screen may be as follows: Figure 2 As shown, the keywords "window", "bed" and "painting" are included in the text information and are distributed in specific positions respectively. In addition, composition auxiliary lines 1 are also made on the first picture to serve as spacing lines between the window, bed and painting in the generated scene picture.

[0068] By adopting the drawing method described in this embodiment, taking advantage of the fact that Chinese characters are pictographic characters, after inputting text information on the screen, the position and proportional size of the scenery to be presented in the scene picture can be determined based on the font, position and size of the input text information, so as to generate images of different styles.

[0069] In the embodiment of the present invention, after obtaining the first picture created by the user on the touch screen, the text information in the first picture is further recognized using a text recognition model to recognize the semantics of the text information in the first picture.

[0070] Optionally, as an implementation mode, a CoMER model trained based on a custom dictionary may be used to perform semantic recognition on the text information in the first image. The CoMER model may better utilize the positional relationship between each piece of text information to perform semantic recognition on the text.

[0071] Optionally, the method further comprises:

[0072] Outputting label options corresponding to the text information according to the semantics of the text information;

[0073] Obtaining the label attribute corresponding to the text information selected on the label option;

[0074] According to the semantics of the text information, determining the corresponding vector diagram template includes:

[0075] A corresponding vector graphics template is determined according to the semantics of the text information and the corresponding tag attributes.

[0076] With this implementation, the corresponding label options can be output according to the text information (keywords) in the first picture created by the user, and the user selects the label attribute corresponding to the text information used to generate the picture, and further determines the corresponding vector graphics model, so that the corresponding line drawing graphics can be generated according to the determined vector graphics template. The drawing process can realize phased drawing according to the user's input, ensuring that the picture of the required scene is accurately generated through the text information. Specifically, after the semantic recognition of the text information, the semantics of the text information in the first picture is obtained, such as combining Figure 2 As shown, the semantics of the text information in the first picture are identified as "window", "bed" and "painting", and the corresponding label options are further output based on the identified semantics.

[0077] As an implementation method, the output label options include multiple label attributes, including line size, fill color, size and / or display style, etc. The user selects the display features of the scene corresponding to the corresponding text information by selecting the corresponding label attribute on the presented label options.

[0078] As an implementation manner, optionally, the method further includes:

[0079] Obtaining, according to predefined descriptive words corresponding to the plurality of textual information, descriptive words of the textual information identified in the first image;

[0080] According to the obtained description words, the label options corresponding to the text information are output.

[0081] In this embodiment, the label options displayed on the screen are presented in the form of descriptive words. Optionally, the descriptive words can be adjectives used to describe the appearance, shape, size, style and color of the scene corresponding to the text information.

[0082] In the embodiment of the present invention, optionally, when the first picture is displayed on the current interface of the screen, the label option can be displayed as a pop-up menu near the corresponding text information. Wherein, each text information corresponds to a pop-up label option, allowing the user to select the label option corresponding to the text information to determine the display effect of the corresponding scene in the generated picture.

[0083] In one embodiment, optionally, the method further comprises:

[0084] Identify the coordinate position of the text information in the first picture;

[0085] Wherein, the vector diagram template includes a plurality of line arrays; after determining the corresponding vector diagram template, the method further includes:

[0086] Determine the dot matrix coordinates corresponding to the plurality of line arrays according to the coordinate positions; the coordinate positions are located within the coverage area of ​​the dot matrix coordinates;

[0087] The step of generating a line drawing corresponding to the first picture according to the vector graphics template includes:

[0088] Generate a line drawing corresponding to the first picture according to the dot coordinates corresponding to the plurality of line arrays in the vector graphics template.

[0089] Optionally, after generating the line drawing graphic corresponding to the first image, the line drawing graphic is displayed on the screen.

[0090] Optionally, when identifying the semantics of the text information in the first image, the coordinate position of the text information in the first image is also identified.

[0091] In addition, after the semantics and coordinate position of the text information in the first image are identified, a floating menu pops up on the first image currently displayed on the screen, displaying the semantics and coordinate position of the identified text information for intuitive display to the user for confirmation.

[0092] For example, according to Figure 2 The semantics and coordinate positions of the text information in the first picture displayed and the recognized text information in the first picture are as follows:

[0093] {[

[0094] {“text”:“Six-grid window”,“x”:100,“y”:600},

[0095] {"text":"bed","x":500,"y":400},

[0096] {"text":"drawing","x":800,"y":700},

[0097] ]}.

[0098] In the embodiment of the present invention, optionally, outputting a label option corresponding to the text information according to the semantics of the text information includes:

[0099] According to the semantics of the text information, a label option corresponding to a second target text whose text type is a noun in the text information is output.

[0100] In some implementations, a custom dictionary is pre-set, including text information corresponding to different text types such as nouns, verbs, and adjectives, forming a tree-structured text set. After semantic recognition of the text information in the first image, the text type classified by the custom dictionary for the text information is determined according to the recognized semantics. When the determined text type belongs to a noun, a label option corresponding to the text information (second target text) is output, so that the scene corresponding to the semantics of the second target text is generated according to the label attribute selected by the user on the label option.

[0101] In an embodiment of the present invention, optionally, multiple words in a custom dictionary have corresponding relationships with multiple vector map templates respectively. After obtaining the label attributes of the label option selected by the user, the semantics of the text information and the corresponding label attributes are matched with the vector map templates in the custom dictionary to determine the corresponding vector map template; optionally, text information with different semantics corresponds to different vector map template groups, each vector map template group includes different vector map templates, and when text information with the same semantics corresponds to different label attributes, it corresponds to different vector map templates respectively. In this way, according to the corresponding relationship between the semantics of the text information and the vector map templates corresponding to different label attributes, the vector map template corresponding to the semantics of the identified text information and the selected label attribute can be determined.

[0102] Optionally, each vector image template includes a plurality of line arrays. In one implementation manner, after identifying the coordinate position of the text information in the first image, the method further includes:

[0103] According to the coordinate position, the dot matrix coordinates corresponding to the plurality of line arrays are determined; the coordinate position is located within the coverage area of ​​the dot matrix coordinates.

[0104] In this way, each line array is composed of a plurality of dot coordinates, and the plurality of dot coordinates are used to define the coordinate position of the corresponding line array on the generated line drawing graphic.

[0105] According to the above, in one embodiment, generating a line drawing graphic corresponding to the first picture according to the vector graphics template includes:

[0106] Generate a line drawing corresponding to the first picture according to the dot coordinates corresponding to the plurality of line arrays in the vector graphics template.

[0107] For example, the dot coordinates in the vector graphics template can be expressed in the following form:

[0108] {[

[0109] [{"type":"rectangle","points":[…]},{"type":"rectangle","points":[…]},…],

[0110] [{"type":"rectangle","points":[…]},{"type":"rectangle","points":[…]},…],

[0111] [{"type":"square","points":[…]},{"type":"square","points":[…]},…],

[0112] ]}.

[0113] In one embodiment, optionally, the method further comprises:

[0114] According to the semantics of the text information, when it is determined that the text information includes a first target text whose text type is an adjective or a verb, the dot matrix coordinates corresponding to the multiple line arrays are adjusted according to the first target text.

[0115] Using this implementation, after semantic recognition is performed on the text information in the first image and the semantics of the text information is obtained, the semantics of the recognized text information is compared with a pre-set custom dictionary. When it is determined that the text type of the recognized text information includes an adjective or a verb, the dot coordinates corresponding to the multiple line arrays are adjusted according to the recognized text information (first target text).

[0116] For example, when the semantics of the recognized text information includes the text describing the verb "rotate 45" for the noun "bed", after determining the vector graph template corresponding to the noun "bed", the dot coordinates corresponding to the multiple line arrays in the determined vector graph template are rotated 45 degrees according to the text "rotate 45" in the text information to obtain the dot coordinates corresponding to the multiple line arrays.

[0117] Therefore, in the embodiment of the present invention, the point relationship of the line array in the vector graphics template can be adjusted through the text information of the adjective and the verb in the first picture.

[0118] In some implementations, the nouns in the custom dictionary may include living nouns and rigid nouns, wherein the vector diagram template corresponding to each rigid noun is a border line drawing template, and the vector diagram template corresponding to each living noun is a skeleton line drawing template. The line array in the border line drawing template is used to generate the border lines of the scene corresponding to the text information; the line array in the skeleton line drawing template is used to generate the skeleton lines of the scene corresponding to the text information, and the skeleton lines are also the main frame lines.

[0119] With this implementation, after the semantics of the text information is recognized and it is determined that the text information belongs to a noun, a border line draft template of a rigid noun corresponding to the noun and a skeleton line draft template of a living noun are determined, and the contour lines of the scene corresponding to the text information are generated according to the border line draft template and the skeleton line draft template. In addition, in an embodiment of the present invention, a vector map template for generating a line drawing graphic is determined by mapping the semantics of the text information with the corresponding vector map template through a custom dictionary, so as to achieve the effect of lowering the threshold of AI drawing.

[0120] By adopting this implementation mode, the semantics of the identified text information and the corresponding label attributes are matched one-to-one with multiple vector graphics templates, the corresponding vector graphics template is determined, and the text information in the first image is replaced according to the line graphics corresponding to the dot matrix coordinates of multiple line arrays in the vector graphics template to generate a line drawing graphic corresponding to the first image.

[0121] Specifically, in the line drawing graphic, the scene corresponding to the text information in the first picture is composed of multiple lines.

[0122] In one implementation manner, after generating the line drawing graphic, optionally, the method further includes:

[0123] receiving a fourth input operation for adjusting the shape of the line drawing graphic;

[0124] According to the fourth input operation, the line drawing graphic is adjusted.

[0125] Optionally, the fourth input operation may include one or more operations of dragging, stretching, and rotating the line drawing graphic.

[0126] In the embodiment of the present invention, optionally, the method further includes:

[0127] The text information and the line drawing graphic are input into a multimodal image generation model to generate a first scene image corresponding to the first image.

[0128] Among them, the first scene picture is a scene effect picture that can display the actual scene corresponding to the text information in the first picture.

[0129] The drawing method described in the embodiment of the present invention generates a scene picture of the first picture after generating a line drawing graphic corresponding to the first picture based on the text information in the first picture created by the user on the touch screen. In this way, the precise control of the content details of the generated scene picture can be improved by generating and editing the line drawing graphic.

[0130] Optionally, in some of the implementations, the multimodal image generation model may include a StableDiffusion model, and be paired with a ControlNet component. After the text information and the line drawing graphics are input, the model may output a corresponding first scene image.

[0131] In one embodiment of the present invention, optionally, the method further includes:

[0132] Displaying the line drawing graphic on the screen according to a first input operation of the user;

[0133] Acquire a first adjustment operation for adjusting the position, size and / or shape of the line drawing graphic, and obtain the adjusted line drawing graphic;

[0134] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a second scene image corresponding to the first image.

[0135] With this implementation, after the text information and the line drawing graphics are input into the multimodal image generation model to generate a first scene picture corresponding to the first picture, the line drawing graphics can be re-called on the screen through a first input operation. After the shape of the line drawing graphics is adjusted, the text information and the adjusted line drawing graphics are re-input into the multimodal image generation model to generate a second scene picture, thereby achieving the effect of adjusting the generated scene picture.

[0136] Optionally, the first input operation may include one or more operations such as dragging, stretching and rotating the line drawing graphic. In this way, by dragging, stretching and rotating the line drawing graphic, the size, shape, posture and / or position of the line drawing graphic is changed, and after the scene picture is regenerated, the display effect of the generated scene picture can be adjusted.

[0137] In another embodiment, optionally, the method further comprises:

[0138] Displaying the first picture on the screen according to a second input operation of the user;

[0139] Acquire a second adjustment operation for adjusting the position, size and / or shape of the text information, and obtain the coordinate position of the adjusted text information in the first picture;

[0140] Generate the adjusted line drawing graphic according to the adjusted coordinate position and the corresponding vector diagram template;

[0141] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a third scene image corresponding to the first image.

[0142] Using this implementation mode, after the text information and the line drawing graphics are input into the multimodal image generation model to generate a first scene picture corresponding to the first picture, the first picture can be re-called on the screen through a second input operation on the screen, and the position, size and / or shape of the text information on the first picture can be adjusted through the second input operation. In this way, the line drawing graphics, vector graphics template and scene picture corresponding to the first picture can be regenerated based on the adjusted text information to adjust the display effect of the generated scene picture.

[0143] In another embodiment, optionally, the method further comprises:

[0144] According to a third input operation of the user, a label option corresponding to the text information is displayed on the screen;

[0145] Obtaining a selection operation on the tag option, and obtaining the tag attribute selected by the selection operation;

[0146] Generate the adjusted line drawing graphic according to the selected label attribute and the corresponding vector graphic template;

[0147] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a fourth scene image corresponding to the first image.

[0148] Using this implementation mode, after the text information and the line drawing graphics are input into the multimodal image generation model to generate the first scene picture corresponding to the first picture, the first picture and the corresponding label options can be re-called on the screen through a third input operation on the screen, and the label options can be re-selected through the third input operation. In this way, the label attributes of the generated scene picture can be reselected, and the line drawing graphics, vector graphics template and the scene picture corresponding to the first picture can be regenerated, thereby achieving the purpose of adjusting the display effect of the generated scene picture.

[0149] In an embodiment of the present invention, optionally, the above-mentioned first input operation, second input operation and third input operation can be a sliding operation along a preset trajectory on the touch screen, or a tap operation on a touch button displayed on the touch screen, and can also be other touch operations that can trigger the next execution process. The operation form is not limited here.

[0150] The drawing method described in the embodiment of the present invention utilizes the text information in the first picture created by the user on the screen to complete the line drawings and scene pictures corresponding to the text information in stages on the screen, and can modify each local line drawing, so as to accurately control the overall layout and detailed content of the generated picture. Compared with the traditional method, the present invention greatly reduces the description difficulty of the generative AI prompt engineering, restores the intuitiveness of the painting drawing process, and supports a progressive human-computer interactive collaborative painting experience.

[0151] An embodiment of the present invention further provides a drawing device, such as Figure 3 As shown, the drawing device comprises:

[0152] A first acquisition module 301 is used to acquire a first picture, wherein the first picture includes text information;

[0153] A determination module 302, configured to determine a corresponding vector diagram template according to the semantics of the text information;

[0154] The first generating module 303 is used to generate a line drawing graphic corresponding to the first picture according to the vector graphic template; wherein the line drawing graphic includes the contour lines of the scene corresponding to the semantics of the text information;

[0155] The second generation module 304 is used to input the semantics of the text information and the line drawing graphics into a multimodal image generation model to generate a first scene image corresponding to the first image.

[0156] The drawing device described in this embodiment is used to perform semantic recognition based on the text information (keywords) in the first picture created by the user, determine the corresponding vector graphics model based on the semantics of the recognized text information, and generate the corresponding line drawing graphics. The drawing process can realize the staged drawing from text, line drawing graphics to scene effect graphics, ensuring that the picture of the required scene is accurately generated through the text information. Optionally, the drawing device, wherein the first generation module 303 is also used to:

[0157] Identify the coordinate position of the text information in the first picture;

[0158] The vector diagram template includes a plurality of line arrays; after determining the corresponding vector diagram template, the determination module 302 is further used to:

[0159] Determine the dot matrix coordinates corresponding to the plurality of line arrays according to the coordinate positions; the coordinate positions are located within the coverage area of ​​the dot matrix coordinates;

[0160] The first generating module 303 generates a line drawing corresponding to the first picture according to the vector image template, including:

[0161] Generate a line drawing corresponding to the first picture according to the dot coordinates corresponding to the plurality of line arrays in the vector graphics template.

[0162] Optionally, in the drawing device, the first generating module 303 is further used for:

[0163] According to the semantics of the text information, when it is determined that the text information includes a first target text whose text type is an adjective or a verb, the dot matrix coordinates corresponding to the multiple line arrays are adjusted according to the first target text.

[0164] Optionally, in the mapping device, the determining module 302 is further configured to:

[0165] Outputting label options corresponding to the text information according to the semantics of the text information;

[0166] Obtaining the label attribute corresponding to the text information selected on the label option;

[0167] According to the semantics of the text information, determining the corresponding vector diagram template includes:

[0168] A corresponding vector graphics template is determined according to the semantics of the text information and the corresponding tag attributes.

[0169] Optionally, in the drawing device, the determining module 302 outputs the label options corresponding to the text information according to the semantics of the text information, including:

[0170] According to the semantics of the text information, a label option corresponding to a second target text whose text type is a noun in the text information is output.

[0171] Optionally, in the drawing device, the second generating module 304 is further used for:

[0172] Displaying the line drawing graphic on the screen according to a first input operation of the user;

[0173] Acquire a first adjustment operation for adjusting the position, size and / or shape of the line drawing graphic, and obtain the adjusted line drawing graphic;

[0174] The semantics of the text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a second scene image corresponding to the first image.

[0175] Optionally, in the drawing device, the second generating module 304 is further used for:

[0176] Displaying the first picture on the screen according to a second input operation of the user;

[0177] Acquire a second adjustment operation for adjusting the position, size and / or shape of the text information, and obtain the coordinate position of the adjusted text information in the first picture;

[0178] Generate the adjusted line drawing graphic according to the adjusted coordinate position and the corresponding vector diagram template;

[0179] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a third scene image corresponding to the first image.

[0180] Optionally, in the drawing device, the second generating module 304 is further used for:

[0181] According to a third input operation of the user, a label option corresponding to the text information is displayed on the screen;

[0182] Obtaining a selection operation on the tag option, and obtaining the tag attribute selected by the selection operation;

[0183] Generate the adjusted line drawing graphic according to the selected label attribute and the corresponding vector graphic template;

[0184] The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a fourth scene image corresponding to the first image.

[0185] Optionally, in the drawing device, the first picture further includes auxiliary lines for drawing composition;

[0186] The determination module 302 is further used for:

[0187] According to the auxiliary composition lines, the contour lines of different scenes in the line drawing and / or the interval positions of the contour lines of different scenes are determined.

[0188] Optionally, in the drawing device, the vector image template includes a border line drawing template of a scene corresponding to the semantics of the text information and a skeleton line drawing template of a scene corresponding to the semantics of the text information.

[0189] One embodiment of the present invention further provides a drawing device, comprising: a memory, a processor, and a program stored in the memory and executable on the processor; characterized in that the processor is used to read the program in the memory to implement the drawing method described in any one of the above.

[0190] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A mapping method, characterized in that: include: Get the first picture; The first picture includes text information; Determine a corresponding vector diagram template according to the semantics of the text information; Generate a line drawing graphic corresponding to the first picture according to the vector graphics template; wherein the line drawing graphic includes the contour lines of the scene corresponding to the semantics of the text information; The semantics of the text information and the line drawing graphic are input into a multimodal image generation model to generate a first scene image corresponding to the first image.

2. The drawing method according to claim 1, characterized in that: The method further comprises: Identify the coordinate position of the text information in the first picture; Wherein, the vector diagram template includes a plurality of line arrays; after determining the corresponding vector diagram template, the method further includes: Determine the dot matrix coordinates corresponding to the plurality of line arrays according to the coordinate positions; the coordinate positions are located within the coverage area of ​​the dot matrix coordinates; The step of generating a line drawing corresponding to the first picture according to the vector graphics template includes: Generate a line drawing corresponding to the first picture according to the dot coordinates corresponding to the plurality of line arrays in the vector graphics template.

3. The drawing method according to claim 2, characterized in that: The method further comprises: According to the semantics of the text information, when it is determined that the text information includes a first target text whose text type is an adjective or a verb, the dot matrix coordinates corresponding to the multiple line arrays are adjusted according to the first target text.

4. The drawing method according to claim 1, characterized in that: The method further comprises: Outputting label options corresponding to the text information according to the semantics of the text information; Obtaining the label attribute corresponding to the text information selected on the label option; According to the semantics of the text information, determining the corresponding vector diagram template includes: A corresponding vector graphics template is determined according to the semantics of the text information and the corresponding tag attributes.

5. The drawing method according to claim 4, characterized in that: Outputting label options corresponding to the text information according to the semantics of the text information includes: According to the semantics of the text information, a label option corresponding to a second target text whose text type is a noun in the text information is output.

6. The drawing method according to claim 1, characterized in that: The method further comprises: Displaying the line drawing graphic on the screen according to a first input operation of the user; Acquire a first adjustment operation for adjusting the position, size and / or shape of the line drawing graphic, and obtain the adjusted line drawing graphic; The semantics of the text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a second scene image corresponding to the first image.

7. The drawing method according to claim 1, characterized in that: The method further comprises: Displaying the first picture on the screen according to a second input operation of the user; Acquire a second adjustment operation for adjusting the position, size and / or shape of the text information, and obtain the coordinate position of the adjusted text information in the first picture; Generate the adjusted line drawing graphic according to the adjusted coordinate position and the corresponding vector diagram template; The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a third scene image corresponding to the first image.

8. The drawing method according to claim 4, characterized in that: The method further comprises: According to a third input operation of the user, a label option corresponding to the text information is displayed on the screen; Obtaining a selection operation on the tag option, and obtaining the tag attribute selected by the selection operation; Generate the adjusted line drawing graphic according to the selected label attribute and the corresponding vector graphic template; The text information and the adjusted line drawing graphic are input into the multimodal image generation model to generate a fourth scene image corresponding to the first image.

9. The drawing method according to claim 1, characterized in that: The first picture also includes auxiliary composition lines; Wherein, the method further comprises: According to the auxiliary composition lines, the contour lines of different scenes in the line drawing and / or the interval positions of the contour lines of different scenes are determined.

10. The drawing method according to claim 1, characterized in that: The vector graphics template includes a frame line drawing template of a scene corresponding to the semantics of the text information and a skeleton line drawing template of a scene corresponding to the semantics of the text information.

11. A drawing device, characterized in that: include: A first acquisition module, used to acquire a first picture; The first picture includes text information; A determination module, used to determine a corresponding vector diagram template according to the semantics of the text information; A first generating module is used to generate a line drawing graphic corresponding to the first picture according to the vector graphic template; wherein the line drawing graphic includes the contour lines of the scene corresponding to the semantics of the text information; The second generation module is used to input the semantics of the text information and the line drawing graphics into a multimodal image generation model to generate a first scene image corresponding to the first image.

12. A drawing device comprising: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the drawing method as described in any one of claims 1 to 10.

Images