Data generation method and device, electronic equipment and readable storage medium

Abstract

The invention discloses a data generation method and device, electronic equipment and a readable storage medium, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining a first description code of a sample vector image; converting the sample vector image into a sample grating image; inputting a first cue word and the sample grating image into a graph-to-text model, and outputting a first description text corresponding to the sample vector image; wherein the first description text is used for describing at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle and image proportion; according to a second description text, carrying out image parameter transformation processing on the sample vector image to obtain an auxiliary vector image, and obtaining a second description code of the auxiliary vector image; and generating a group of training data corresponding to the sample vector image based on the first description code, the first description text, the second description code and the second description text.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a data generation method, device, electronic device and readable storage medium. Background Art

[0002] With the continuous development of image processing technology, the application scenarios of images are becoming more and more extensive, such as logo design, graphic design, web page production, advertising production, game development, virtual reality and auxiliary medical care. In order to enrich the number and content of images, the Wensheng graph model came into being. At present, the Wensheng graph model can only generate raster images, not vector images, and cannot be applied to related scenarios of vector images. Therefore, it is necessary to train a Wensheng graph model that can generate vector images.

[0003] However, in the related art, it is difficult to obtain the training data required in the training process, which requires a lot of time and money, resulting in a relatively high training cost for the text graph model. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a data generation method, device, electronic device and readable storage medium, which can easily generate training data, avoid spending a lot of time and money costs, and thus reduce the training cost of the text graph model.

[0005] In a first aspect, an embodiment of the present application provides a data generation method, the method comprising:

[0006] Obtain a first description code of the sample vector image;

[0007] converting the sample vector image into a sample raster image;

[0008] Inputting the first prompt word and the sample raster image into the image-to-text model, and outputting a first description text corresponding to the sample vector image; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image scale;

[0009] According to the second description text, transform the image parameters of the sample vector image to obtain an auxiliary vector image, and obtain a second description code of the auxiliary vector image;

[0010] Based on the first description code, the first description text, the second description code and the second description text, a set of training data corresponding to the sample vector image is generated; wherein the training data is used to train a text graph model.

[0011] In a second aspect, an embodiment of the present application provides a data generating device, the device comprising:

[0012] A first acquisition module, used for acquiring a first description code of a sample vector image;

[0013] A first processing module, configured to convert the sample vector image into a sample raster image;

[0014] A second processing module is used to input the first prompt word and the sample raster image into the image-to-text model, and output a first description text corresponding to the sample vector image; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image scale;

[0015] A third processing module is used to transform the image parameters of the sample vector image according to the second description text to obtain an auxiliary vector image, and obtain a second description code of the auxiliary vector image;

[0016] The first generating module is used to generate a set of training data corresponding to the sample vector image based on the first description code, the first description text, the second description code and the second description text; wherein the training data is used to train the text graph model.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the data generation method described in the first aspect are implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the data generation method described in the first aspect are implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the steps of the data generation method described in the first aspect.

[0020] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the data generation method as described in the first aspect.

[0021] In an embodiment of the present application, a first description code of a sample vector image is obtained; the sample vector image is converted into a sample raster image; a first prompt word and a sample raster image are input into a graph-based text model, and a first description text corresponding to the sample vector image is output; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, image ratio; according to the second description text, the sample vector image is subjected to image parameter transformation processing to obtain an auxiliary vector image, and a second description code of the auxiliary vector image is obtained; based on the first description code, the first description text, the second description code, and the second description text, a set of training data corresponding to the sample vector image is generated; wherein the training data is used to train the graph-based text model. It can be seen that in an embodiment of the present application, the above process of generating training data can conveniently generate training data without the user having to manually collect training data or spend money to purchase training data, thereby avoiding the high time and money costs, and thus reducing the training cost of the graph-based text model. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a data generation method provided by some embodiments of the present application;

[0023] Figure 2 is an example diagram of a sample vector image provided by some embodiments of the present application;

[0024] Figure 3 is an example diagram of a generation process of a sample grating image provided by some embodiments of the present application;

[0025] Figure 4 is a flowchart of a training process of a text graph model provided by some embodiments of the present application;

[0026] Figure 5 is a flow chart of a process of using a Wensheng graph model provided in some embodiments of the present application;

[0027] Figure 6 It is one of the example diagrams of the text image interface of the electronic device provided in some embodiments of the present application;

[0028] Figure 7 This is a second example diagram of a text image interface of an electronic device provided in some embodiments of the present application;

[0029] Figure 8 This is a third example diagram of a text image interface of an electronic device provided in some embodiments of the present application;

[0030] Fig. 9 This is a fourth example diagram of a text image interface of an electronic device provided in some embodiments of the present application;

[0031] Fig.10 This is a fifth example diagram of a text image interface of an electronic device provided in some embodiments of the present application;

[0032] Fig.11 This is a sixth example diagram of a text image interface of an electronic device provided in some embodiments of the present application;

[0033] Fig.12 is a structural block diagram of a data generating device provided by some embodiments of the present application;

[0034] Fig.13 is a schematic diagram of the structure of an electronic device provided by some embodiments of the present application;

[0035] Fig.14 It is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings of the embodiments of the present application to clearly describe the technical solutions of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present application belong to the scope of protection of this application.

[0037] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally a class, and the number of objects is not limited. For example, the target object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0038] To facilitate understanding, some concepts and application scenarios involved in the embodiments of the present application are first introduced.

[0039] 1. Concept

[0040] Raster images, also known as bitmap images, are a type of image that uses a grid of pixels to represent an image. Each pixel in a raster image has its own color and position information. Since the number of pixels in a raster image is fixed, raster images will be distorted at different resolutions. Raster images are often used for scenes that require rich details and realism, such as photographs, graphic designs, and website images. Common formats include: PNG, JPG, WEBP, etc.

[0041] Vector images are a type of image that uses mathematical formulas to represent images. These formulas describe the various shapes, lines, colors, and positions in an image, so vector images can maintain their clarity without distortion at different sizes and resolutions. Vector images are suitable for icons, logos, diagrams, and simple illustrations that need to be scaled up or down while maintaining clarity. Common formats include: SVG, AI, EPS, etc.

[0042] Scalable Vector Graphics (SVG) is a format for vector images and an XML markup language used to describe two-dimensional graphics. Vector images in SVG format are stored in text form. Since they are based on mathematical descriptions rather than pixels, they can be scaled to any size without distortion.

[0043] Extensible Markup Language (XML) is a simplified and modified version of Standard Generalized Markup Language (SGML). It mainly uses Extensible Markup Language and Extensible Stylesheet Language.

[0044] Generative Pre-trained Transformer (GPT) is a natural language processing model that can learn the structure and rules of language through a large amount of text training data to generate high-quality text.

[0045] Prompt is a natural language text that describes the task that the artificial intelligence should perform.

[0046] Stable Diffusion is a stable diffusion model based on latent variable models, which is mainly used to generate detailed images based on text descriptions. It is a generative AI that operates in the latent space through forward diffusion and backward diffusion processes, and combines variational autoencoders to compress image information. The principle of Stable Diffusion is to gradually transform the noisy image into the target image through a simulated diffusion process. It has strong stability and controllability, and can generate high-quality, high-resolution and diverse images. It can also be applied to other tasks, such as infill drawing, outfill drawing, and the transformation of image-to-image under the guidance of prompt words.

[0047] The Image-to-Text model is a multimodal model that converts image content into text descriptions. It uses deep learning technology, especially the combination of convolutional neural networks and recurrent neural networks, to understand and analyze input images and convert them into natural language text.

[0048] The Text-to-Image model, also a multimodal model, is a generative model that combines natural language processing with computer vision technology. It generates corresponding images by parsing and understanding the input text description. Its core function is to interpret the natural language descriptions input by users and generate visual content based on these descriptions.

[0049] A data generation method provided in an embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0050] It should be noted that the data generation method provided in the embodiment of the present application is applicable to electronic devices. In practical applications, the electronic devices include but are not limited to: mobile terminals such as mobile phones, tablet computers, laptops, PDAs, and computer devices such as servers and desktops.

[0051] Figure 1 is a flow chart of a data generation method provided by some embodiments of the present application, such as Figure 1 As shown, the method includes the following steps: step 101, step 102, step 103, step 104 and step 105;

[0052] In step 101, a first description code of a sample vector image is obtained.

[0053] In the embodiment of the present application, a basic training set can be obtained, which includes multiple different sample vector images, for example, 1000 different sample vector images. For each sample vector image, the processing in steps 101 to 105 is performed to obtain training data corresponding to each sample vector image. The training data corresponding to all sample vector images is used to train a text-generated graph model, wherein the text-generated graph model can be used to generate a vector image desired by a user according to the text input by the user.

[0054] For example, a sample vector image in the basic training set is Figure 2 As shown, the sample vector image contains a black running person.

[0055] In an embodiment of the present application, the format of the sample vector image may be SVG format, or may be other formats, such as CSS format, AI format, EPS format, etc.

[0056] In the embodiment of the present application, the first description code is an extensible markup language (XML) code, or may be other types of codes.

[0057] Considering the following advantages of XML code: 1) Wide applicability, XML can be widely used anywhere on the Web to meet the needs of network applications; 2) Powerful data processing capabilities, XML is based on text format, can be freely marked, and has good scalability; 3) Structured data, XML files strictly stipulate the structural hierarchy of data, and show the relationship between data through a tree structure, which is convenient for program parsing and processing; 4) Self-explanatory, XML documents usually contain descriptions of relevant data, and tags and element names usually clearly express the meaning of the data and are easy to understand; 5) Cross-platform compatibility, XML is a format that is independent of programming languages ​​and operating systems, and can read and parse XML data on different systems and platforms; 6) Text readability, XML documents are stored in plain text, with good readability and editability, and people can directly view and modify the content in XML files; 7) Custom tags, XML allows users to customize tags (labels), each tag represents a specific type of data or meaning, and this feature enables it to describe various complex data structures.

[0058] In some embodiments, the first description code is an XML code. Accordingly, the above step 201 may include the following steps: determining the image format of the sample vector image; when the image format of the sample vector image is a scalable vector graphics format, obtaining the first description code from the sample vector image; when the image format of the sample vector image is a format other than the scalable vector graphics format, converting the sample vector image into an image in a scalable vector graphics format, and obtaining the first description code from the converted image in a scalable vector graphics format.

[0059] In the embodiment of the present application, for vector images in SVG format, no other conversion operations are required, and the XML code of SVG can be directly obtained through the file. For vector images in other formats such as CSS, AI, EPS, etc., vector images in other formats can be converted into vector images in SVG format using software such as Illustrator, Inkscape, Sketch or Affinity Designer, and finally the corresponding XML code is obtained.

[0060] In the embodiment of the present application, since XML code has the advantages of wider applicability, scalability, easy processing, self-explanatory and readability, the XML code of the sample vector image is used as one of the training data of the text graph model, which can improve the flexibility of model training.

[0061] In step 102, a sample vector image is converted into a sample raster image.

[0062] In the embodiment of the present application, since the current image-generated text model can accept raster images, if you want to obtain the description text of the sample vector image, first, you need to convert the sample vector image into a sample raster image; then, input the sample raster image into the image-generated text model, and output the description text of the sample raster image. Since the sample raster image and the sample vector image are only different in image type, but the image parameters are the same, and the description text is used to describe the image parameters of the image, the description text of the sample raster image is the description text of the sample vector image.

[0063] In some embodiments, the above step 102 may include the following steps: step 1021 and step 1022;

[0064] In step 1021, the sample vector image is converted into a four-channel raster image in RGBA format; wherein RGBA represents a red channel, a green channel, a blue channel, and a transparent channel, respectively.

[0065] In the embodiment of the present application, considering that a vector image is converted into a raster image, due to the presence of a transparent layer, the converted raster image is usually a four-channel raster image in RGBA format.

[0066] In the embodiment of the present application, a raster graphics algorithm may be used to convert the sample vector image into a four-channel raster image in RGBA format.

[0067] In step 1022, for each pixel position in the four-channel raster image, if the value of the transparent channel at the pixel position is zero, the value of the transparent channel at the pixel position is deleted, and the values ​​of the red channel, green channel, and blue channel at the pixel position are retained; if the value of the transparent channel at the pixel position is not zero, the value of the transparent channel at the pixel position is deleted, and the values ​​of the red channel, green channel, and blue channel at the pixel position are replaced with the values ​​of the red channel, green channel, and blue channel corresponding to the preset background pattern, respectively, to obtain a sample raster image in RGB format.

[0068] In the embodiment of the present application, considering that the raster image obtained by converting the sample vector image is a four-channel raster image in RGBA format, and the existing image-to-text model accepts raster images in RGB format, it is necessary to convert the four-channel raster image in RGBA format into a raster image in RGB format that can be processed by the image-to-text model.

[0069] In the embodiment of the present application, the preset background pattern may be a pattern composed of "white-gray" squares, or may be other patterns, which is not limited in the embodiment of the present application.

[0070] For example, the setting is as follows Figure 3The "white-gray" checkered background pattern 301 shown is used to fill the background of the four-channel raster image in RGBA format. When the transparency channel, that is, the A channel, has a non-zero value, it means that this position is transparent, corresponding to the background area, and the preset background pattern is used to fill this position. After deleting the A channel data, the values ​​of the three channels of the filling pattern are retained; when the value of the A channel is 0, it means that this position is opaque, corresponding to the location of the element, after deleting the A channel data, only the initial values ​​of the other three channels are retained. Applying the above processing method, such as Figure 3 As shown, based on the background pattern 301 and the four-channel raster image 302 in the RGBA format, a style raster image 303 in the RGB format is obtained.

[0071] Different from the related art in which a raster image in RGBA format is converted into a raster image in RGB format by directly deleting the value of the transparent channel, which will affect the subsequent image-generated text model's recognition of the image. In the embodiment of the present application, for each pixel position in a four-channel raster image, if the value of the transparent channel at the pixel position is zero, the value of the transparent channel at the pixel position is deleted, and the values ​​of the red channel, green channel, and blue channel at the pixel position are retained; if the value of the transparent channel at the pixel position is not zero, the value of the transparent channel at the pixel position is deleted, and the values ​​of the red channel, green channel, and blue channel at the pixel position are replaced with the values ​​of the red channel, green channel, and blue channel corresponding to the preset background pattern, respectively, to obtain a sample raster image in RGB format, thereby avoiding affecting the subsequent image-generated text model's recognition of the image.

[0072] In step 103, the first prompt word and the sample raster image are input into the image-to-text model, and the first description text corresponding to the sample vector image is output; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image ratio.

[0073] In the embodiment of the present application, the input of the image-to-text model is a first prompt word (prompt) and a sample raster image, and the output of the image-to-text model is a description text of the sample raster image. Since the sample raster image and the sample vector image are only different in image type but have the same image parameters, and the description text is used to describe the image parameters of the image, the description text of the sample raster image is the first description text corresponding to the sample vector image.

[0074] In the embodiment of the present application, the first prompt word is used to prompt the image-to-text model to output a description text of the sample raster image according to the sample raster image. For example, the first prompt word is: "This is an image with a transparent grid as the background. Please describe the image in detail."

[0075] For example, the first prompt word is: "This is an image with a transparent grid as the background. Please describe this image in detail." Figure 2 The sample raster image corresponding to the sample vector image shown is input into the image-to-text model, and the following first description text is output: "A simple black human outline symbol; the human figure is in a running posture, and the limbs have a clear sense of movement; the head is a circle; the body is composed of a straight line and two oblique lines, and is slightly inclined; the left arm is extended forward, and the right arm is swung backward, showing the dynamics of running; the left leg is bent and raised, and the right leg is spread backward, indicating that it is moving quickly."

[0076] In the embodiment of the present application, the first description text corresponding to the sample vector image is also one of the training data of the text graph model.

[0077] In step 104, according to the second description text, the sample vector image is subjected to image parameter transformation processing to obtain an auxiliary vector image, and a second description code of the auxiliary vector image is obtained.

[0078] In an embodiment of the present application, in order to ensure that the finally trained text graph model has the ability to conduct multiple rounds of dialogue and modify vector images, the image parameters of the sample vector image can be transformed according to the second description text, and the corresponding description code, i.e., the second description code, can be retained.

[0079] In the embodiments of the present application, the transformation processing of image parameters includes, but is not limited to: color replacement processing, background replacement processing, rotation processing, translation processing, scaling processing, adding / deleting elements, etc.

[0080] In the embodiment of the present application, the second description text is used to indicate which image parameters of the sample vector image are to be modified, and the specific modified values ​​used.

[0081] Exemplarily, the second description text is: "Change the color of the person in the image to red".

[0082] Exemplarily, the second description text is: “Replace the background of the image with a stripe pattern”.

[0083] Exemplarily, the second description text is: "Rotate the person in the image 90 degrees".

[0084] Exemplarily, the second description text is: "Translate the person in the image 1 cm to the left".

[0085] Exemplarily, the second description text is: "Scale the person in the image by 0.5 times".

[0086] Exemplarily, the second description text is: "Delete the building in the image".

[0087] Exemplarily, the second description text is: "Add a puppy at the lower left corner of the image."

[0088] In some embodiments, when performing color replacement processing on the sample vector image, the color specified by the fill field in the first description code can be replaced according to actual needs. For example, if the color of an element is changed to red, the fill field is "ff0000".

[0089] In some embodiments, when performing background replacement processing on a sample vector image, taking an element in SVG format as an example, the background is all SVG elements such as rect, path, and polygon, which is the first element in the code. If the first element in the first description code is rect, and the width and height are equal to the viewBox of SVG, then the rect element is replaced with the desired background element; if the first element is not the background of SVG, then the desired background element is directly added.

[0090] In some embodiments, when rotating the elements in the sample vector image, the position of each element may be recalculated according to the rotation requirements, including the rotation center and the rotation angle.

[0091] In some embodiments, when translating elements in a sample vector image, there are two ways to implement the translation of the elements: the first is to recalculate the specific coordinate position of the element according to the translation requirements, including the translation distance of left, right, up, and down; the second is to use the transform field to translate. For example,<rect width="15"height="15"fill="none" / > It needs to be translated downward by 3 and to the right by 4. In the first method,<rect x="3"y="4"width="15"height="15"fill="none" / > In the second method,<rect width="15"height="15"fill="none"transform="translate(3,4)" / > .

[0092] In some embodiments, when scaling the elements in the sample vector image, there are two ways to implement the scaling of the elements: the first is to recalculate the specific coordinate position of the element according to the scaling requirements, including the scaling scale; the second is to use the transform field to perform scaling. For example:<rect width="15"height="15"fill="none" / > It needs to be reduced by 0.5 times. In the first method,<rect width="7.5"height="7.5"fill="none" / > In the second method,<rect width="15"height="15"fill="none"transform="scale(0.5)" / > In addition, if you need to scale the entire element inside, you can modify the value of the viewbox.

[0093] In some embodiments, when adding or deleting elements in a sample vector image, designated elements may be deleted or added according to actual needs; wherein, the modification of the code may be completed by debugging by relevant developers.

[0094] In the embodiment of the present application, the second description text and the second description code are also one of the training data of the text graph model.

[0095] In step 105, a set of training data corresponding to the sample vector image is generated based on the first description code, the first description text, the second description code and the second description text; wherein the training data is used to train the text graph model.

[0096] In the embodiment of the present application, each set of training data includes: a first description code, a first description text, a second description code, and a second description text corresponding to a sample vector image.

[0097] In the embodiment of the present application, when using each set of training data to train the text graph model, the first description code is used as input and the first description text is used as the corresponding output target; the second description code is used as input and the second description text is used as the corresponding output target.

[0098] For example, Figure 2 The sample vector image shown has the first description code:

[0099] <svg xmlns="http: / / www.w3.org / 2000 / svg"

[0100] width="1em"height="1em"viewBox="0 0 15 15">

[0101] <rect width="15"height="15"fill="none" / >

[0102] <path fill="currentColor"d="M5 3a1 1 0 1 1 0-2a1 1 0 0 1 0 2m7.57H10L97L8 5.25L9 5l2.3 1a.539.539 0 0 0.4-1L9 4H7L5 5L4 6H2.5a.5.5 0 0 001H5l1-1l1 2l-2 2v3.5a.5.5 0 0 0 1 0v-3.11L8 9l1 2h3.5a.5.5 0 0 0 0-1" / >

[0103] .

[0104] Will Figure 2 The sample vector image shown is converted into a raster image and input into the graphic text model. The first descriptive text output is: "A simple black human outline symbol; the human is in a running posture, with obvious movement of the limbs; the head is a circle; the body is composed of a straight line and two oblique lines, and is slightly tilted; the left arm is extended forward, and the right arm is swung backward, showing the dynamics of running. The left leg is bent and raised, and the right leg is spread backward, indicating that it is moving quickly."

[0105] Will Figure 2 The color of the person in the sample vector image shown is changed to red, and the second description text is: "Change the color of the person in the image to red".

[0106] Accordingly, the second description code is:

[0107] <svg xmlns="http: / / www.w3.org / 2000 / svg"

[0108] width="1em"height="1em"viewBox="0 0 15 15">

[0109] <rect width="15"height="15"fill="none" / >

[0110] <path fill="#ff0000"d="M5 3a1 1 0 1 1 0-2a1 1 0 0 1 0 2m7.5 7H10L97L85.25L9 5l2.3 1a.539.539 0 0 0.4-1L9 4H7L5 5L4 6H2.5a.5.5 0 0 0 0 1H5l1-1l12l-2 2v3.5a.5.5 0 0 0 1 0v-3.11L8 9l1 2h3.5a.5.5 0 0 0 0-1" / >

[0111] .

[0112] The first description code, the first description text, the second description code and the second description text in the above example constitute Figure 2 The sample vector images shown correspond to the training data.

[0113] It can be seen that in the embodiment of the present application, in the above process of generating training data, on the one hand, the training data required for the text-based image model training can be automatically generated without user participation, which is relatively convenient. On the other hand, the training data includes basic training data such as the first description code and the first description text. The basic training data can train a model with the basic text-to-image function. In addition, it also includes advanced training data such as the second description code and the second description text. The advanced training data cooperates with the basic training data to train a model that not only has the ability to generate images from text, but also has multi-round dialogue capabilities and supports users to perform secondary editing and modification on the generated vector images, so that the quality of the generated training data is higher.

[0114] As can be seen from the above embodiment, in this embodiment, a first description code of a sample vector image is obtained; the sample vector image is converted into a sample raster image; the first prompt word and the sample raster image are input into the graph-text model, and a first description text corresponding to the sample vector image is output; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, image ratio; according to the second description text, the sample vector image is subjected to image parameter transformation processing to obtain an auxiliary vector image, and a second description code of the auxiliary vector image is obtained; based on the first description code, the first description text, the second description code and the second description text, a set of training data corresponding to the sample vector image is generated; wherein the training data is used to train the graph-text model. It can be seen that in the embodiment of the present application, the above process of generating training data can conveniently generate training data without the user having to manually collect training data or spend money to purchase training data, thereby avoiding the user from spending more time and money costs, thereby reducing the training cost of the graph-text model.

[0115] In another embodiment provided in the present application, based on Figure 1 The training data generated in the embodiment shown in the figure is used to train the text graph model, such as Figure 4 As shown, the training process of the Wensheng graph model may include the following steps: step 401, step 402 and step 403;

[0116] In step 401, an initial backbone network is constructed based on the generative pre-trained transformer.

[0117] In some embodiments, the open source GPT model can be directly determined as the initial backbone network.

[0118] In some embodiments, in order to improve the coding efficiency of the GPT model, the above step 401 may include the following steps: step 4011, step 4011 and step 4013;

[0119] In step 4011, a first word in the first description code is extracted.

[0120] In an embodiment of the present application, it is taken into consideration that some new words will be added to the description code of the sample vector image. For example, some special characters are newly created in the XML code of the sample vector image, such as svg, xmlns, stroke-width, etc. Since the above newly added words exceed the vocabulary range of the GPT model's vocabulary, in order to improve the encoding efficiency of the GPT model, it is necessary to expand the vocabulary of the GPT model.

[0121] In an embodiment of the present application, a vocabulary extraction tool, such as SentencePiece, tiktoken, etc., may be used to perform vocabulary extraction on the description codes of all sample vector images.

[0122] In step 4012, if the first word is not in the vocabulary of the generative pre-trained transformer, the first word is added to the vocabulary of the generative pre-trained transformer.

[0123] In the embodiment of the present application, if the first word is in the vocabulary of the GPT model, for example, the first word is width, height, etc., then the vocabulary of the GPT model is not updated. If the first word is not in the vocabulary of the GPT model, for example, the first word is svg, xmlns, stroke-width, etc., then the first word is added to the vocabulary of the GPT model, and the feature vector of the corresponding word is randomly initialized to a mean of 0 and a variance of 0.02.

[0124] In step 4013, the output dimension of the output layer of the generative pre-trained converter is adjusted according to the first vocabulary, and the adjusted generative pre-trained converter is determined as the initial backbone network.

[0125] In an embodiment of the present application, if new vocabulary is added to the vocabulary of the GPT model, the same number of model outputs must also be added to the final output layer of the GPT model, and the corresponding weights are also initialized to a mean of 0 and a variance of 0.02.

[0126] It can be seen that in the embodiment of the present application, the vocabulary of the GPT model can be updated accordingly according to the vocabulary of the code described in the training data, so as to improve the encoding efficiency of the GPT model and thereby improve the performance of the text graph model.

[0127] In step 402, the first description text in the training data is used as the input of the initial backbone network, the first description code in the training data is used as the output target of the initial backbone network, and the initial backbone network is trained to obtain an intermediate model.

[0128] In an embodiment of the present application, the first description text in the training data is used as the input of the initial backbone network, the initial backbone network outputs the predicted description code, the loss value between the predicted description code and the first description code is calculated, and the network parameters of the initial backbone network are updated according to the loss value. For the network obtained after the parameter update, the first description text of the next sample vector image is continued to be used as the input of the network, the network outputs the predicted description code, the loss value between the predicted description code and the first description code is calculated, and the parameters of the network are updated again according to the loss value. The above training process is repeated using the first description texts and first description codes of different sample vector images until the network converges and an intermediate model is obtained.

[0129] In the embodiment of the present application, the intermediate model is also a multimodal model in nature. In the actual use stage, the input of the model is the description text, and the output of the model is the description code of the vector image. The vector image can be obtained by rendering the description code.

[0130] In the embodiment of the present application, since the training data of the intermediate model is the original description text and original description code of the sample vector image, the intermediate model has the function of generating images from text.

[0131] In step 403, the second description text in the training data is used as the input of the intermediate model, and the second description code in the training data is used as the output target of the intermediate model. The intermediate model is trained to obtain a text graph model.

[0132] In the embodiment of the present application, in order to finally obtain the trained text-generated graph model that not only has the ability to generate images from text, but also has the ability of multi-round dialogue and the ability to support users to perform secondary editing and modification on the generated vector images, the second description text and the second description code obtained by secondary editing and modification of the sample vector image can be used to perform model training on the basis of the intermediate model to obtain the text-generated graph model.

[0133] It can be seen that in the embodiments of the present application, based on the training data corresponding to different sample vector images, a text-generated graph model capable of outputting vector images can be trained. The text-generated graph model not only has the ability to generate images from text, but also has multi-round dialogue capabilities and the ability to support users to perform secondary editing and modification of the generated vector images. More vector images can be obtained, thereby enriching the application scenarios of the text-generated graph model.

[0134] In another embodiment provided by the present application, after the text graph model is trained, a vector image can be generated based on the text graph model, that is, the use stage of the text graph model, such as Figure 5 As shown, the process of using the Wensheng graph model may include the following steps: step 501, step 502, step 503, step 504, step 505 and step 506;

[0135] In step 501, a third description text input by a user is received; wherein the third description text is used to describe image parameters of the first vector image.

[0136] In the embodiment of the present application, the first vector image is a vector image generated during the use phase of the text graph model, and is a vector image that the user expects to generate.

[0137] In the embodiment of the present application, when the Wensheng graph model is actually used, the user can describe the appearance of the desired vector image in words to form a third description text. For example, the third description text is: "Please draw a black running person, a simple drawing".

[0138] In step 502, the second prompt word and the third description text are input into the text graph model for processing, and the third description code of the first vector image is output.

[0139] In the embodiment of the present application, the second prompt word is used to prompt the Wensheng image model to output the corresponding third description code according to the third description text. For example, the second prompt word is: "This is a paragraph of text, please generate a vector image according to this paragraph of text."

[0140] In the embodiment of the present application, the input of the text-based graph model is the second prompt word (prompt) and the third description text, and the output of the text-based graph model is the third description code of the first vector image.

[0141] For example, the third description text is: "Please draw a black running person, a simple sketch", and the second prompt word is: "This is a paragraph of text, please generate a vector image based on this paragraph of text". After the user enters the third description text, the text-generated graph model outputs the following code based on the third description text and the second prompt word, i.e., the third description code:

[0142] <svg xmlns="http: / / www.w3.org / 2000 / svg"

[0143] width="1em"height="1em"viewBox="0 0 15 15">

[0144] <rect width="15"height="15"fill="none" / >

[0145] <path fill="currentColor"d="M5 3a1 1 0 1 1 0-2a1 1 0 0 1 0 2m7.57H10L97L8 5.25L9 5l2.3 1a.539.539 0 0 0.4-1L9 4H7L5 5L4 6H2.5a.5.5 0 0 001H5l1-1l1 2l-2 2v3.5a.5.5 0 0 0 1 0v-3.11L8 9l1 2h3.5a.5.5 0 0 0 0-1" / >

[0146] .

[0147] In step 503, image rendering is performed based on the third description code to display the first vector image.

[0148] For example, the Wensheng graph model runs on an electronic device, and the third description text is: "Please draw a black running person, a simple drawing", such as Figure 6 As shown, a Wensheng graph model is running on the electronic device 60. The user can input a third description text 62 on the display screen 61 of the electronic device 60. The electronic device 60 transmits the third description text 62 to the Wensheng graph model. The Wensheng graph model outputs a corresponding third description code. The electronic device 60 renders the third description code into a first vector image 63 and displays it on the display screen 61 in the form of a dialog box.

[0149] In some embodiments, considering the wide application of vector images, different needs may require quite different image formats. For example, users need to use vector images in SVG format when making PPT, drawing Visio images, graphic design and other scenarios. They need to use SVG code when making front-end pages, and need PNG raster images for daily image viewing and sharing. In order to meet the needs of users in different scenarios, four controls can be designed in the interactive interface for users to use.

[0150] For example, Figure 6 As shown, four controls can also be displayed around the first vector image 63 (or the user can long press the image to trigger the four controls to pop up), namely an "Edit" control 64, a "Save" control 65, a "Copy" control 66 and a "Find" control 67; among which, the "Edit" control 64 is used to trigger editing of the first vector image 63, the "Save" control 65 is used to trigger saving of the first vector image 63, the "Copy" control 66 is used to trigger copying of the first vector image 63, and the "Find" control 67 is used to trigger searching for vector images with contents similar to the first vector image 63.

[0151] Exemplarily, when the user clicks the "Save" control 65, a window may pop up, and the user may save a vector image in SVG format, or a raster image in PNG, JPG, etc. In addition, when the user saves a raster image, the resolution of the image may be customized, or the default resolution may be 512×512.

[0152] Exemplarily, when the user clicks the "copy" control 66, the vector image in SVG format is copied by default. When the user pastes it into software such as PPT and Visio, the vector image in SVG format is directly pasted; when the user pastes it into the chat interface, the vector image in SVG format is automatically converted into a raster image.

[0153] Exemplarily, when the user clicks the "Find" control 67, other vector images with similar contents to the first vector image previously generated by the text graph model will pop up in the form of a window, and the user can select a satisfactory image. After selecting, exit the window and the current image will be replaced with the image selected by the user.

[0154] In step 504, a fourth description text input by a user for the first vector image is received; wherein the fourth description text is used to instruct to perform image parameter transformation processing on the first vector image.

[0155] In some embodiments, the fourth description text is a description text input for the first vector image. For example, the fourth description text is: "Change the image to dark gray", or the fourth description text is: "Reduce the image by 0.5 times", or the fourth description text is: "Enlarge the head of the person in the image".

[0156] In some embodiments, the fourth description text is a description text input to the third description code of the first vector image. In this case, the fourth description text is used to transform the image parameters of the first vector image by modifying the code content of the third description code.

[0157] In step 505, a fourth description code of a second vector image is generated according to a fourth description text; wherein the second vector image is an image obtained by transforming image parameters of a first vector image based on the fourth description text.

[0158] In some embodiments, when the fourth description text is a description text input for the first vector image, the above step 505 may include the following steps: inputting the third prompt word and the fourth description text into the text graph model for processing, and outputting a fourth description code.

[0159] In some embodiments, when the fourth description text is a description text input for the third description code of the first vector image, the above step 505 may include the following steps: modifying the third description code according to the fourth description text to obtain a fourth description code.

[0160] In step 506, image rendering is performed based on the fourth description code to display the second vector image.

[0161] For example, the fourth description text is: "Change the person in the image to dark gray", and the user clicks Figure 6 After the "Edit" control 64 in Figure 7 As shown, the fourth description text 70 is input, the electronic device 60 transmits the fourth description text 70 and the historical input text to the Wensheng graph model, the Wensheng graph model outputs the corresponding fourth description code, the electronic device 60 renders the fourth description code into a second vector image 71, and displays it in the form of a dialog box on the display screen 61. In addition, four controls can also be displayed around the second vector image 71.

[0162] For example, the fourth description text is: "Reduce the image by 0.5 times", and the user clicks Figure 6 After the "Edit" control 64 in Figure 8As shown, the fourth description text 80 is input, the electronic device 60 transmits the fourth description text 80 and the historical input text to the Wensheng graph model, the Wensheng graph model outputs the corresponding fourth description code, the electronic device 60 renders the fourth description code into a second vector image 81, and displays it in the form of a dialog box on the display screen 61. In addition, four controls can also be displayed around the second vector image 81.

[0163] For example, the fourth description text is: "Enlarge the head of the person in the image", and the user clicks Figure 6 After the "Edit" control 64 in Fig. 9 As shown, the fourth description text 90 is input, and the electronic device 60 transmits the fourth description text 90 and the historical input text to the text graph model, and the text graph model outputs the corresponding fourth description code, such as Fig. 9 As shown, the electronic device 60 renders the fourth description code into a second vector image 91, and displays it in a dialog box on the display screen 61. In addition, four controls can also be displayed around the second vector image 91.

[0164] For example, sometimes the user needs to make some very minor modifications to the first vector image output by the text model. In this case, if the user is familiar with the SVG code format, the third description code output by the text model can be directly edited. Figure 6 After the "Edit" control 64 in Fig.10 As shown, an editing window 100 is displayed in a pop-up window below the display screen 61 of the electronic device 60, and the editing window 100 contains the XML code of the first vector image. The user can modify the XML code in the editing window 100, and after the user modifies the XML code, the electronic device 60 can re-render the modified image according to the modified XML code.

[0165] In some embodiments, after the above step 503, the following steps may also be included: according to the third description text, obtaining at least one candidate description code from the historical description codes output by the text graph model; wherein the content similarity between the third description text and the historical description text corresponding to the candidate description code is higher than a first threshold; performing image rendering based on at least one candidate description code, and displaying at least one candidate vector image similar to the first vector image, thereby providing the user with more optional vector images.

[0166] For example, the user clicks Figure 6 After the "Find" control 67 in Fig.11As shown, the electronic device 60 displays other vector images similar to the first vector image 63 previously generated by the text graph model in the form of a window 110 on the display screen 61 , and the user can select a vector image that meets his or her needs from the window 110 .

[0167] In an embodiment of the present application, it is also possible to support users uploading description codes of existing vector images. After uploading, the code will be presented in the dialog box in the form of a vector image, and the user can continue to edit the image in the form of text.

[0168] It can be seen that in the embodiments of the present application, the Wensheng graph model can act as a professional graphic design assistant. The user inputs the drawing requirements into the Wensheng graph model, and the Wensheng graph model generates a description code for the vector image desired by the user. The front-end page renders the description code to generate the vector image desired by the user, and finally presents it to the user. It can not only help the user generate vector images using text, but also edit and modify the vector images according to the user's own needs, or copy them to other applications for use, enriching the application scenarios of the Wensheng graph model.

[0169] The data generation method provided in the embodiment of the present application can be executed by a data generation device. In the embodiment of the present application, the data generation device provided in the embodiment of the present application is described by taking the data generation method executed by the data generation device as an example.

[0170] Fig.12 is a structural block diagram of a data generating device provided by some embodiments of the present application, such as Fig.12 As shown, the data generating device 1200 may include: a first acquisition module 1201, a first processing module 1202, a second processing module 1203, a third processing module 1204 and a first generating module 1205;

[0171] A first acquisition module 1201 is used to acquire a first description code of a sample vector image;

[0172] A first processing module 1202, configured to convert the sample vector image into a sample raster image;

[0173] The second processing module 1203 is used to input the first prompt word and the sample raster image into the image-to-text model, and output a first description text corresponding to the sample vector image; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image ratio;

[0174] The third processing module 1204 is used to transform the image parameters of the sample vector image according to the second description text to obtain an auxiliary vector image, and obtain a second description code of the auxiliary vector image;

[0175] The first generating module 1205 is used to generate a set of training data corresponding to the sample vector image based on the first description code, the first description text, the second description code and the second description text; wherein the training data is used to train the text graph model.

[0176] As can be seen from the above embodiment, in this embodiment, the first description code of the sample vector image is obtained; the sample vector image is converted into a sample raster image; the first prompt word and the sample raster image are input into the image-generated text model, and the first description text corresponding to the sample vector image is output; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, image ratio; according to the second description text, the sample vector image is transformed to obtain an auxiliary vector image according to the image parameter transformation processing, and the second description code of the auxiliary vector image is obtained; based on the first description code, the first description text, the second description code and the second description text, a set of training data corresponding to the sample vector image is generated; wherein the training data is used to train the Wensheng graph model. It can be seen that in the embodiment of the present application, the above process of generating training data can generate training data conveniently without the user manually collecting training data or spending money to purchase training data, thereby avoiding the user spending more time and money costs, thereby reducing the training cost of the Wensheng graph model. .

[0177] Optionally, as an embodiment, the first description code is an extensible markup language code;

[0178] The first acquisition module 1201 may include:

[0179] A first determination submodule is used to determine the image format of the sample vector image;

[0180] A first acquisition submodule, configured to acquire the first description code from the sample vector image when the image format of the sample vector image is a scalable vector graphics format;

[0181] The second acquisition submodule is used to convert the sample vector image into an image in a scalable vector graphics format when the image format of the sample vector image is a format other than the scalable vector graphics format, and to acquire the first description code from the converted image in the scalable vector graphics format.

[0182] Optionally, as an embodiment, the first processing module 1202 may include:

[0183] A first processing submodule, configured to convert the sample vector image into a four-channel raster image in RGBA format; wherein RGBA represents a red channel, a green channel, a blue channel, and a transparent channel respectively;

[0184] The second processing submodule is used for, for each pixel position in the four-channel grating image, if the value of the transparent channel of the pixel position is zero, deleting the value of the transparent channel of the pixel position and retaining the values ​​of the red channel, green channel and blue channel of the pixel position; if the value of the transparent channel of the pixel position is not zero, deleting the value of the transparent channel of the pixel position and replacing the values ​​of the red channel, green channel and blue channel of the pixel position with the values ​​of the red channel, green channel and blue channel corresponding to the preset background pattern, respectively, to obtain a sample grating image in RGB format.

[0185] Optionally, as an embodiment, the data generating device 1200 may further include:

[0186] A building module for building an initial backbone network based on a generative pre-trained transformer;

[0187] A first training module, used for taking the first description text in the training data as the input of the initial backbone network, taking the first description code in the training data as the output target of the initial backbone network, and training the initial backbone network to obtain an intermediate model;

[0188] The second training module is used to use the second description text in the training data as the input of the intermediate model, use the second description code in the training data as the output target of the intermediate model, and train the intermediate model to obtain the text graph model.

[0189] Optionally, as an embodiment, the building module may include:

[0190] An extraction submodule, configured to extract a first word from the first description code;

[0191] An adding submodule, for adding the first vocabulary to the vocabulary of the generative pre-trained converter if the first vocabulary is not in the vocabulary of the generative pre-trained converter;

[0192] The second determination submodule is used to adjust the output dimension of the output layer of the generative pre-trained converter according to the first vocabulary, and determine the adjusted generative pre-trained converter as the initial backbone network.

[0193] Optionally, as an embodiment, the data generating device 1200 may further include:

[0194] A first receiving module, configured to receive a third description text input by a user; wherein the third description text is used to describe image parameters of the first vector image;

[0195] A fourth processing module, configured to input the second prompt word and the third description text into the text graph model for processing, and output a third description code of the first vector image;

[0196] A first display module, configured to perform image rendering based on the third description code and display the first vector image;

[0197] a second receiving module, configured to receive a fourth description text input by a user for the first vector image; wherein the fourth description text is used to instruct to perform image parameter transformation processing on the first vector image;

[0198] A second generating module is used to generate a fourth description code of a second vector image according to the fourth description text; wherein the second vector image is an image obtained by performing image parameter transformation processing on the first vector image based on the fourth description text;

[0199] The second display module is used to perform image rendering based on the fourth description code to display the second vector image.

[0200] Optionally, as an embodiment, the data generating device 1200 may further include:

[0201] A second acquisition module is used to acquire at least one candidate description code from the historical description codes output by the document graph model according to the third description text; wherein the content similarity between the third description text and the historical description text corresponding to the candidate description code is higher than a first threshold;

[0202] The third display module is used to perform image rendering based on the at least one candidate description code, and display at least one candidate vector image similar to the first vector image.

[0203] The device in the embodiment of the present application can be a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR) / virtual reality (Virtual Reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (Ultra-mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0204] The device of the embodiment of the present application may be a device having an action system. The action system may be an Android action system, an IOS action system, or other possible action systems, which are not specifically limited in the embodiment of the present application.

[0205] The device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0206] Alternatively, if Fig.13 As shown, an embodiment of the present application also provides an electronic device 1300, including a processor 1301 and a memory 1302, a program or instruction stored in the memory 1302 and executable on the processor 1301, and the program or instruction, when executed by the processor 1301, implements the various steps of the above-mentioned data generation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0207] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0208] Fig.14 is a schematic diagram of the hardware structure of an electronic device provided by some embodiments of the present application, such as Fig.14As shown, the electronic device 1400 includes but is not limited to: a radio frequency unit 1401, a network module 1402, an audio output unit 1403, an input unit 1404, a sensor 1405, a display unit 1406, a user input unit 1407, an interface unit 1408, a memory 1409, a processor 1410 and other components.

[0209] Those skilled in the art will appreciate that the electronic device 1400 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1410 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.14 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0210] In an electronic device 1400, a processor 1410 is used to obtain a first description code of a sample vector image; convert the sample vector image into a sample raster image; input a first prompt word and the sample raster image into a graph-text model, and output a first description text corresponding to the sample vector image; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image scale; according to a second description text, transform the image parameters of the sample vector image to obtain an auxiliary vector image, and obtain a second description code for the auxiliary vector image; based on the first description code, the first description text, the second description code, and the second description text, generate a set of training data corresponding to the sample vector image; wherein the training data is used to train the graph-text model.

[0211] It can be seen that in the embodiment of the present application, the above process of generating training data can conveniently generate training data without the need for users to manually collect training data or spend money to purchase training data, thereby avoiding users from spending more time and money costs, thereby reducing the training cost of the text graph model.

[0212] Optionally, as an embodiment, the first description code is an extensible markup language code; the processor 1410 is specifically used to determine the image format of the sample vector image; when the image format of the sample vector image is a scalable vector graphics format, obtain the first description code from the sample vector image; when the image format of the sample vector image is a format other than the scalable vector graphics format, convert the sample vector image into an image in a scalable vector graphics format, and obtain the first description code from the converted image in the scalable vector graphics format.

[0213] Optionally, as an embodiment, the processor 1410 is specifically used to convert the sample vector image into a four-channel raster image in RGBA format; wherein RGBA represents a red channel, a green channel, a blue channel and a transparent channel, respectively; for each pixel position in the four-channel raster image, if the value of the transparent channel of the pixel position is zero, the value of the transparent channel of the pixel position is deleted, and the values ​​of the red channel, green channel and blue channel of the pixel position are retained; if the value of the transparent channel of the pixel position is not zero, the value of the transparent channel of the pixel position is deleted, and the values ​​of the red channel, green channel and blue channel of the pixel position are replaced with the values ​​of the red channel, green channel and blue channel corresponding to the preset background pattern, respectively, to obtain a sample raster image in RGB format.

[0214] Optionally, as an embodiment, the processor 1410 is further used to construct an initial backbone network based on a generative pre-trained converter; use the first description text in the training data as the input of the initial backbone network, use the first description code in the training data as the output target of the initial backbone network, and train the initial backbone network to obtain an intermediate model; use the second description text in the training data as the input of the intermediate model, use the second description code in the training data as the output target of the intermediate model, and train the intermediate model to obtain the text graph model.

[0215] Optionally, as an embodiment, the processor 1410 is specifically used to extract a first vocabulary in the first description code; if the first vocabulary is not in the vocabulary of the generative pre-trained converter, add the first vocabulary to the vocabulary of the generative pre-trained converter; adjust the output dimension of the output layer of the generative pre-trained converter according to the first vocabulary, and determine the adjusted generative pre-trained converter as the initial backbone network.

[0216] Optionally, as an embodiment, the processor 1410 is further configured to receive a third description text input by a user; wherein the third description text is used to describe image parameters of the first vector image; input the second prompt word and the third description text into the text graph model for processing, and output a third description code of the first vector image;

[0217] A display unit 1406 is configured to perform image rendering based on the third description code to display the first vector image;

[0218] The processor 1410 is further configured to receive a fourth description text input by a user for the first vector image; wherein the fourth description text is used to instruct to perform image parameter transformation processing on the first vector image; and to generate a fourth description code of the second vector image according to the fourth description text; wherein the second vector image is an image obtained by performing image parameter transformation processing on the first vector image based on the fourth description text.

[0219] The display unit 1406 is further configured to perform image rendering based on the fourth description code to display the second vector image.

[0220] Optionally, as an embodiment, the processor 1410 is further configured to obtain at least one candidate description code from the historical description codes output by the text graph model according to the third description text; wherein the content similarity between the third description text and the historical description text corresponding to the candidate description code is higher than a first threshold;

[0221] The display unit 1406 is further configured to perform image rendering based on the at least one candidate description code, and display at least one candidate vector image similar to the first vector image.

[0222] It should be understood that in the embodiment of the present application, the input unit 1404 may include a graphics processor (Graphics Processing Unit, GPU) 14041 and a microphone 14042, and the graphics processor 14041 processes the image data of the static picture or video image obtained by the image capture device (such as a camera) in the video image capture mode or the image capture mode. The display unit 1406 may include a display panel 14061, and the display panel 14061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1407 includes a touch panel 14071 and at least one of other input devices 14072. The touch panel 14071 is also called a touch screen. The touch panel 14071 may include two parts, a touch detection device and a touch controller. Other input devices 14072 may include, but are not limited to, a physical keyboard, a function key (such as a volume control button, a switch button, etc.), a trackball, a mouse, and an action rod, which will not be repeated here. The memory 1409 can be used to store software programs and various data, including but not limited to applications and action systems. The processor 1410 may integrate an application processor and a modem processor, wherein the application processor mainly processes the action system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 1410.

[0223] The memory 1409 can be used to store software programs and various data. The memory 1409 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1409 may include a volatile memory or a non-volatile memory, or the memory 1409 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1409 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0224] The processor 1410 may include one or more processing units; optionally, the processor 1410 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1410.

[0225] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by the processor, each process of the above-mentioned data generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the processor is the processor in the electronic device in the above-mentioned embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a disk or an optical disk, etc.

[0226] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned data generation method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0227] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0228] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned data generation method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0229] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0230] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0231] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A data generation method, characterized in that: The method comprises: Obtain a first description code of the sample vector image; converting the sample vector image into a sample raster image; Inputting the first prompt word and the sample raster image into the image-to-text model, and outputting a first description text corresponding to the sample vector image; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image scale; According to the second description text, transform the image parameters of the sample vector image to obtain an auxiliary vector image, and obtain a second description code of the auxiliary vector image; Based on the first description code, the first description text, the second description code and the second description text, a set of training data corresponding to the sample vector image is generated; wherein the training data is used to train a text graph model.

2. The method according to claim 1, characterized in that The first description code is an extensible markup language code; The step of obtaining a first description code of the sample vector image includes: Determine the image format of the sample vector image; When the image format of the sample vector image is a scalable vector graphics format, obtaining the first description code from the sample vector image; In a case where the image format of the sample vector image is a format other than the scalable vector graphics format, the sample vector image is converted into an image in the scalable vector graphics format, and the first description code is obtained from the converted image in the scalable vector graphics format.

3. The method according to claim 1, characterized in that The converting the sample vector image into a sample raster image comprises: Convert the sample vector image into a four-channel raster image in RGBA format; wherein RGBA represents a red channel, a green channel, a blue channel and a transparent channel respectively; For each pixel position in the four-channel raster image, if the value of the transparent channel of the pixel position is zero, the value of the transparent channel of the pixel position is deleted, and the values ​​of the red channel, green channel, and blue channel of the pixel position are retained; if the value of the transparent channel of the pixel position is not zero, the value of the transparent channel of the pixel position is deleted, and the values ​​of the red channel, green channel, and blue channel of the pixel position are replaced with the values ​​of the red channel, green channel, and blue channel corresponding to the preset background pattern, respectively, to obtain a sample raster image in RGB format.

4. The method according to claim 1, characterized in that After the step of generating a set of training data corresponding to the sample vector image based on the first description code, the first description text, the second description code and the second description text, the method further includes: Build the initial backbone network based on the generative pre-trained transformer; Using the first description text in the training data as the input of the initial backbone network, using the first description code in the training data as the output target of the initial backbone network, and training the initial backbone network to obtain an intermediate model; The second description text in the training data is used as the input of the intermediate model, the second description code in the training data is used as the output target of the intermediate model, and the intermediate model is trained to obtain the text graph model.

5. The method according to claim 4, characterized in that The initial backbone network is constructed based on the generative pre-trained converter, including: extracting a first word from the first description code; If the first vocabulary is not in the vocabulary of the generative pre-trained transformer, adding the first vocabulary to the vocabulary of the generative pre-trained transformer; The output dimension of the output layer of the generative pre-trained converter is adjusted according to the first vocabulary, and the adjusted generative pre-trained converter is determined as an initial backbone network.

6. The method according to claim 4, characterized in that After the step of using the second description text in the training data as the input of the intermediate model, using the second description code in the training data as the output target of the intermediate model, and training the intermediate model to obtain a text graph model, the method further includes: receiving a third description text input by a user; wherein the third description text is used to describe image parameters of the first vector image; Inputting the second prompt word and the third description text into the text graph model for processing, and outputting a third description code of the first vector image; Perform image rendering based on the third description code to display the first vector image; receiving a fourth description text input by a user for the first vector image; wherein the fourth description text is used to instruct to perform image parameter transformation processing on the first vector image; Generate a fourth description code of a second vector image according to the fourth description text; wherein the second vector image is an image obtained by performing image parameter transformation processing on the first vector image based on the fourth description text; Image rendering is performed based on the fourth description code to display the second vector image.

7. The method according to claim 6, characterized in that After the step of performing image rendering based on the third description code and displaying the first vector image, the method further includes: According to the third description text, at least one candidate description code is obtained from the historical description codes output by the text graph model; wherein the content similarity between the third description text and the historical description text corresponding to the candidate description code is higher than a first threshold; Image rendering is performed based on the at least one candidate description code to display at least one candidate vector image similar to the first vector image.

8. A data generating device, characterized in that: The device comprises: A first acquisition module, used for acquiring a first description code of a sample vector image; A first processing module, configured to convert the sample vector image into a sample raster image; A second processing module is used to input the first prompt word and the sample raster image into the image-to-text model, and output a first description text corresponding to the sample vector image; wherein the first description text is used to describe at least one of the following image parameters of the sample vector image: image content, image size, image rotation angle, and image scale; A third processing module is used to transform the image parameters of the sample vector image according to the second description text to obtain an auxiliary vector image, and obtain a second description code of the auxiliary vector image; The first generating module is used to generate a set of training data corresponding to the sample vector image based on the first description code, the first description text, the second description code and the second description text; wherein the training data is used to train the text graph model.

9. The device according to claim 8, characterized in that The first description code is an extensible markup language code; The first acquisition module includes: A first determination submodule is used to determine the image format of the sample vector image; A first acquisition submodule, configured to acquire the first description code from the sample vector image when the image format of the sample vector image is a scalable vector graphics format; The second acquisition submodule is used to convert the sample vector image into an image in a scalable vector graphics format when the image format of the sample vector image is a format other than the scalable vector graphics format, and to acquire the first description code from the converted image in the scalable vector graphics format.

10. The device according to claim 8, characterized in that The first processing module comprises: A first processing submodule, configured to convert the sample vector image into a four-channel raster image in RGBA format; wherein RGBA represents a red channel, a green channel, a blue channel, and a transparent channel respectively; The second processing submodule is used for, for each pixel position in the four-channel grating image, if the value of the transparent channel of the pixel position is zero, deleting the value of the transparent channel of the pixel position and retaining the values ​​of the red channel, green channel and blue channel of the pixel position; if the value of the transparent channel of the pixel position is not zero, deleting the value of the transparent channel of the pixel position and replacing the values ​​of the red channel, green channel and blue channel of the pixel position with the values ​​of the red channel, green channel and blue channel corresponding to the preset background pattern, respectively, to obtain a sample grating image in RGB format.

11. The device according to claim 8, characterized in that The device also includes: A building module for building an initial backbone network based on a generative pre-trained transformer; A first training module, used for taking the first description text in the training data as the input of the initial backbone network, taking the first description code in the training data as the output target of the initial backbone network, and training the initial backbone network to obtain an intermediate model; The second training module is used to use the second description text in the training data as the input of the intermediate model, use the second description code in the training data as the output target of the intermediate model, and train the intermediate model to obtain the text graph model.

12. The device according to claim 11, characterized in that The building blocks include: An extraction submodule, configured to extract a first word from the first description code; An adding submodule, for adding the first vocabulary to the vocabulary of the generative pre-trained converter if the first vocabulary is not in the vocabulary of the generative pre-trained converter; The second determination submodule is used to adjust the output dimension of the output layer of the generative pre-trained converter according to the first vocabulary, and determine the adjusted generative pre-trained converter as the initial backbone network.

13. The device according to claim 11, characterized in that The device also includes: A first receiving module, configured to receive a third description text input by a user; wherein the third description text is used to describe image parameters of the first vector image; A fourth processing module, configured to input the second prompt word and the third description text into the text graph model for processing, and output a third description code of the first vector image; A first display module, configured to perform image rendering based on the third description code and display the first vector image; a second receiving module, configured to receive a fourth description text input by a user for the first vector image; wherein the fourth description text is used to instruct to perform image parameter transformation processing on the first vector image; A second generating module is used to generate a fourth description code of a second vector image according to the fourth description text; wherein the second vector image is an image obtained by performing image parameter transformation processing on the first vector image based on the fourth description text; The second display module is used to perform image rendering based on the fourth description code to display the second vector image.

14. The device according to claim 13, characterized in that The device also includes: A second acquisition module is used to acquire at least one candidate description code from the historical description codes output by the document graph model according to the third description text; wherein the content similarity between the third description text and the historical description text corresponding to the candidate description code is higher than a first threshold; The third display module is used to perform image rendering based on the at least one candidate description code, and display at least one candidate vector image similar to the first vector image.

15. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the data generation method according to any one of claims 1 to 7 are implemented.

16. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the data generation method according to any one of claims 1 to 7 are implemented.

Images