Image generation method and device, electronic equipment, storage medium and program product
By first generating the page text content, then generating the page code file based on the text rendering parameters, and finally generating the target image, the problem of poor text rendering effect in the generation of text-containing images from large models is solved, and the clarity and controllability of text content and the rationality of layout are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING BAIDU NETCOM SCI & TECH CO LTD
- Filing Date
- 2026-03-23
- Publication Date
- 2026-06-23
Smart Images
Figure CN122265458A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to large model technology or computer vision. Specifically, this disclosure relates to an image generation method, apparatus, electronic device, storage medium, and program product. Background Technology
[0002] With the development of large model technology, large models are increasingly being used to generate images in various scenarios. In some scenarios, it is necessary to generate images containing text, such as product posters, magazine covers, and travel brochures.
[0003] Currently, text-containing images generated using large models may suffer from poor text rendering. Summary of the Invention
[0004] This disclosure provides an image generation method, apparatus, electronic device, storage medium, and program product to improve the presentation of text content in a target image.
[0005] According to one aspect of this disclosure, an image generation method is provided, comprising: Generate page text content based on image description information; Based on the page text content and a preset set of text rendering parameters, a page code file is generated. The page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters. The target image corresponding to the image description information is generated based on the page code file.
[0006] According to another aspect of this disclosure, an image generation apparatus is provided, comprising: The content generation unit is configured to generate page text content based on image description information; The code generation unit is configured to generate a page code file based on the page text content and a preset set of text rendering parameters. The page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters. The image generation unit is configured to generate a target image corresponding to the image description information based on the page code file.
[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in the embodiments of this disclosure.
[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the methods described in embodiments of this disclosure.
[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described in the embodiments of this disclosure.
[0010] This disclosure first generates page text content based on image description information, then generates a page code file based on the page text content and a preset set of text rendering parameters, and finally obtains the target image corresponding to the image description information based on the page code file. This method decouples the target image generation process; that is, it first generates page text content, then generates the page code file, and finally obtains the target image. This decoupling facilitates independent control of the page text content and text rendering parameters, improving the controllability and accuracy of the target image, and thus enhancing the presentation effect of the text content in the target image.
[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure.
[0013] Figure 1 This is the system architecture diagram to which this disclosure applies.
[0014] Figure 2 This is a flowchart of the image generation method provided in this disclosure.
[0015] Figure 3 This is a schematic diagram illustrating the steps involved in generating page text content as provided in this publication.
[0016] Figure 4 This is a schematic diagram illustrating the steps involved in generating a page code file, as provided in this publication.
[0017] Figure 5 This is a schematic diagram illustrating another step in generating a page code file provided in this disclosure.
[0018] Figure 6This is a schematic diagram illustrating another step in generating a page code file provided in this publication.
[0019] Figure 7 This is a schematic block diagram of the image generation apparatus provided in this disclosure.
[0020] Figure 8 This is a block diagram of an electronic device used to implement the image generation method of the embodiments of this disclosure. Detailed Implementation
[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.
[0022] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.
[0023] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0024] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."
[0025] With the rapid development of large-scale modeling technology, its application in image generation is becoming increasingly widespread. More and more scenarios are relying on large-scale models to directly synthesize target images based on text descriptions. In some applications, it is necessary to generate images containing text, i.e., text-containing images. These images are crucial in product promotion, information dissemination, and interface design, and typical examples include, but are not limited to, product posters, magazine covers, travel brochures, and infographics.
[0026] However, current methods for generating text-containing images using large models often fail to produce satisfactory text rendering. Specifically, large models typically treat text as a visual texture or pattern within the text-containing image and perform pixel-level synthesis, essentially "drawing" shapes resembling characters. This frequently results in issues such as missing characters, redundant character insertion, spelling errors (especially for long words, technical terms, or non-Latin characters), reversed character order, and even the generation of meaningless pseudo-character sequences. Furthermore, large models lack the ability to treat text as an independent entity for precise layout, easily generating text-containing images with chaotic text layouts. For example, the positional relationship between titles and body text in the text-containing image may be disordered, or text may obscure key content within the image.
[0027] In addition, the style attributes of text, such as font, font size, font weight, color, line spacing, and character spacing, are difficult to control precisely and stably through natural language prompts, resulting in poor text presentation. For example, a product poster with a strong sense of technology may contain text with a handwritten style.
[0028] In view of this, this disclosure provides a new approach. To facilitate understanding of this disclosure, the system architecture on which this disclosure is based will first be described. Figure 1 Exemplary system architectures that can be applied to embodiments of this disclosure are shown, such as Figure 1 As shown, the system architecture may include: a client and a server.
[0029] The server side and the client side are the two main components of an application service. The server side uses a server as its primary hardware infrastructure and may include one or more software service modules. The server side and the client side form a collaborative front-end and back-end.
[0030] The client can be set on the terminal device. In this embodiment of the disclosure, the client can be a local application, a mini-program, or a web application running through a browser on the terminal device.
[0031] Terminal devices can include, but are not limited to, smart mobile terminals, wearable devices, PCs (Personal Computers), and smart home devices. Smart mobile devices can include devices such as mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), and connected car terminals. Wearable devices can include devices such as smartwatches, smart glasses, smart bracelets, VR (Virtual Reality) devices, AR (Augmented Reality) devices, and mixed reality devices (devices that support both virtual and augmented reality). Smart home devices can include devices such as smart TVs and smart refrigerators with displays.
[0032] A server can be a single server, a server cluster consisting of multiple servers, or a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a hosting product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability.
[0033] It should be understood that Figure 1 The number of client and server components shown is merely illustrative. Depending on implementation needs, there can be any number of client and server components.
[0034] As one embodiment, after a user inputs image description information on the user-side device, the user-triggered generation function sends a generation request to the server. This generation request includes the image description information. Upon receiving the generation request, the server parses the image description information from it, generates page text content based on the image description information, generates a page code file based on the page text content and a preset set of text rendering parameters, generates the target image corresponding to the image description information based on the page code file, and returns the target image to the user-side device for display.
[0035] Figure 2 This is a flowchart of an image generation method provided in an embodiment of the present disclosure. The method can be generated by… Figure 1 The server-side execution in the system shown. For example... Figure 2 As shown, the method may include the following steps: Step 201: Generate page text content based on image description information.
[0036] Step 202: Based on the page text content and a preset set of text rendering parameters, generate a page code file. The page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters.
[0037] Step 203: Generate the target image corresponding to the image description information based on the page code file.
[0038] As can be seen from the above process, this disclosure can first generate page text content based on image description information, then generate a page code file based on the page text content and a preset set of text rendering parameters, and finally obtain the target image corresponding to the image description information based on the page code file. This method decouples the process of generating the target image; that is, it first generates the page text content, then generates the page code file, and finally obtains the target image. This decoupling facilitates independent control of the page text content and text rendering parameters, improves the controllability and accuracy of the target image, and thus enhances the presentation effect of the text content in the target image.
[0039] In addition, this solution generates the page text content first and then obtains the page code file. Therefore, the text content in the target image will not have issues such as missing characters, extra characters, or spelling errors. Compared to using a large model to directly synthesize the text content in the target image at the pixel level, the text content generated by this solution is more readable.
[0040] The following describes in detail each step of the above process and the effects that can be further produced, with reference to the embodiments.
[0041] First, the above step 201, namely "generating page text content based on image description information", will be described in detail with reference to the embodiments.
[0042] This disclosure first allows for the acquisition of image description information, which can be directly input by the user through a user terminal or extracted from documents uploaded by the user.
[0043] Image description information can refer to natural language descriptive text, such as "generate a banner ad about a certain product promotion", or it can refer to a set of keywords, such as {a certain product promotion, banner ad}.
[0044] Then, based on the image description information, the page text content is generated. The page text content can refer to the text elements expected to be displayed in the generated target image and associated with the image description information, such as text fragments like titles, paragraphs, list items, button labels, and product descriptions written in natural language.
[0045] For example, when the image description is "Generate a news report", the page text content can include the news headline (such as "Price of a certain product increased") and a summary paragraph (such as "Price of a certain product increased by X yuan on XX year XX month XX day"); when the image description is "Generate a product poster", the page text content can include the product's brand (such as "XX brand") and a brief introduction (such as "Using the XXXX production method, the effect is better").
[0046] As can be seen, the process of generating page text content is essentially to instantiate the abstract semantic description (i.e., image description information) into a specific string sequence (i.e., page text content).
[0047] As one feasible implementation, at least one keyword can be determined based on image description information, and at least one page text content can be matched from a preset page text library based on the at least one keyword.
[0048] As another possible implementation, such as Figure 3 As shown, a third prompt instruction can be generated based on image description information. The third prompt instruction is used to instruct the content generation model to generate page text content based on the image description information. Then, the third prompt instruction is input into the content generation model to obtain the page text content output by the content generation model.
[0049] In the embodiments of this disclosure, semantic tags corresponding to image description information can be determined. These semantic tags are used to characterize the category to which the main object displayed in the target image belongs. For example, if the target image is a car poster, then the semantic tag can refer to the car category to which the car displayed in the car poster belongs, such as sedan, commercial vehicle, SUV, etc.
[0050] Then, based on semantic tags and image description information, page text content can be generated. At this time, the page text content is associated with the category of the main object displayed in the target image. Continuing with the previous example, if the semantic tag is "off-road vehicle", the page text content generated based on the semantic tag and image description information can be "good performance and high horsepower".
[0051] In addition, the page structure type corresponding to the image description information can be determined, such as horizontal poster, vertical poster, product promotional image, application splash screen ad, infographic, etc. The page text content generated based on different page structure types are different. For example, when generating page text content based on page structure type and image description information, if the page structure type is a horizontal poster, the generated page text content can be more detailed, while if the page structure type is an application splash screen ad, the generated page text content can be more concise.
[0052] Of course, the page structure type and semantic tags corresponding to the image description information can be determined at the same time, and the page text content can be generated based on the semantic tags and page structure type, as well as the image description information.
[0053] This approach can determine the semantic tags and / or page structure type based on image description information, and generate page text content accordingly. This ensures that the generated page text content closely follows a specific semantic category or page format from the beginning, improving the accuracy of the page text content and laying a solid foundation for the subsequent generation of target images that conform to the image description information.
[0054] The following describes in detail step 202, namely, "based on the page text content and based on a preset set of text rendering parameters, a page code file is generated, the page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters," with reference to the embodiments.
[0055] First, the preset text rendering parameter set can refer to the set of parameters used to control the visual presentation effect of the text content on the page. For example, the text rendering parameter set may include, but is not limited to, the following text rendering parameters: font parameters, font size parameters, font weight parameters, color parameters, alignment parameters, line spacing parameters, paragraph spacing parameters, layout parameters, background style parameters, shadow effect parameters, blur level parameters, noise intensity parameters, etc. These text rendering parameters can correspond to a predefined discrete list. For example, the font parameter can correspond to the discrete list {"Arial", "Times New Roman", "SimSun", "Microsoft YaHei"}, the font size parameter can correspond to the numerical range [12px, 36px, 72px], the layout parameter can correspond to the discrete list {"top layer", "bottom layer", "bottom right corner", "top left corner"}, the alignment parameter can correspond to the discrete list {"left alignment", "right alignment", "center", "justified"}, and so on.
[0056] The process of generating a page code file based on the page text content can be understood as selecting at least one text rendering parameter from a preset set of text rendering parameters for this generation (this could be a random selection, a selection that matches the page text content, or a sampling of at least one text rendering parameter based on a preset data distribution within the set of text rendering parameters), and encoding these text rendering parameters along with the page text content into the page code file. For example, the generated page code file might specify the font parameter of a certain page text content as "SimSun", the font size parameter as "24px", and the color parameter as "#FF0000".
[0057] There are several ways to achieve the above steps of generating page code files. One possible method is to select target text rendering parameters that match the page text content from a preset set of text rendering parameters, and then fill the page text content and its corresponding target text rendering parameters into the corresponding positions of a preset template (such as an HTML template, CSS template, etc.) to obtain the page code file.
[0058] As another possible approach, such as Figure 4 As shown, a first prompt instruction can be generated based on the page text content and the set of text rendering parameters. The first prompt instruction is used to instruct the code generation model to select target text rendering parameters that match the page text content from the set of text rendering parameters, and generate a page code file based on the target text rendering parameters and the page text content. Then, the first prompt instruction is input into the code generation model to obtain the page code file output by the code generation model.
[0059] In other words, the initial prompt input into the code generation model includes the page text content (such as "Science and Technology Development Plan") and a set of text rendering parameters (such as "optional font parameters: SimSun, KaiTi; optional color parameters: #FF0000, #00FF00, #0000FF"). Based on its understanding of the page text content, the code generation model selects target text rendering parameters from the set of text rendering parameters that match the page text content (such as selecting more technologically advanced font and color parameters for "Science and Technology Development Plan"), and generates the page code file based on the page text content and the target text rendering parameters.
[0060] For example, the first prompt could be: "Please generate an HTML file that displays the following text content: '[Page Text Content]'. Please select appropriate parameters from the following set of text rendering parameters to enhance it: Optional font parameters: [Font List], Optional color parameters: [Color List]. Please ensure the code is concise and renders correctly in modern browsers." By adopting the above method, a code generation model can be introduced to automatically generate page code files based on the first prompt instruction. This allows the target text rendering parameters to be determined based on the semantic understanding ability of the code generation model, rather than relying on fixed or random selections. This significantly improves the adaptability and rationality of the page text content in the page code file in terms of font, color, size, layout, etc., and effectively enhances the quality and efficiency of the generation process.
[0061] As another feasible approach, such as Figure 5As shown, based on the semantic tags corresponding to the image description information and / or the corresponding page structure type, and based on the page text content and the set of text rendering parameters, a first prompt instruction is generated. The first prompt instruction is used to instruct the code generation model to select target text rendering parameters that match the page text content from the set of text rendering parameters based on the semantic tags and / or page structure type, and to generate a page code file based on the target text rendering parameters and the page text content. Then, the first prompt instruction is input into the code generation model to obtain the page code file output by the code generation model.
[0062] In other words, in this embodiment, the target text rendering parameters that accurately match the page text content can be further determined based on semantic tags and / or page structure type. For example, if the semantic tag is "off-road vehicle," the font parameters matched for the page text content can correspond to fonts that can reflect effects such as off-road and freedom; as another example, if the page structure type is "banner poster," the color parameters matched for the page text content can correspond to colors with higher saturation; and as yet another example, if the page structure type is "infographic," the font parameters matched for the page text content can correspond to more formal fonts such as Songti and Heiti.
[0063] It can be seen that by further generating the first prompt instruction based on the semantic tags and / or page structure type corresponding to the image description information, the first prompt instruction can guide the code generation model to select matching target text rendering parameters based on higher-level semantic or structural information, so that the final generated target image is more consistent with the image description information, thereby improving the accuracy of the target image.
[0064] To generate more accurate page code files, as a specific implementation, the first prompt instruction can also be used to instruct the code generation model to generate, while generating the page code file, the component structure of the page corresponding to the page (such as whether the components containing the page text content are compact or distributed), and the position of the page text content on the page corresponding to the page code file. This approach can constrain the code generation model to think according to the component structure and the position of the page text content, improving the quality of the final generated page. It can also serve as an explicit expression of the internal reasoning steps or logic of the code generation model, providing a basis for fault diagnosis and recovery when the quality of the reasoning result (i.e., the page code file) is poor.
[0065] In addition, the first prompt can also be used to instruct the code generation model on the reasoning behind the generation of page code files. For example, the reasoning can include a brief explanation from the code generation model on why a certain font parameter was chosen or why a certain layout parameter was used, such as, "I chose font A because it appears stable and is suitable for formal text; I chose dark blue because the page text content is related to technology, and blue often represents a sense of technology." By requiring the code generation model to output its reasoning rationale along with the output page code file, the decision-making process of parameter selection and code generation becomes transparent and traceable. This helps to analyze the behavior of the code generation model, lays the foundation for subsequent optimization of the first prompt instruction, and enhances the interpretability and controllability of the entire automated generation process, making it easier to discover and correct potential problems in the generation process.
[0066] Furthermore, the content generated in step 201 based on the image description information can include not only page text content but also page image content. Page image content can refer to image elements (such as JPG or PNG images) that are expected to be displayed in the generated target image and are associated with the image description information. For example, if the image description information is "Generate an introductory poster for XX company," then the page image content could be a logo image of XX company; if the image description information is "Generate a promotional poster for XX scenic area," then the page image content could be a landscape image of XX scenic area.
[0067] Furthermore, in step 202, when generating the page code file, the page code file can be generated simultaneously based on the page image content, the page text content, and a preset set of text rendering parameters. The page corresponding to the final generated page code file contains page text content and page image content, and the text rendering parameters in the page code file belong to the set of text rendering parameters. These text rendering parameters match the page image content. For example, the color parameters of the page text content need to have sufficient contrast with the hue of the page image content to ensure readability.
[0068] In this way, the page corresponding to the page code file will contain both page text content and page image content, further realizing precise control over the target image. Moreover, the selection of text rendering parameters also needs to consider the matching degree with the page image content, thus improving the display effect of the target image.
[0069] As a specific embodiment, such as Figure 6 As shown, a second prompt instruction is generated based on the page text content, page image content, and a set of text rendering parameters. This second prompt instruction instructs the code generation model to select target text rendering parameters from the set of text rendering parameters that match both the page text content and the page image content. Based on the target text rendering parameters, the page text content, and the page image content, a page code file is generated. Then, the second prompt instruction is input into the code generation model to obtain the page code file output by the code generation model.
[0070] It can be seen that by constructing a second prompt instruction to guide the code generation model to select target text rendering parameters that match both the page text content and the page image content, the presentation effect of the text content in the final generated target image can be coordinated with the visual effect of the image content, thereby improving the rationality and presentation effect of the target image.
[0071] It should be noted that page image content can refer to a specific image, an address link to a specific image, or graphic information drawn using code (such as SVG).
[0072] It should also be noted that, in addition to the preset set of text rendering parameters, this disclosure also includes a preset set of image rendering parameters. Similar to the text rendering parameter set, the preset set of image rendering parameters can refer to a set of parameters used to control the visual presentation effect of page image content. For example, the set of image rendering parameters may include, but is not limited to, the following image rendering parameters: image size parameters, position parameters, border parameters, filter effect parameters, blur degree parameters, noise intensity parameters, etc.
[0073] Specifically, based on the page image content and page text content, and based on a preset set of text rendering parameters and a preset set of image rendering parameters, a page code file is generated. The image rendering parameters in the page code file belong to the set of image rendering parameters.
[0074] In other words, the system selects target text rendering parameters that match the page text content from a preset set of text rendering parameters, and selects target image rendering parameters that match the page image content from a preset set of image rendering parameters. These target text rendering parameters are then encoded together with the page text content, and the target image rendering parameters are encoded together with the page image content into the page code file. For example, the generated page code file might specify the font parameter of a certain page text content as "SimSun", the font size parameter as "24px", the color parameter as "#FF0000", and the image size parameter of a certain page image content as "200×200", with the filter effect parameter set to "high saturation filter".
[0075] By further introducing a set of preset image rendering parameters, systematic and parameterized fine control over the image content on the page is achieved, improving the controllability of the generated target image and thus enhancing the display effect of the image content in the target image.
[0076] The following describes step 203, namely "generating the target image corresponding to the image description information based on the page code file", in detail with reference to the embodiments.
[0077] Step 203 is the final output stage of the image generation method, aiming to convert the page code file obtained in step 202 into the final output target image. Specifically, a headless browser (i.e., Headless Chrome) can be invoked to execute the page code file. The headless browser can parse the page code file (such as HTML, CSS, and JavaScript code files) and then render a complete target page. Then, the target page is captured as a bitmap image (such as PNG or JPEG format) using the screenshot function, and this bitmap image is the target image. Since the page code file is obtained based on the page text content and target text rendering parameters, this scheme can precisely control the visual representation of the page text content on the target page based on the text rendering parameters during rendering, thereby improving the presentation effect of the page text content on the target page. Furthermore, the text content in the final generated target image has extremely high clarity and readability.
[0078] It should be noted that when generating the target image corresponding to the image description information based on the page code file, the text rendering parameters in the page code file can also be validated first. Specifically, the validation process may include, but is not limited to, the following steps: determining the contrast between the color parameter in the text rendering parameters and the background color parameter corresponding to the page code file; if the contrast does not meet the contrast requirements, correcting the color parameter to make the contrast meet the requirements; determining whether the font size parameter in the text rendering parameters meets the font size requirements (e.g., ensuring that the font size parameter is greater than or equal to 12px); if not, correcting the font size parameter to make the font size parameter meet the requirements; and determining whether the font corresponding to the font parameter in the text rendering parameters is in the preset font library; if not, replacing the font corresponding to the font parameter with the specified font.
[0079] In addition to the aforementioned validation of text rendering parameters, it can also check whether the code in the page code file conforms to the syntax rules, or check whether the page text content exceeds the page area, and so on.
[0080] By performing these verification and repair operations, the poor page rendering caused by problems in the page code file itself can be effectively reduced, such as unclear contrast between background and text, fonts that are too small or too large, etc., which greatly improves the usability and presentation of the generated target image.
[0081] In addition to generating product posters, magazine covers, travel brochures, and infographics, the aforementioned method for generating target images can also be used to generate training samples. As an feasible approach, target images can be used to generate text recognition training samples for text recognition models. Specifically, based on the page text content, real text annotations are generated, and the target image and real text annotations are combined into a text recognition training sample. When training the text recognition model based on the text recognition training sample, the target image is input into the text recognition model to obtain the predicted text output by the text recognition model. The text recognition model is trained with the goal of minimizing the difference between the predicted text and the real text annotations.
[0082] As another feasible approach, the target image can be used to generate classification training samples for a classification model. Specifically, based on the target image, a large language model is used to predict the category corresponding to the target image to obtain a verification category. If the verification category is consistent with the semantic label corresponding to the image description information, the semantic label or verification category is used as the ground truth label. The target image and the ground truth label are combined into a classification training sample. When training the classification model based on the classification training sample, the target image is input into the classification model to obtain the predicted classification output by the classification model. The classification model is trained with the goal of minimizing the difference between the predicted classification and the ground truth classification label.
[0083] This approach improves the quality of classification training samples by using a large language model to predict the category of the generated target image and verifying its consistency with the semantic labels used during generation. This effectively ensures the accuracy and reliability of the target images used for training and improves training efficiency.
[0084] Furthermore, after training for a period of time, erroneous cases of the classification model or text recognition model on the training samples can be obtained. Based on the analysis results of the erroneous cases, new training samples are generated to enhance the training of the classification model or text recognition model. For example, erroneous cases of the text recognition model on the text recognition training samples are obtained. After analyzing the erroneous cases, it is found that when the font parameters of the target image in the text recognition training samples are SimSun or Microsoft YaHei, the difference between the output predicted text and the real text annotation is large. Therefore, the discrete list of font parameters in the text rendering parameter set {"Arial", "Times New Roman", "SimSun", "Microsoft YaHei"} can be adjusted to {"SimSun", "Microsoft YaHei"} so that the font corresponding to the text content in the newly generated target image is SimSun or Microsoft YaHei. New text recognition training samples are generated based on the newly generated target image to enhance the training of the text recognition model.
[0085] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0086] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0087] According to another embodiment, an image generation apparatus is provided. Figure 7 A schematic block diagram of an image generating apparatus according to one embodiment is shown, the image generating apparatus being disposed in Figure 1 The server side in the illustrated architecture. For example... Figure 7 As shown, the image generation device 700 includes a content generation unit 701, a code generation unit 702, and an image generation unit 703, and further includes a sample generation unit 704 and a parameter verification unit 705. The main functions of each component are as follows: The content generation unit 701 is configured to generate page text content based on image description information.
[0088] The code generation unit 702 is configured to generate a page code file based on the page text content and a preset set of text rendering parameters. The page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters.
[0089] The image generation unit 703 is configured to generate a target image corresponding to the image description information based on the page code file.
[0090] As one possible implementation method, when generating a page code file based on page text content and a preset set of text rendering parameters, the code generation unit 702 can be specifically configured to: generate a first prompt instruction based on the page text content and the set of text rendering parameters; the first prompt instruction is used to instruct the code generation model to select target text rendering parameters that match the page text content from the set of text rendering parameters, and generate a page code file based on the target text rendering parameters and the page text content; input the first prompt instruction into the code generation model to obtain the page code file output by the code generation model.
[0091] As one possible implementation method, when generating the first prompt instruction based on the page text content and the text rendering parameter set, the code generation unit 702 can be specifically configured to: generate the first prompt instruction based on the semantic tags corresponding to the image description information and / or the corresponding page structure type, and based on the page text content and the text rendering parameter set. The first prompt instruction is used to instruct the code generation model to select target text rendering parameters that match the page text content from the text rendering parameter set based on the semantic tags and / or the page structure type, and generate a page code file based on the target text rendering parameters and the page text content.
[0092] Furthermore, the sample generation unit 704 can be specifically configured to: input the target image into the large language model so that the large language model outputs the verification label of the target image; in response to the consistency between the verification label and the semantic label corresponding to the image description information, determine the classification training sample based on the target image and the verification label, and use the classification training sample to train the preset classification model.
[0093] As one possible approach, the first prompt instruction is also used to indicate the component structure of the page corresponding to the page code file output by the code generation model, and the position of the page text content in the page corresponding to the page code file.
[0094] As one possible implementation method, when generating page text content based on image description information, the content generation unit 701 can be specifically configured to: determine the semantic tags and / or the corresponding page structure type corresponding to the image description information; and generate page text content based on the semantic tags and / or the page structure type, and based on the image description information.
[0095] As one possible implementation method, when generating page text content based on image description information, the content generation unit 701 can be specifically configured to generate page text content and page image content based on image description information.
[0096] The code generation unit 702, when generating a page code file based on page text content and a preset set of text rendering parameters, can be specifically configured to: generate a page code file based on page image content and page text content, and based on a preset set of text rendering parameters. The page corresponding to the page code file contains page text content and page image content, and the text rendering parameters in the page code file belong to the set of text rendering parameters.
[0097] As one possible implementation method, when generating a page code file based on page text content and a preset set of text rendering parameters, the code generation unit 702 can be specifically configured to: generate a second prompt instruction based on the page text content, page image content, and the set of text rendering parameters; the second prompt instruction is used to instruct the code generation model to select target text rendering parameters from the set of text rendering parameters that match both the page text content and the page image content, and generate a page code file based on the target text rendering parameters, the page text content, and the page image content; input the second prompt instruction into the code generation model to obtain the page code file output by the code generation model.
[0098] As one possible implementation method, when the code generation unit 702 generates a page code file based on the page image content and page text content, and based on a preset set of text rendering parameters, it can be specifically configured to generate a page code file based on the page image content and page text content, and based on a preset set of text rendering parameters and a preset set of image rendering parameters, wherein the image rendering parameters in the page code file belong to the set of image rendering parameters.
[0099] As one possible implementation method, the parameter verification unit 705, when verifying the text rendering parameters in the page code file, can be specifically configured to at least one of the following: determine the contrast between the color parameter in the text rendering parameters and the background color parameter corresponding to the page code file; if the contrast does not meet the contrast requirements, correct the color parameter so that the contrast meets the contrast requirements; determine whether the font size parameter in the text rendering parameters meets the font size requirements; if not, correct the font size parameter so that the font size parameter meets the font size requirements; determine whether the font corresponding to the font parameter in the text rendering parameters is in a preset font library; if not, replace the font corresponding to the font parameter with a specified font.
[0100] As one possible implementation method, when generating a target image corresponding to image description information based on a page code file, the image generation unit 703 can be specifically configured to: render a target page based on the page code file; and determine the target image corresponding to the image description information based on the target page.
[0101] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0102] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0103] like Figure 8 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.
[0104] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0105] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as image generation methods. For example, in some embodiments, the image generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the image generation method by any other suitable means (e.g., by means of firmware).
[0106] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0107] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0108] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0109] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0110] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0111] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0112] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.
[0113] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.
Claims
1. An image generation method, comprising: Generate page text content based on image description information; Based on the page text content and a preset set of text rendering parameters, a page code file is generated. The page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters. The target image corresponding to the image description information is generated based on the page code file.
2. The method according to claim 1, wherein, The step of generating a page code file based on the page text content and a preset set of text rendering parameters includes: A first prompt instruction is generated based on the page text content and the text rendering parameter set. The first prompt instruction is used to instruct the code generation model to select a target text rendering parameter that matches the page text content from the text rendering parameter set, and to generate the page code file based on the target text rendering parameter and the page text content. The first prompt instruction is input into the code generation model to obtain the page code file output by the code generation model.
3. The method according to claim 2, wherein, The generation of the first prompt instruction based on the page text content and the text rendering parameter set includes: Based on the semantic tags corresponding to the image description information and / or the corresponding page structure type, and based on the page text content and the text rendering parameter set, a first prompt instruction is generated. The first prompt instruction is used to instruct the code generation model to select target text rendering parameters that match the page text content from the text rendering parameter set based on the semantic tags and / or the page structure type, and to generate the page code file based on the target text rendering parameters and the page text content.
4. The method according to claim 3, further comprising: The target image is input into a large language model so that the large language model outputs the verification label of the target image; In response to the consistency between the verification label and the semantic label corresponding to the image description information, a classification training sample is determined based on the target image and the verification label, and the classification training sample is used to train a preset classification model.
5. The method according to claim 2 or 3, wherein, The first prompt instruction is also used to prompt the code generation model to output the component structure of the page corresponding to the page code file, and the position of the page text content in the page corresponding to the page code file.
6. The method according to any one of claims 1-5, wherein, The process of generating page text content based on image description information includes: Determine the semantic tags and / or page structure types corresponding to the image description information; The page text content is generated based on the semantic tags and / or the page structure type, and based on the image description information.
7. The method according to claim 1, wherein, The process of generating page text content based on image description information includes: Based on the image description information, generate page text content and page image content; The step of generating a page code file based on the page text content and a preset set of text rendering parameters includes: Based on the page image content and the page text content, and based on a preset set of text rendering parameters, a page code file is generated. The page corresponding to the page code file contains the page text content and the page image content, and the text rendering parameters in the page code file belong to the set of text rendering parameters.
8. The method according to claim 7, wherein, The step of generating a page code file based on the page text content and a preset set of text rendering parameters includes: A second prompt instruction is generated based on the page text content, page image content, and the set of text rendering parameters. The second prompt instruction is used to instruct the code generation model to select target text rendering parameters from the set of text rendering parameters that match both the page text content and the page image content, and to generate the page code file based on the target text rendering parameters, the page text content, and the page image content. The second prompt instruction is input into the code generation model to obtain the page code file output by the code generation model.
9. The method according to claim 7 or 8, wherein, The step of generating a page code file based on the page image content and the page text content, and based on a preset set of text rendering parameters, includes: Based on the page image content and the page text content, and based on a preset set of text rendering parameters and a preset set of image rendering parameters, a page code file is generated, wherein the image rendering parameters in the page code file belong to the set of image rendering parameters.
10. The method according to any one of claims 1-9, further comprising, after generating the page code file: Perform at least one of the following checks on the text rendering parameters in the page code file: Determine the contrast between the color parameter in the text rendering parameters and the background color parameter corresponding to the page code file. If the contrast does not meet the contrast requirements, correct the color parameter so that the contrast meets the contrast requirements. Determine whether the font size parameter in the text rendering parameters meets the font size requirements. If not, correct the font size parameter to make it meet the font size requirements. Determine whether the font corresponding to the font parameter in the text rendering parameters is in the preset font library. If not, replace the font corresponding to the font parameter with the specified font.
11. The method according to any one of claims 1-9, wherein, The step of generating the target image corresponding to the image description information based on the page code file includes: The target page is rendered based on the page code file; The target image corresponding to the image description information is determined based on the target page.
12. An image generation apparatus, comprising: The content generation unit is configured to generate page text content based on image description information; The code generation unit is configured to generate a page code file based on the page text content and a preset set of text rendering parameters. The page corresponding to the page code file contains the page text content, and the text rendering parameters in the page code file belong to the set of text rendering parameters. The image generation unit is configured to generate a target image corresponding to the image description information based on the page code file.
13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.
14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-11.
15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-11.