Image generation method and device, storage medium and computer program product

Abstract

The invention discloses an image generation method and device, a storage medium and a computer program product, and the method comprises the steps: determining the image information of a to-be-generated image, can guarantee that a subsequently generated target composite image is highly consistent with an expected scene, and improves the pertinence and practicality of the generated image. Secondly, the image description information is input into the text image model, diversified background images can be generated, the visual diversity and authenticity of the images are improved, the text elements are further accurately fused into the background images, and element information of the text elements is reserved in the generated initial synthesis image. And finally, the extraction of the boundary information can ensure that the image enhancement model can accurately identify and process the boundary of each element in the subsequent image enhancement, which is beneficial to reducing the boundary blurring and artifacts and improving the definition and identifiability of the image elements. Through the processing process, a high-quality target composite image can be generated.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to an image generation method and apparatus, a storage medium, and a computer program product. Background Technology

[0002] In the field of intelligent character recognition (ICR), data collection faces numerous challenges. First, ICR tasks require handling diverse character styles, including different fonts, writing styles, writing angles, and speeds, making it extremely difficult to collect sufficiently diverse and representative training data. Second, manually annotating character data is both time-consuming and costly. To overcome these challenges, synthetic data generation methods are widely used in the ICR field, aiming to generate high-quality, diverse synthetic data using less or no labeled data to compensate for the lack of real-world data.

[0003] In related technologies, when performing data synthesis, the synthetic data is generally created by randomly selecting font styles from a font library and drawing specified text content onto a blank or predefined background.

[0004] However, while this method can quickly generate a large number of text images, the generated images often lack realism and diversity in complex backgrounds and varied scenes. Summary of the Invention

[0005] This application provides an image generation method and apparatus, storage medium, and computer program product to at least solve the problem of lack of realism and diversity in generated images in related technologies.

[0006] According to one embodiment of this application, an image generation method is provided, comprising: determining image information of an image to be generated; wherein the image information includes: image description information of the image to be generated and element information of text elements of the image to be generated; inputting the image description information into a pre-trained text-to-image model, and determining a background image of the image to be generated based on the text-to-image model; performing a fusion processing on the text elements and the background image based on the element information of the text elements to obtain an initial synthesized image of the image to be generated, and determining boundary information of the initial synthesized image; inputting the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain a target synthesized image of the image to be generated.

[0007] In an exemplary embodiment, the training steps of the text-based image model include: acquiring sample image information; wherein the sample image information includes: a sample image and the sample image information itself; the sample image information includes: image background description text and image element description text; inputting the sample image and the image background description text into the text-based image model to be trained to perform stylistic training on the text-based image model, thereby obtaining a stylized text-based image model; inputting the sample image and the image element description text into the stylized text-based image model to perform detail training on the stylized text-based image model, thereby obtaining a trained text-based image model.

[0008] In an exemplary embodiment, the step of inputting the sample image and the image background description text into a text-based image model to be trained for stylistic training to obtain a stylized text-based image model includes: reconstructing the image background description text based on the text-based image model to be trained to obtain a first reconstructed image; determining the style loss and content loss between the first reconstructed image and the sample image; weighting the style loss and the content loss to obtain a first target loss; adjusting the parameters of the text-based image model to be trained according to the first target loss, and continuing the step of reconstructing the image background description text based on the text-based image model to be trained to obtain the first reconstructed image until a first stopping condition is met, thereby obtaining a stylized text-based image model.

[0009] In an exemplary embodiment, the step of inputting the sample image and the image element description text into the stylized-trained text-to-image model to perform detailed training on the stylized-trained text-to-image model to obtain a trained text-to-image model includes: reconstructing the image element description text based on the stylized-trained text-to-image model to obtain a second reconstructed image; determining the boundary loss and visual consistency loss between the second reconstructed image and the sample image; weighting the boundary loss and the visual consistency loss to obtain a second target loss; adjusting the parameters of the text-to-image model to be trained according to the second target loss, and continuing the step of reconstructing the image element description text based on the stylized-trained text-to-image model to obtain a second reconstructed image until a second stopping condition is met, thereby obtaining a trained text-to-image model.

[0010] In an exemplary embodiment, the element information of the text element includes: element content and element coordinates of the element content; the step of fusing the text element and the background image based on the element information of the text element to obtain an initial composite image of the image to be generated includes: determining the element position of the element content in the background image based on the element coordinates of the element content; and drawing the element content at the corresponding element position in the background image to obtain the initial composite image of the image to be generated.

[0011] In an exemplary embodiment, determining the boundary information of the initial synthesized image includes: denoising the initial synthesized image to obtain a smoothed initial synthesized image; calculating the gradient of each pixel in the smoothed initial synthesized image to determine the gradient intensity of each pixel; comparing the gradient intensity of each pixel with a preset gradient intensity threshold to obtain a comparison result; filtering the pixels based on the comparison result, and determining the boundary information of the initial synthesized image based on the filtering result.

[0012] In an exemplary embodiment, the step of inputting the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated, includes: extracting features from the initial synthesized image based on the image enhancement model to obtain visual features of the initial synthesized image; and extracting features from the boundary information based on the image enhancement model to obtain boundary features; fusing the visual features and the boundary features, and updating the boundary region of the initial synthesized image based on the fusion result to obtain the target synthesized image of the image to be generated.

[0013] According to another embodiment of the present application, an image generation apparatus is also provided, comprising: a first determining module, configured to determine image information of an image to be generated; wherein the image information includes: image description information of the image to be generated and element information of text elements of the image to be generated; a second determining module, configured to input the image description information into a pre-trained text-to-image model, and determine a background image of the image to be generated based on the text-to-image model; a first processing module, configured to perform fusion processing on the text elements and the background image based on the element information of the text elements to obtain an initial synthesized image of the image to be generated, and determine the boundary information of the initial synthesized image; and a second processing module, configured to input the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain a target synthesized image of the image to be generated.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the above-described method at runtime.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above-described method through the computer program.

[0016] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0017] In this embodiment, determining the image information of the image to be generated ensures that the subsequently generated target synthetic image is highly consistent with the expected scene, improving the relevance and practicality of the generated image. Secondly, inputting image description information into the text-to-image model generates diverse background images, increasing the visual diversity and realism of the image. Furthermore, text elements are accurately integrated into the background image, and the generated initial synthetic image retains the element information of the text elements. Finally, the extraction of boundary information ensures that the image enhancement model can accurately identify and process the boundaries of each element in subsequent image enhancement, which helps reduce boundary blurring and artifacts, improving the clarity and recognizability of image elements. Through the above processing, a high-quality target synthetic image can be generated. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1 This is a hardware structure block diagram of a computer device for an image generation method according to an embodiment of this application;

[0020] Figure 2 This is a flowchart of an image generation method according to an embodiment of this application;

[0021] Figure 3 This is a flowchart of an image generation method according to an embodiment of this application;

[0022] Figure 4 This is a structural block diagram of an image generation apparatus according to an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus; "a plurality" means two or more.

[0025] The methods and embodiments provided in this application can be executed on a computer device or similar computing device, a cloud platform, a standalone physical server, or a software platform, wherein the aforementioned software platform runs through one or more servers. Taking running on a computer device as an example, Figure 1 This is a hardware structure block diagram of an image generation method according to an embodiment of this application. For example... Figure 1 As shown, a computer device may include one or more ( Figure 1 (Only one is shown in the image) A processor 102 and a memory 104 for storing data. In one exemplary embodiment, the computer device may further include a transmission device 106 for communication functions and an input / output device 108. The processor 102 may include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA). Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer device described above. For example, the computer device may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 Equivalent functions or ratios shown Figure 1 The functions shown have more different configurations.

[0026] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to computer devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer equipment. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0028] This embodiment provides an image generation method, which is applied to the aforementioned computer device, which can be a server device. Figure 2 This is a flowchart of an image generation method according to an embodiment of this application, which includes the following steps:

[0029] Step S202: Determine the image information of the image to be generated; wherein, the image information includes: image description information of the image to be generated and element information of text elements of the image to be generated.

[0030] Image information refers to the data set used to describe the various attributes and content of the image to be generated. It is the key data guiding the image synthesis and enhancement process. Image information can include not only image description information and text element information, but also detailed descriptions of multiple dimensions such as image style, scene layout, color scheme, and texture features, ensuring that the generated image meets specific visual style and information content requirements.

[0031] Image description information is a general description of the overall visual features of the image to be generated, including but not limited to the image's background, color, texture, and scene layout. For example, in the scenario of generating a bank card image, the image description information may include "dark blue background, gold embossed pattern, bank logo in the upper left corner," etc.

[0032] The element information of a text element describes the specific attributes of the text in the image to be generated, including but not limited to text content, font style, color, size, position coordinates, and any special effects (such as shadows and transparency). In the example of generating a bank card image, the text element information may include: card number: 1XXX; font: XX; color: X; position: (x=200, y=350); size: XX, and other detailed information.

[0033] Step S204: Input the image description information into the pre-trained text-to-image model, and determine the background image of the image to be generated based on the text-to-image model.

[0034] Among them, the text-to-image (t2i) model is a deep learning architecture that can generate corresponding images based on given text descriptions. This model can be based on generative adversarial networks (GANs) and variational autoencoders (VAEs), and by learning the complex relationship between text and images, it transforms the semantic information in the text into specific visual features, generating images that match the description.

[0035] Specifically, image description information can be input into a pre-trained text-to-image model. Based on information such as background color, embossed patterns, and logos described in the text, the model generates a background image that matches the description. For example, based on the image description information, the model can generate a dark blue background with finely embossed patterns at the bottom and a clear logo in the lower right corner. This background image will serve as the basis for subsequent text synthesis.

[0036] Step S206: Based on the element information of the text elements and the background image, perform fusion processing to obtain the initial synthesized image of the image to be generated, and determine the boundary information of the initial synthesized image.

[0037] Understandably, the background image, as the foundation or backdrop of the main image, carries the overall visual style, color scheme, and scene layout. Blending is a technique used in image compositing to precisely combine text elements with the background image. It involves seamlessly embedding the text content into the background image while considering effects such as text transparency, reflection, and shadows to ensure the naturalness and harmony of the resulting image.

[0038] The initial synthesized image refers to the preliminary synthesized result produced after the fusion processing step, where the element information of the text elements is combined with the background image. This image contains all the necessary visual elements. Boundary information refers to the edge position of each element or region in the image, which is crucial for subsequent image enhancement applications. For example, in a bank card image, the four corners of the card and the boundaries between different functional areas are all part of the boundary information.

[0039] In some embodiments, the coordinate information of the text element is extracted from the element information of the text element to determine its exact position in the background image. Simultaneously, the size of the text element is adjusted according to the size and proportion requirements in the element information to ensure its proportion is harmonious with the background image. Then, based on the exact position and adjusted size, the text element and the background image are merged to obtain an initial composite image.

[0040] In some embodiments, after generating the initial composite image, it needs to be carefully inspected to ensure that all text elements are correctly blended with the background image. If any visual imperfections are found, such as poorly processed edges or color inconsistencies, they can be corrected by fine-tuning the blending parameters or applying additional image processing techniques.

[0041] Step S208: Input the initial synthesized image and boundary information into the pre-trained image enhancement model so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated.

[0042] Boundary information refers to the edge positions and characteristics (such as edge intensity and direction) of each element or region in an image. Boundary information is crucial for guiding image enhancement processing, especially for maintaining the relative positions and layout of elements.

[0043] As is understandable, image enhancement models are deep learning models used to improve the visual quality of images. They can enhance the realism and aesthetics of images through various methods such as increasing image details, adjusting colors, improving sharpness, and correcting lighting. In this application, the image enhancement model has been pre-trained and is capable of performing targeted enhancement processing on the synthesized image based on the input boundary information. The final synthesized image processed by the image enhancement model exhibits significantly improved visual quality and realism, more closely resembling images in real-world scenes. The target synthesized image can be used to train the ICR model, helping the model learn to more accurately recognize characters in complex and varied scenes.

[0044] Specifically, after the initial image synthesis is completed, an initial synthesized image is obtained, but it may still be insufficient in terms of detail processing and realism. To improve the image quality and make it closer to the real scene, the next step is to apply image enhancement processing. Specifically, edge detection algorithms (such as the Canny algorithm) can be used to extract boundary information from the initial synthesized image. This includes the edges of each text element, the contours of the background image, and the boundaries of any important structures or patterns. The initial synthesized image and the extracted boundary information are then input into a pre-trained image enhancement model. The image enhancement model uses the boundary information as a control signal to guide the image enhancement process, ensuring that while enhancing image details and realism, the relative positions and layout between elements remain unchanged.

[0045] Steps S202-S208 above determine the image information of the image to be generated, ensuring that the subsequently generated target synthetic image is highly consistent with the expected scene, thus improving the relevance and practicality of the generated image. Secondly, inputting image description information into the text-to-image model enables the generation of diverse background images, increasing the visual diversity and realism of the image. Furthermore, text elements are accurately integrated into the background image, and the generated initial synthetic image retains the element information of the text elements. Finally, the extraction of boundary information ensures that the image enhancement model can accurately identify and process the boundaries of each element in subsequent image enhancement, which helps reduce boundary blurring and artifacts, improving the clarity and recognizability of image elements. Through the above processing, a high-quality target synthetic image can be generated.

[0046] In an exemplary embodiment, the training steps of the text-based image model include: acquiring sample image information; wherein the sample image information includes: sample images and sample image information; the sample image information includes: image background description text and image element description text; inputting the sample images and image background description text into the text-based image model to be trained to perform stylistic training on the text-based image model, thereby obtaining a stylistically trained text-based image model; inputting the sample images and image element description text into the stylistically trained text-based image model to perform detail training on the stylistically trained text-based image model, thereby obtaining a trained text-based image model.

[0047] It's important to note that sample image information refers to the collection of actual images used to train the text-based image model, along with their associated text descriptions. The text descriptions in the sample image information are divided into image background description text and image element description text, which together guide the model to generate images with similar visual features. Image background description text describes the background environment, color, texture, and other non-textual elements of the image. These descriptions help the text-based image model understand the image's base style and atmosphere to generate a visually harmonious background. Image element description text details the required text content in the image, its position, font, color, and other attributes. Image element description text is the key instruction guiding the text-based image model to accurately add text information on top of the background.

[0048] Specifically, the training steps for the text-based image model mainly include the following two stages: **Stylization Training Stage:** First, a series of sample images and their corresponding image background description texts need to be collected. These samples cover a variety of visual styles and background features to ensure the model can learn a wide range of image generation capabilities. The image background description text, along with the corresponding sample images, is input into the text-based image model to be trained. The model can learn how to generate background images with specific styles from the image background description text. The goal of this stage is to enable the model to master the visual features of different styles and generate background images with consistent styles based on the input text description. **Detailing Training Stage:** After stylization training, using the same set of sample images, this time focusing on the image element description text, the sample images and image element description text are input again into the stylized-trained text-based image model. In this stage, the model's learning objective shifts to accurately adding and rendering text elements on the existing background, ensuring that the visual effects of the elements (such as font, color, and shadow) are coordinated with the background style, while precisely adhering to the requirements of position and size.

[0049] After the two training phases described above, the text-based image model learns how to generate backgrounds with specific styles based on text descriptions, and then accurately and meticulously adds text elements to these backgrounds. This means that the model can not only generate diverse background images, but also ensure that the text information added to these backgrounds looks natural, harmonious, and correctly positioned. The trained text-based image model can be used to generate high-quality synthetic images in large batches for training ICR models, significantly improving their ability to recognize text in highly sensitive scenes.

[0050] In the above embodiments, through a two-stage training process, the text-based image model can gradually master and generate complex, realistic images with highly personalized features, thereby greatly expanding the data range and quality level of ICR technology when dealing with scenarios where sample acquisition is difficult. This is one of the key technologies for improving the performance of the ICR system.

[0051] In an exemplary embodiment, a sample image and image background description text are input into a text-based image model to be trained for stylistic training to obtain a stylized text-based image model. This includes: reconstructing the image background description text based on the text-based image model to be trained to obtain a first reconstructed image; determining the style loss and content loss between the first reconstructed image and the sample image; weighting the style loss and content loss to obtain a first target loss; adjusting the parameters of the text-based image model to be trained according to the first target loss; and continuing the step of reconstructing the image background description text based on the text-based image model to obtain the first reconstructed image until a first stopping condition is met, thereby obtaining the stylized text-based image model.

[0052] Stylization training is used to teach the model how to generate images with specific artistic styles or visual features. In this embodiment, the goal of stylization training is to enable the text-based image model to generate stylized background images that meet the requirements based on the image background description text. Image reconstruction refers to the image generated by the text-based image model based on the input text description. In this context, the first reconstructed image specifically refers to the background image generated based on the image background description text.

[0053] It's important to note that style loss measures the style difference between the model-generated image and the sample image, typically evaluated by calculating the statistical differences (such as mean and variance) between the image's feature layers. A high style loss indicates that the style of the generated image does not match that of the sample image. Content loss evaluates the content similarity between the model-generated image and the sample image, calculated by comparing the pixel-level differences between their feature layers. A large content loss indicates that the generated image fails to faithfully reproduce the content of the sample image.

[0054] The first objective loss is a comprehensive loss function formed during model training by combining style loss and content loss with different weights. It guides the model to learn how to balance style and content accuracy to generate images that reflect style while maintaining content realism. The first stopping condition is a metric set during training; when the model reaches this condition, training will stop. This may be based on a threshold of the loss function, a limit on the number of training epochs, performance metrics on the validation set, or other pre-defined training termination criteria.

[0055] Specifically, text containing background feature descriptions is input into the text-based image model, which attempts to generate a first reconstructed image based on these descriptions. Next, the generated first reconstructed image is compared with the sample image. Using a predefined algorithm, style loss (reflecting stylistic differences) and content loss (reflecting content errors) are calculated separately. These loss values ​​provide a quantitative comparison between the model-generated image and the expected target. To balance the importance of style and content, different weights are assigned to the style loss and content loss, and then the weighted loss values ​​are summed to form the first target loss. This weighting process can be adjusted according to the application scenario and training objectives. Based on the gradient information of the first target loss function, backpropagation is used to adjust the model's parameters, enabling the model to gradually learn how to reduce style and content errors and generate a reconstructed image that more closely resembles the sample image.

[0056] In the above embodiments, the text-generated image model can gradually master the ability to generate background images with a specific style from text descriptions, ensuring that the synthesized image has both the target visual style and can faithfully present the content of the text description.

[0057] In an exemplary embodiment, a sample image and image element description text are input into a stylized-trained text-to-image model to perform detailed training on the stylized-trained text-to-image model, thereby obtaining a trained text-to-image model. This includes: reconstructing the image element description text based on the stylized-trained text-to-image model to obtain a second reconstructed image; determining the boundary loss and visual consistency loss between the second reconstructed image and the sample image; weighting the boundary loss and visual consistency loss to obtain a second target loss; adjusting the parameters of the text-to-image model to be trained according to the second target loss; and continuing the step of reconstructing the image element description text based on the stylized-trained text-to-image model to obtain a second reconstructed image until a second stopping condition is met, thereby obtaining a trained text-to-image model.

[0058] Understandably, detailed training refers to further refining the learning objectives of a model after it has learned a certain level of style or basic skills. This focuses on the accuracy and detail of the images generated by the model, ensuring that it can produce highly realistic and detailed images. The second reconstructed image is generated based on the stylized text-to-image model, using text describing image elements. This reconstructed image includes specific details such as text content, position, size, and font.

[0059] Boundary loss is a loss function that measures the difference between the boundaries of text or elements in the model-generated image and the boundaries in the real sample image. The introduction of boundary loss ensures that the element boundaries of the generated image are clear, accurate, and close to the boundary features of the real image. Visual consistency loss is used to evaluate the consistency between the model-generated image and the real sample image in terms of visual style, color, texture, and lighting. The calculation of this loss helps maintain visual harmony between the generated image and the background image, avoiding unnatural visual abruptness. Combining boundary loss and visual consistency loss yields a second objective loss, which is a comprehensive loss function obtained through weighted calculation to guide detailed training. The optimization objective of the second objective loss is to maintain both the accuracy of element boundaries and visual consistency with the background when generating images.

[0060] Specifically, the text-based image model trained on stylization takes image elements describing text (including specific text content, position, size, font, and other details) as input to generate a second reconstructed image containing these elements. The boundary loss between the second reconstructed image and the sample image is calculated to ensure that the element boundaries of the generated image are clear, accurate, and match the boundaries in the real sample. Simultaneously, the visual consistency loss is calculated to evaluate the similarity in visual style between the generated image and the sample image, ensuring that the generated image maintains consistency with the background in terms of color, texture, and lighting. The boundary loss and visual consistency loss are weighted to obtain the second objective loss. Based on the gradient information of the second objective loss function, the parameters of the text-based image model are adjusted through backpropagation, enabling the model to generate images with more accurate boundaries and more harmonious visual styles in subsequent iterations.

[0061] In the above embodiments, the text-based image model, building upon its existing stylization capabilities, further enhances the realism of element boundaries and the consistency of visual style in the generated images. Ultimately, the trained text-based image model will be able to generate images that are both stylistically consistent and rich in detail with clear boundaries, providing high-quality training data for the ICR model and significantly improving its performance in recognizing complex scenes.

[0062] In an exemplary embodiment, the element information of the text element includes: element content and element coordinates of the element content; based on the element information of the text element, the text element and the background image are fused to obtain an initial composite image of the image to be generated, including: determining the element position of the element content in the background image based on the element coordinates of the element content; and drawing the element content at the corresponding element position in the background image to obtain the initial composite image of the image to be generated.

[0063] Specifically, based on the coordinates of the text elements, the exact location of the element content on the background image can be determined. For example, if the element coordinates indicate that the "card number" should appear in a certain area in the upper left corner of the image, then the precise location of this area can be found based on the coordinate information. Next, using text generation technology, the element content is drawn onto the corresponding position in the background image according to its specified coordinates and format (such as font, size, color, etc.). After determining the position of the card number, the card number: 1XXXX can be drawn in the upper left corner of the background image using the specified font and size. After completing the drawing of all the above text elements, the resulting image is the initial composite image. It is a preliminary composite result, containing all the planned text information and background, but there may still be room for improvement in details and visual effects, requiring further optimization through subsequent image enhancement processing.

[0064] In the above embodiments, the element information of text elements can be used to accurately synthesize the required text content on the background image, creating an initial synthetic image containing key visual information. This step is the foundation of the entire image synthesis and enhancement workflow, providing an accurate starting point for subsequent image enhancement processing, and ensuring that the synthesized image maintains visual realism while meeting the accuracy and diversity requirements of intelligent character recognition (ICR) training.

[0065] In an exemplary embodiment, determining the boundary information of the initial synthesized image includes: denoising the initial synthesized image to obtain a smoothed initial synthesized image; calculating the gradient of each pixel in the smoothed initial synthesized image to determine the gradient intensity of each pixel; comparing the gradient intensity of each pixel with a preset gradient intensity threshold to obtain a comparison result; filtering the pixels based on the comparison result, and determining the boundary information of the initial synthesized image based on the filtering result.

[0066] It's important to note that denoising is a crucial preprocessing step in edge detection, aiming to reduce random noise in the image and improve the accuracy and stability of subsequent edge detection. Noise can cause false edge responses, affecting edge continuity and localization. Image smoothing techniques are employed, such as Gaussian filtering, mean filtering, or median filtering. Among these, Gaussian filtering is the most commonly used method, effectively removing noise while maintaining relatively clear object boundaries in the image. Gradient calculation is used to detect changes in brightness or color gradients in the image. Pixels with strong gradients are typically located at edges. Commonly used gradient operators include the Sobel operator and the Prewitt operator. The Sobel operator uses two 3×3 matrices to calculate the gradients in the horizontal and vertical directions of the image, respectively, and then calculates the gradient strength at each pixel. The Prewitt operator is similar to the Sobel operator, but uses a simpler 3×3 matrix, making its calculation method simpler.

[0067] Specifically, the gradient operator is applied by convolving the selected gradient operator with the smoothed image to obtain the gradient values ​​of each pixel in both directions. The gradient intensity of each pixel is typically calculated using the Euclidean norm (the square root of the sum of the squares of the gradient values ​​in both directions) or the sum of absolute values ​​(the sum of the absolute values ​​of the gradients in both directions). By setting a gradient intensity threshold, true edges can be distinguished from false edges caused by noise. Pixels with gradient intensities higher than the threshold are considered part of an edge. The gradient intensity of each pixel is compared with the gradient intensity threshold. If the gradient intensity is greater than the threshold, the pixel is marked as an edge point; otherwise, it is marked as a non-edge point.

[0068] In some embodiments, after marking edge points, the neighborhood of each edge point can be examined. If the gradient intensity of a point is not a local maximum, it is considered not a true edge point and can be suppressed or excluded. Further, by tracing the continuity of edge points, discontinuous edges are connected to form complete boundaries. The filtered and refined set of edge points forms boundary information. Boundary information is typically stored in the form of pixel coordinates, representing the position and shape of the boundary in the image. Boundary information can be further used for tasks such as image enhancement, feature extraction, or object recognition.

[0069] In the above embodiments, boundary information can be accurately extracted from the initial synthesized image. The accuracy and completeness of the boundary information directly affect the image enhancement effect and the performance of the model in applications such as ICR.

[0070] In an exemplary embodiment, the step of inputting the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated, includes: extracting features from the initial synthesized image based on the image enhancement model to obtain visual features of the initial synthesized image; and extracting features from the boundary information based on the image enhancement model to obtain boundary features; fusing the visual features and the boundary features, and updating the boundary region of the initial synthesized image based on the fusion result to obtain the target synthesized image of the image to be generated.

[0071] Specifically, the first step in an image enhancement model is to perform multi-layer feature extraction on the initial synthetic input image. This typically involves using convolutional neural network layers to capture different levels of image features, including texture, color, and shape. These features are then transformed into a series of vectors or feature maps representing the image's visual characteristics. Feature extraction of boundary information is a crucial step in the image enhancement process. Boundary information is usually given in the form of a binary image, representing the edges of key elements in the image. The model uses special layers or components to process boundary information, converting it into boundary features that the model can understand and utilize.

[0072] After feature extraction, visual and boundary features can be fused to generate a comprehensive feature representation. After fusing visual and boundary features, the model performs image enhancement based on boundary guidance. This means the model prioritizes boundary features, ensuring high structural stability in the processing of regions near boundaries and avoiding boundary blurring or misalignment. During image enhancement, special attention is paid to boundary regions to ensure that boundary information is not compromised even during style adjustments or detail enhancements. This may include special processing of pixels near boundaries to maintain line continuity and shape accuracy.

[0073] The embodiments described above are merely some embodiments of this application, and not all embodiments. To better understand the above methods, the following description, in conjunction with embodiments, illustrates the process, but is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0074] This application aims to address the problem of scarce training data in highly sensitive OCR (Optical Character Recognition) scenarios (such as bank cards). Through innovative image generation technology, it achieves high-quality synthesis and enhancement of ICR (Intelligent Character Recognition) training data, improving model performance in scenarios where sample acquisition is difficult. References Figure 3 The following is a flowchart illustrating the image generation method of this application:

[0075] The image description information is determined, and the labels of the synthetic data are organized according to the designed image element meta-structure, which includes three main parts: text description (i.e., image description information), text-coordinate data pairs, and image-coordinate data pairs. Among them, image-coordinate is a non-essential data element, while image description information and text-coordinate are required inputs.

[0076] Image description information is input into the text-based image model, which generates the overall background pattern, i.e., the background image, thus broadening the diversity of background images. The text-based image model can be trained on demand using the LoRA (Low-Rank Adaptation) training method, requiring only a small number of samples and ensuring the transfer of various styles. For example, under the category of bank card certificates, it can generate various styles such as realistic background images, cartoon-style images, and annual card limited edition images.

[0077] The LoRA training process is as follows: 1) Data preparation: Select a stylization / style dataset containing two sets of text and their corresponding target images. The difference between the two sets of text is that one set only describes the background image, while the other set provides a detailed description of the elements within the image. For example, if the goal is to stylize an image into a bank card style, images with bank card designs need to be collected, along with two sets of aligned text: one set contains a simple background image description, while the other contains full text including bank card information such as the card number and date. 2) LoRA stylization training: In this stage, the LoRA low-rank adaptation layer is used for training to integrate the features of the target style into the generation process. The training objective of the model is to learn how to fuse the visual features of the original image with those of the target style. By calculating the style loss for each image (such as the content loss and style loss in style transfer), the generated images gradually approach the target style. In this stage, a standard loss function is used, and the training labels use simple background descriptions. 3) LoRA detail training: After initial training, the model is able to generate images with basic style features. The style refinement stage then proceeds, primarily by finely adjusting the parameters of the low-rank adaptation layer to further enhance the detail representation of the image style. At this point, more detail loss can be introduced, and the label uses a full description version with complete text information.

[0078] Based on the background image, we first determine whether image-coordinates exist. If they do, we can scale the image to the corresponding position in the background image based on its position information. If they do not exist, we can use a text generation model to draw the corresponding text in the specified coordinate area of ​​the background image based on the text-coordinate information, that is, the element information of the text element. By fusing the background image information, we can obtain the initial composite image.

[0079] Based on the initial synthesized image, boundary information can be extracted. This boundary information, along with the initial synthesized image, is then input into an image enhancement model to perform a process of enhancing the synthesized data and redrawing the initial card face to obtain the final target synthesized image. The image enhancement model is responsible for maintaining the invariance of the card layout and text information, and for reusing the text-to-image model, but employing an image-to-image computation method. Based on the initial synthesized image, it re-optimizes to address the unnatural boundary issues between elements. Finally, by integrating the tag meta information and the target synthesized image, the required text description and image-text pair dataset for ICR is automatically generated.

[0080] The image generation method provided in this application structurally defines the meta-data of synthetic elements, including text descriptions, text-coordinate data pairs, and image-coordinate data pairs, which serve as the basis for subsequent data generation. Utilizing a text-to-image model, synthetic images are automatically generated based on the defined structured data. Image enhancement techniques are applied to maintain the spatial layout of the synthesized text and background within the image, enhancing the visual effect of the synthetic image and making the target synthetic image more natural, realistic, and harmonious. Finally, it can be used for ICR model training, overcoming the limitations of insufficient real data and improving the model's recognition performance in complex scenes.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0082] This application also provides a structural block diagram of an image generation device. Figure 4 This is a structural block diagram of an image generation apparatus according to an embodiment of this application; as shown below. Figure 4 As shown, it includes:

[0083] The first determining module 42 is used to determine the image information of the image to be generated; wherein, the image information includes: image description information of the image to be generated, and element information of the text elements of the image to be generated;

[0084] The second determining module 44 is used to input the image description information into a pre-trained text-to-image model and determine the background image of the image to be generated based on the text-to-image model.

[0085] The first processing module 46 is used to perform fusion processing on the text element and the background image based on the element information of the text element to obtain an initial composite image of the image to be generated, and to determine the boundary information of the initial composite image.

[0086] The second processing module 48 is used to input the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated.

[0087] By using the aforementioned apparatus to determine the image information of the image to be generated, it is ensured that the subsequently generated target synthetic image is highly consistent with the expected scene, improving the relevance and practicality of the generated image. Secondly, inputting image description information into the text-to-image model enables the generation of diverse background images, increasing the visual diversity and realism of the image. Furthermore, text elements are accurately integrated into the background image, and the generated initial synthetic image retains the element information of the text elements. Finally, the extraction of boundary information ensures that in subsequent image enhancement, the image enhancement model can accurately identify and process the boundaries of each element. This helps reduce boundary blurring and artifacts, improving the clarity and recognizability of image elements. Through the above processing, a high-quality target synthetic image can be generated.

[0088] In one exemplary embodiment, the apparatus further includes a training module; the training module is configured to acquire sample image information; wherein the sample image information includes: a sample image, and the sample image information includes: image background description text and image element description text; inputting the sample image and the image background description text into a text-based image model to be trained to perform stylization training on the text-based image model to obtain a stylization-trained text-based image model; inputting the sample image and the image element description text into the stylization-trained text-based image model to perform detail training on the stylization-trained text-based image model to obtain a trained text-based image model.

[0089] In an exemplary embodiment, the training module is further configured to: reconstruct the image background description text based on the text-to-image model to be trained to obtain a first reconstructed image; determine the style loss and content loss between the first reconstructed image and the sample image; weight the style loss and the content loss to obtain a first target loss; adjust the parameters of the text-to-image model to be trained according to the first target loss; and return to the step of reconstructing the image background description text based on the text-to-image model to be trained to obtain the first reconstructed image, continuing until a first stopping condition is met, thereby obtaining a stylized trained text-to-image model.

[0090] In an exemplary embodiment, the training module is further configured to: reconstruct the image element description text based on the stylized trained text-to-image model to obtain a second reconstructed image; determine the boundary loss and visual consistency loss between the second reconstructed image and the sample image; weight the boundary loss and the visual consistency loss to obtain a second target loss; adjust the parameters of the text-to-image model to be trained according to the second target loss; and return to continue executing the step of reconstructing the image element description text based on the stylized trained text-to-image model to obtain a second reconstructed image until a second stopping condition is met, thereby obtaining a trained text-to-image model.

[0091] In an exemplary embodiment, the element information of the text element includes: element content and element coordinates of the element content; the first processing module 46 is further configured to determine the element position of the element content in the background image based on the element coordinates of the element content; and draw the element content at the corresponding element position in the background image to obtain an initial composite image of the image to be generated.

[0092] In an exemplary embodiment, the first processing module 46 is further configured to denoise the initial synthesized image to obtain a smoothed initial synthesized image; perform gradient calculation on each pixel of the smoothed initial synthesized image to determine the gradient intensity of each pixel; compare the gradient intensity of each pixel with a preset gradient intensity threshold to obtain a comparison result; filter each pixel based on the comparison result, and determine the boundary information of the initial synthesized image based on the filtering result.

[0093] In an exemplary embodiment, the second processing module 48 is further configured to perform feature extraction on the initial synthesized image based on the image enhancement model to obtain visual features of the initial synthesized image, and to perform feature extraction on the boundary information based on the image enhancement model to obtain boundary features; fuse the visual features and the boundary features, and update the boundary region of the initial synthesized image based on the fusion result to obtain the target synthesized image of the image to be generated.

[0094] Embodiments of this application also provide a storage medium including a stored program, wherein the program executes any of the methods described above when it is run.

[0095] Optionally, in this embodiment, the storage medium may be configured to store program code for performing the following steps:

[0096] S1, determine the image information of the image to be generated; wherein, the image information includes: image description information of the image to be generated, and element information of the text elements of the image to be generated;

[0097] S2, input the image description information into the pre-trained text-to-image model, and determine the background image of the image to be generated based on the text-to-image model;

[0098] S3, based on the element information of the text element, perform a fusion process on the text element and the background image to obtain an initial composite image of the image to be generated, and determine the boundary information of the initial composite image;

[0099] S4, the initial synthesized image and the boundary information are input into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated.

[0100] Embodiments of this application also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0101] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0102] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0103] S1, determine the image information of the image to be generated; wherein, the image information includes: image description information of the image to be generated, and element information of the text elements of the image to be generated;

[0104] S2, input the image description information into the pre-trained text-to-image model, and determine the background image of the image to be generated based on the text-to-image model;

[0105] S3, based on the element information of the text element, perform a fusion process on the text element and the background image to obtain an initial composite image of the image to be generated, and determine the boundary information of the initial composite image;

[0106] S4, the initial synthesized image and the boundary information are input into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated.

[0107] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0108] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0109] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0110] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0111] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0112] Embodiments of this application also provide a computer program that includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps in any of the above method embodiments.

[0113] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0114] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0115] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for generating an image, characterized in that, include: Determine the image information of the image to be generated; wherein, the image information includes: image description information of the image to be generated, and element information of text elements of the image to be generated; The image description information is input into a pre-trained text-to-image model, and the background image of the image to be generated is determined based on the text-to-image model. Based on the element information of the text element, the text element and the background image are fused to obtain an initial composite image of the image to be generated, and the boundary information of the initial composite image is determined. The initial synthesized image and the boundary information are input into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated.

2. The method according to claim 1, characterized in that, The training steps for the text image model include: Obtain sample image information; wherein, the sample image information includes: a sample image, and the sample image information includes: image background description text and image element description text; The sample image and the image background description text are input into the text-based image model to be trained, so as to perform stylization training on the text-based image model and obtain the stylized text-based image model. The sample image and the image element description text are input into the stylized text-to-image model to perform detailed training on the stylized text-to-image model, thereby obtaining the trained text-to-image model.

3. The method according to claim 2, characterized in that, The step of inputting the sample image and the image background description text into the text-based image model to be trained, so as to perform stylization training on the text-based image model and obtain the stylized text-based image model, includes: Based on the text-to-image model to be trained, the background description text of the image is reconstructed to obtain a first reconstructed image; Determine the style loss and content loss between the first reconstructed image and the sample image; The style loss and the content loss are weighted to obtain the first target loss; The parameters of the text-based image model to be trained are adjusted according to the first target loss, and the step of reconstructing the image background description text based on the text-based image model to be trained and obtaining the first reconstructed image continues until the first stopping condition is met, thereby obtaining the stylized trained text-based image model.

4. The method according to claim 2, characterized in that, The step of inputting the sample image and the image element description text into the stylized-trained text-to-image model to perform detail training on the stylized-trained text-to-image model and obtain the trained text-to-image model includes: Based on the stylized trained text-to-image model, the image element description text is reconstructed to obtain a second reconstructed image; Determine the boundary loss and visual consistency loss between the second reconstructed image and the sample image; The boundary loss and the visual consistency loss are weighted to obtain the second target loss; The parameters of the text-based image model to be trained are adjusted according to the second target loss, and the process of reconstructing the image element description text based on the stylized text-based image model and obtaining the second reconstructed image continues until the second stopping condition is met, thus obtaining the trained text-based image model.

5. The method according to claim 1, characterized in that, The element information of the text element includes: element content and element coordinates of the element content; the process of fusing the text element and the background image based on the element information of the text element to obtain the initial composite image of the image to be generated includes: Based on the element coordinates of the element content, determine the element position of the element content in the background image; The element content is drawn at the corresponding element positions in the background image to obtain the initial composite image of the image to be generated.

6. The method according to claim 1, characterized in that, Determining the boundary information of the initial synthesized image includes: The initial synthesized image is denoised to obtain a smoothed initial synthesized image; Gradient calculation is performed on each pixel of the smoothed initial synthesized image to determine the gradient intensity of each pixel; The gradient intensity of each pixel is compared with a preset gradient intensity threshold to obtain the comparison result; The pixels are filtered based on the comparison results, and the boundary information of the initial synthesized image is determined based on the filtering results.

7. The method according to claim 1, characterized in that, The step of inputting the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information, to obtain the target synthesized image of the image to be generated, includes: Based on the image enhancement model, feature extraction is performed on the initial synthesized image to obtain the visual features of the initial synthesized image; and based on the image enhancement model, feature extraction is performed on the boundary information to obtain the boundary features. By fusing the visual features and the boundary features, the boundary region of the initial synthesized image is updated based on the fusion result to obtain the target synthesized image of the image to be generated.

8. An image generation apparatus, characterized in that, include: The first determining module is used to determine the image information of the image to be generated; wherein, the image information includes: image description information of the image to be generated, and element information of the text elements of the image to be generated; The second determining module is used to input the image description information into a pre-trained text-to-image model and determine the background image of the image to be generated based on the text-to-image model. The first processing module is used to perform fusion processing on the text element and the background image based on the element information of the text element to obtain an initial composite image of the image to be generated, and to determine the boundary information of the initial composite image. The second processing module is used to input the initial synthesized image and the boundary information into a pre-trained image enhancement model, so that the image enhancement model performs image enhancement processing on the initial synthesized image based on the boundary information to obtain the target synthesized image of the image to be generated.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images