Artificial intelligence-based image generation method and device, computer device and medium

Abstract

This application belongs to the field of artificial intelligence technology and relates to an image generation method, apparatus, computer device, and storage medium based on artificial intelligence. The method includes: receiving an original image generated by an image generation model and corresponding text prompts; performing semantic segmentation and element recognition processing on the original image based on the text prompts to obtain element recognition results; generating layer structure data based on the element recognition results; receiving editing instructions from a user corresponding to the layer structure data via an interactive interface; performing layer editing operations on the original image based on the layered editing engine and corresponding to the editing instructions to obtain an image result; performing layer synthesis based on an intelligent synthesis algorithm and the image result to obtain a target image; and outputting the target image. This application can be applied to image generation scenarios in the fields of fintech and digital healthcare, improving the intelligence of image generation and ensuring the quality of the generated target image.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology and can be applied to fields such as fintech and healthcare, particularly to image generation methods, devices, computer equipment, and storage media based on artificial intelligence. Background Technology

[0002] In the current field of image generation technology, mainstream image generation models generally rely on text-based commands to generate complete images as their core mode. While some tools possess basic local redrawing capabilities, they lack true layered editing capabilities, failing to support users in performing fine-grained editing operations on layers. This results in low intelligence in image generation, making it difficult to guarantee the quality of the generated target image, often leading to situations where images failing to meet user requirements are generated. Specifically, traditional methods generate images solely based on text commands, lacking mechanisms for independent processing and optimization of each image element, making it difficult to accurately meet users' requirements for modifying image details.

[0003] For example, in the financial insurance sector, when insurance companies create promotional posters, if they want to modify and adjust the background layer and text layer of the insurance product image to highlight different information, traditional image generation models cannot directly achieve this. They can only regenerate the entire image, which is inefficient and difficult to accurately achieve the desired effect. In the healthcare sector, when doctors create medical teaching illustrations, if they want to modify the color, shape, or add annotations to a specific organ layer in the illustration, traditional models also cannot provide effective layered editing methods, affecting the accuracy and teaching practicality of medical illustrations.

[0004] Therefore, there is an urgent need to provide an image generation method with image editing capabilities to improve the intelligence and quality of image generation and meet the diverse image generation needs of various fields. Summary of the Invention

[0005] The purpose of this application is to propose an image generation method, apparatus, computer device, and storage medium based on artificial intelligence, in order to solve the technical problem that existing image generation models lack layered editing capabilities, resulting in low intelligence in image generation and difficulty in guaranteeing the quality of the generated target image.

[0006] Firstly, an artificial intelligence-based image generation method is provided, including: Receives an original image generated by a preset image generation model, and a text prompt word input by the user corresponding to the original image; Based on the text prompts, semantic segmentation and element recognition are performed on the original image to obtain the corresponding element recognition results. Based on the element recognition results, layer structure generation processing is performed to obtain the corresponding layer structure data; Based on a preset interactive interface, the system receives editing instructions from the user that correspond to the layer structure data. Based on a preset layered editing engine, the original image is subjected to layer editing operations corresponding to the editing instruction information to obtain the edited image result; Based on a preset intelligent synthesis algorithm, the image results are combined with layers to obtain the corresponding target image; The target image is output and processed based on a preset output format.

[0007] Secondly, an image generation device based on artificial intelligence is provided, comprising: The first receiving module is used to receive the original image generated by the preset image generation model, and the text prompt words input by the user corresponding to the original image; The processing module is used to perform semantic segmentation and element recognition processing on the original image based on the text prompt words to obtain the corresponding element recognition results; The generation module is used to perform layer structure generation processing based on the element recognition results to obtain the corresponding layer structure data. The second receiving module is used to receive editing instruction information issued by the user corresponding to the layer structure data based on a preset interactive interface; The operation module is used to perform layer editing operations on the original image based on a preset layered editing engine, corresponding to the editing instruction information, to obtain the edited image result; The compositing module is used to perform layer compositing processing based on a preset intelligent compositing algorithm and the image results to obtain the corresponding target image; The output module is used to process the target image based on a preset output format.

[0008] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described artificial intelligence-based image generation method.

[0009] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-described artificial intelligence-based image generation method.

[0010] In the aforementioned scheme implemented by the AI-based image generation method, apparatus, computer device, and storage medium, this application first receives an original image generated by a preset image generation model and a text prompt word input by the user corresponding to the original image; then, based on the text prompt word, semantic segmentation and element recognition processing are performed on the original image to obtain the corresponding element recognition result; and based on the element recognition result, layer structure generation processing is performed to obtain the corresponding layer structure data; subsequently, based on a preset interactive interface, editing instruction information corresponding to the layer structure data issued by the user is received; subsequently, based on a preset layered editing engine, layer editing operations corresponding to the editing instruction information are performed on the original image to obtain the edited image result; further, based on a preset intelligent synthesis algorithm and the image result, layer synthesis processing is performed to obtain the corresponding target image; finally, the target image is output based on a preset output format. Based on the above automated processing flow, this application employs an image processing method that performs element recognition, interactive interface processing, layer editing operations, and layer synthesis and output on the original image generated by the image generation model. Through intelligent layered editing functions, the original image generated by the image generation model can be transformed into a layered and editable layer structure, and users can perform fine-grained editing operations on the layers. Finally, the target image that meets the user's editing needs is synthesized and output, which effectively improves the intelligence of image generation and ensures the quality of the generated target image, thereby improving the user experience. Attached Figure Description

[0011] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the AI-based image generation method according to this application; Figure 3 This is a schematic diagram of a structure of an embodiment of the artificial intelligence-based image generation apparatus according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0014] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0016] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0017] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0018] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0019] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0020] It should be noted that the AI-based image generation method provided in this application is generally executed by a server / terminal device, and correspondingly, the AI-based image generation device is generally located in the server / terminal device.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0022] Continue to refer to Figure 2 The flowchart illustrates an embodiment of the AI-based image generation method according to this application. The order of steps in the flowchart can be changed, and some steps can be omitted, depending on different needs. The AI-based image generation method provided in this application can be applied to any scenario requiring image generation, and thus can be applied to products in these scenarios, such as image generation products in the financial insurance and digital healthcare fields. The AI-based image generation method includes the following steps: Step S201: Receive the original image generated by the preset image generation model, and the text prompt word input by the user corresponding to the original image.

[0023] In this embodiment, the artificial intelligence-based image generation method runs on an electronic device (e.g., Figure 1The server / terminal device shown can acquire the original image generated by a preset image generation model and the text prompts input by the user corresponding to the original image via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future wireless connection methods. The executing entity of this application is specifically an image generation system, which can be simply referred to as the system. The aforementioned image generation model can adopt mainstream AIGC image generation models. These AIGC image generation models generally use text command-driven generation of complete images as their core mode. Some tools only support basic local redrawing functions but lack true layered editing capabilities. The aforementioned original image is the original pixel image generated by the image generation model, and the aforementioned text prompts refer to the prompts used when generating the original image.

[0024] This application can be applied to image generation scenarios in the financial insurance and digital healthcare fields. For example, in a vehicle damage image recognition scenario within an insurance claims context in the financial insurance field, the original pixel image could be: a photograph generated by an image generation model, simulating a vehicle after a collision in a traffic accident. The image shows a car with severely deformed front, a detached bumper, a raised hood, and obvious scratches and dents on the body. The surrounding environment is likely a road scene with road surface and traffic signs. The corresponding text prompt could be: "Generate an image of a car damaged in a traffic accident. The front of the vehicle is severely deformed, the bumper is detached, the hood is raised, the body has scratches and dents, the scene is a city road with clear road surface and traffic signs."

[0025] Alternatively, in a financial investment chart analysis and image recognition scenario, the aforementioned original pixel image could be a chart image generated by an image generation model, containing various financial investment data. For example, it could include a line chart showing the price trend of a stock over a period of time, a bar chart comparing the price-to-earnings ratios of different industries, and a pie chart showing the proportion of various assets in a portfolio. The chart may have different colors to distinguish different data series and include elements such as axes and legends. The corresponding text prompt could be: "Generate a financial investment chart image, including a line chart of a stock's price trend (blue line, time span of one year), a bar chart comparing the price-to-earnings ratios of different industries (red and green bars), and a pie chart showing the proportion of assets in a portfolio (sectors of different colors), with clear axes and legends."

[0026] Furthermore, in the context of tumor identification in medical imaging within the healthcare field, the aforementioned raw pixel image could be: a simulated medical X-ray or CT image generated by an image generation model, showing a tumor in a certain part of the body (such as the lungs). The image shows the outlines of normal tissues and organs, while the tumor region exhibits a different density or grayscale value compared to the surrounding tissues, and may have an irregular shape, blurred edges, or spiky edges. The corresponding text prompt could be: "Generate a medical X-ray image of the lungs, showing an irregularly shaped tumor with spiky edges. The tumor density differs from the surrounding normal lung tissue, and the image clearly shows the lung outline and major vascular structures."

[0027] Alternatively, in the context of text and chart recognition in medical reports, the aforementioned original pixel image could be: an image of a medical report generated by an image generation model. The report includes the patient's personal information (such as name, age, gender, etc.), a description of the illness, examination items and results (which may be described in detail in text or include simple charts, such as a blood glucose fluctuation curve), as well as the doctor's diagnosis and prescription recommendations. The text portion has different fonts and sizes, and the charts have corresponding titles and labels. The corresponding text prompt could be: "Generate a medical report image containing the patient's name 'Zhang San', age '35 years old', gender 'male', etc. The illness description is 'recently experiencing headaches and dizziness'. The examination items and results have detailed text descriptions and a blood glucose fluctuation curve (blue lines, marking blood glucose values ​​at different time points). The doctor's diagnosis is 'possible tendency towards hypertension,' and the prescription recommendation is 'low-salt diet, regular blood pressure monitoring.'"

[0028] Step S202: Based on the text prompt words, perform semantic segmentation and element recognition processing on the original image to obtain the corresponding element recognition results.

[0029] In this embodiment, the specific implementation process of performing semantic segmentation and element recognition processing on the original image based on the text prompt words to obtain the corresponding element recognition results will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0030] Step S203: Based on the element recognition results, perform layer structure generation processing to obtain the corresponding layer structure data.

[0031] In this embodiment, the specific implementation process of generating layer structure data based on the element recognition results will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0032] Step S204: Receive editing instructions from the user corresponding to the layer structure data based on a preset interactive interface.

[0033] In this embodiment, the aforementioned interactive interface may include a professional mode interface for designers and a simplified mode interface for non-professional users.

[0034] The professional mode interface includes the following functionalities: 1) Professional Layer Operation Panel: Provides a Photoshop-like layer operation panel with various professional-grade interactive methods. It supports keyboard shortcuts, allowing designers to quickly perform operations such as showing / hiding, locking, and deleting layers by pressing specific shortcuts, such as pressing "Ctrl + H" to hide a layer. It also provides a pen tool, allowing designers to draw paths on layers and precisely select and edit paths by adjusting path nodes, facilitating complex image editing. 2) File Format Compatibility: Supports importing / exporting layered PSD files, achieving seamless integration with existing professional design tools. During import, the system parses the PSD file structure, converting layer information into a format it can recognize; during export, it organizes the layer information according to PSD format specifications, generating a PSD file for designers to continue processing in other professional design software. 3) Advanced features: Provides layer mask functionality. After designers create a layer mask, they can control the layer display area by drawing black and white colors on the mask. The black area hides the layer content, while the white area is displayed. Blending modes change the way colors are mixed between layers. For example, the Multiply mode mixes the colors of the upper layer with the colors of the lower layer to produce a darker effect, creating various special effects to meet the refined needs of professional designers.

[0035] Among them, the professional mode provides professional designers with a familiar and powerful editing environment. Through professional layer operation panels, file format compatibility and advanced functions, it enables designers to perform complex image editing efficiently and seamlessly integrate with existing workflows.

[0036] The simplified interface features include: 1) Drag-and-drop interface: A simple and intuitive drag-and-drop interface allows users to directly drag layers to adjust their positions and click on elements to trigger one-click enhancement, replacement, and other functions. For example, a user can drag a text layer to a suitable position, click on the text element, and select one-click enhancement; the system will automatically adjust the text color, font, and other parameters for optimization. The replacement function allows users to select a new image from their local storage to replace the original element image. 2) Templated layer combinations: Built-in templated layer combination schemes provide multiple preset layer combination templates for common graphic design needs, such as posters and WeChat official account header images. After selecting a suitable template, users can make simple modifications to the elements within the template. For example, to replace an image, click on the image element in the template and select a new image from their local storage; to modify text, double-click on the text element in the template to edit, quickly generating graphic materials that meet the requirements. 3) Intelligent suggestion function: Based on the user's layer layout, the system automatically analyzes the overall effect of the image and recommends suitable color schemes, fonts, and element combinations. For example, when a user creates a natural landscape-themed image, the system analyzes the image elements and colors, recommends green color schemes, provides nature-related fonts such as KaiTi and XingShu, and suggests suitable elements to add, such as birds and flowers, to help the user improve the quality of their design.

[0037] The simplified mode provides easy-to-use design tools for non-professional users. Through drag-and-drop interface, templated layer combinations, and intelligent suggestion function, it lowers the design threshold and enables users to quickly generate satisfactory graphic materials.

[0038] In addition, in the multi-mode interaction interface, users issue commands through various interaction methods (such as keyboard shortcuts, drag-and-drop operations, button clicks, etc.). These commands are passed to the layered editing engine in step two, triggering corresponding editing operations. For example, in professional mode, when a user presses a specific keyboard shortcut, the layered editing engine will perform corresponding layer display / hide, lock, delete, or other operations; in simple mode, when a user drags a layer to adjust its position, the layered editing engine will update the layer's position information based on the user's dragging operation.

[0039] Furthermore, the intelligent suggestion function in the multi-mode interaction interface automatically analyzes the overall effect of the image based on the user's layer layout and recommends suitable color schemes, fonts, and element combinations. This suggestion information is fed back to the layered editing engine, influencing the user's editing decisions. For example, the user may adjust the colors of layers in the layered editing engine based on the system's recommended color scheme; or add or replace corresponding layer elements in the layered editing engine based on the recommended element combinations.

[0040] Step S205: Based on the preset layered editing engine, perform layer editing operations on the original image corresponding to the editing instruction information to obtain the edited image result.

[0041] In this embodiment, the specific implementation process of performing layer editing operations on the original image based on the preset layered editing engine and corresponding to the editing instruction information to obtain the edited image result will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0042] Step S206: Perform layer synthesis processing based on the preset intelligent synthesis algorithm and the image result to obtain the corresponding target image.

[0043] In this embodiment, the specific implementation process of performing layer synthesis processing based on the preset intelligent synthesis algorithm and the image result to obtain the corresponding target image will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0044] Step S207: Output the target image based on a preset output format.

[0045] In this embodiment, the above-mentioned output format can refer to multi-format output, specifically including: 1. Finished image: Supports exporting to common formats such as JPG, PNG, and WebP. JPG format is suitable for lossy compression, with small file size and fast loading speed, suitable for scenarios where image quality requirements are not high but fast loading is needed, such as website images; PNG format supports transparency channels, suitable for images that need to retain transparent backgrounds, such as icons; WebP format combines the advantages of JPG and PNG, with high compression ratio and good image quality, suitable for network transmission. 2. Layered source files: Supports exporting to PSD, XD, and other formats, facilitating designers to continue editing in professional design tools. These formats retain layer layer information, allowing designers to further modify and improve each layer after importing the file, such as adjusting layer colors and adding new elements. 3. Vector format: For vector element layers, supports exporting to SVG format. SVG format is based on XML to describe images, supports lossless scaling, and will not cause pixel distortion no matter how many times the image is enlarged or reduced, suitable for scenarios requiring high-quality output, such as printed materials and logo design.

[0046] By offering multiple output formats, it caters to the needs of different users and usage scenarios. The finished image format allows for direct publishing and display, the layered source file format facilitates further editing by designers, and the vector format ensures high-quality image output, providing users with flexible output options.

[0047] This application first receives an original image generated by a preset image generation model and a text prompt word input by the user corresponding to the original image; then, based on the text prompt word, it performs semantic segmentation and element recognition processing on the original image to obtain the corresponding element recognition result; and based on the element recognition result, it performs layer structuring generation processing to obtain the corresponding layer structure data; then, it receives editing instruction information issued by the user corresponding to the layer structure data based on a preset interactive interface; subsequently, it performs layer editing operations on the original image based on a preset layered editing engine, corresponding to the editing instruction information, to obtain the edited image result; further, it performs layer compositing processing based on a preset intelligent synthesis algorithm and the image result to obtain the corresponding target image; finally, it outputs the target image based on a preset output format. Based on the above automated processing flow, this application employs an image processing method that performs element recognition, interactive interface processing, layer editing operations, and layer synthesis and output on the original image generated by the image generation model. Through intelligent layered editing functions, the original image generated by the image generation model can be transformed into a layered and editable layer structure, and users can perform fine-grained editing operations on the layers. Finally, the target image that meets the user's editing needs is synthesized and output, which effectively improves the intelligence of image generation and ensures the quality of the generated target image, thereby improving the user experience.

[0048] In some alternative implementations, step S202 includes the following steps: The original image is processed by visual semantic segmentation based on a preset visual segmentation model to obtain the corresponding visual elements.

[0049] In this embodiment, the aforementioned visual segmentation model can specifically employ a semantic segmentation model based on Visual Transformer (ViT). The input original image is processed using this visual semantic segmentation model. The model segments the image into multiple small blocks, analyzes each block and its relationship with other blocks using a self-attention mechanism, and extracts pixel features. Then, based on these pixel features, the model traverses each pixel of the original image, classifying it into different semantic categories, such as people, background, etc., thereby identifying and labeling visual elements with independent semantic meaning. Specifically, by comparing the similarity of pixel features with predefined feature templates for each category, the category with the highest similarity is taken as the semantic category of that pixel. Finally, the model labels each pixel in the image with its corresponding semantic category, thereby identifying and labeling visual elements with independent semantic meaning. For example, for an image containing people and scenery, the model can classify the person pixels into one category and the scenery pixels into another.

[0050] Call the preset large language model.

[0051] In this embodiment, the selection of the large language model (LLM) is not specifically limited and can be determined according to actual business needs. For example, GPT4, Qianwen, etc. can be used.

[0052] Based on the text prompts, the large language model is used to perform semantic association verification and result correction on the visual elements to obtain the corresponding output results.

[0053] In this embodiment, the semantic association verification and result correction processing includes: 1) Analyzing element semantics: Combining the text prompts used when generating the image, the visual segmentation result is verified using a Large Language Model (LLM). The LLM performs semantic understanding on the text prompts, analyzing the various elements described therein and their features. For example, if the prompt is "Draw an old man wearing a hat walking in the park," the LLM will identify the two main elements, "old man wearing a hat" and "park," as well as their related features.

[0054] 2) Determining Segmentation Correctness: The elements obtained from visual segmentation are compared with the semantics of the elements analyzed by the large language model. Visual segmentation may contain ambiguities, such as misclassifying objects with special meanings, like misclassifying a hat as background. The large language model will determine whether the visual segmentation result is correct based on the description of the elements in the text prompts. If a segmentation error is found, such as the hat being misclassified as background in the above example, the large language model will identify this error and correct the segmentation result according to the correct semantics in the text prompts, ensuring that each layer corresponds to a clear semantic unit.

[0055] The output includes pixel masks, semantic labels, and spatial bounding boxes. Specifically: Pixel Mask Generation: A pixel mask, a matrix of the same size as the image, is generated for each identified element. Each element in the matrix has a value of 0 or 1, where 1 indicates that the pixel belongs to the corresponding element, and 0 indicates that it does not. For example, for a person element, the pixel mask shows a value of 1 for the pixel at the person's location and 0 for other locations, thus clearly defining the element's location within the image. Semantic Labeling: Each element is assigned a semantic label, such as "person," "tree," or "building," to identify its category. Spatial Bounding Box Determination: By calculating the minimum and maximum horizontal and vertical coordinates of the element's pixels, the rectangular region occupied by the element in the image, i.e., the spatial bounding box, is obtained.

[0056] The output result is used as the element identification result.

[0057] Based on the above processing flow, this application performs preliminary identification of elements in an image through visual semantic segmentation, and analyzes and classifies pixel features using a self-attention mechanism. Then, semantic association verification, combined with textual prompts and a large language model, is used to correct the segmentation results and resolve potential segmentation ambiguities. Finally, the output includes pixel masks, semantic labels, and spatial bounding boxes. This information provides an accurate data foundation for subsequent layer generation, ensuring that each element has a clear definition and location description.

[0058] In some optional implementations of this embodiment, step S203 includes the following steps: Based on the element recognition results, layer file generation processing is performed to obtain the corresponding editable layer file.

[0059] In this embodiment, the layer file generation process includes: extracting pixel data: based on the pixel mask in the element recognition result, extracting pixel data for each element from the original image. For each element, traversing its pixel mask matrix, when a matrix element value is 1, extracting the RGB value of the pixel from the corresponding position in the original image. For example, for a person element, based on its pixel mask, finding all pixel positions in the original image with a mask value of 1, and extracting the values ​​of the red (R), green (G), and blue (B) channels at these positions. Organizing pixel data: organizing the extracted pixel data in RGBA (red, green, blue, transparency) format. Initially, transparency (A) can be set to a default value, such as 255 (representing complete opacity). Storing the pixel data of each element in this way generates an independent layer file. For example, for a person element, extracting the RGB values ​​of the corresponding pixels from the original image based on its pixel mask, setting the default transparency, and storing it in RGBA format as a person layer file, which only contains person information.

[0060] Based on the element identification results, a spatial relationship mapping table is established to obtain the corresponding spatial relationship mapping table.

[0061] In this embodiment, the spatial relationship mapping table establishment process includes: recording original coordinates: determining the starting position of the layer in the image, i.e., the original coordinates, through the coordinate information of the pixel mask. For the pixel mask of each element, find its minimum horizontal and vertical coordinates; these coordinates are the starting position of the layer in the image. For example, if the minimum horizontal coordinate of the pixel mask of a person element is x0 and the minimum vertical coordinate is y0, then the starting position of the person layer in the image is (x0, y0). Determining the stacking order: determining the display level of layers from top to bottom based on the front-to-back relationship of elements in the image. Generally, the layer containing elements closer to the observer is displayed on the upper layer, and the layer containing elements farther from the observer is displayed on the lower layer. For example, in an image containing a person and a background, the person is closer to the observer, and the background is farther from the observer, so the person layer should be displayed above the background layer. Setting transparency information: initially setting a default transparency value for each layer, which can be adjusted later according to actual needs. The transparency value range is usually 0-255, where 0 represents completely transparent and 255 represents completely opaque.

[0062] Based on the element recognition results, layer metadata generation processing is performed to obtain the corresponding layer metadata.

[0063] In this embodiment, the layer metadata generation process includes: determining element type: generating element type information for each layer, such as "person," "building," "vehicle," etc., which corresponds to the semantic tags obtained from the preceding semantic segmentation and recognition. Extracting prompt word fragments: extracting the part related to the element from the text prompt words of the generated image. For example, if the prompt word is "draw a girl in a red dress playing in the garden," for the person layer, the extracted prompt word fragment could be "girl in a red dress." Calculating element size: obtaining the element size by calculating the width and height of the element pixel mask. Assuming the minimum x-coordinate of the element pixel mask is Xmin, the maximum x-coordinate is Xmax, the minimum y-coordinate is Ymin, and the maximum y-coordinate is Ymax, then the element width = Xmax. Xmin, height = Ymax Ymin.

[0064] The editable layer file, the spatial relationship mapping table, and the layer metadata are integrated to obtain the corresponding integrated data.

[0065] In this embodiment, the editable layer file, spatial relationship mapping table, and layer metadata can be integrated and processed, and the resulting integrated data can be used as the corresponding layer structure data.

[0066] The output consists of a set of independent editable layer files, metadata for each layer, and a spatial relationship mapping table between the layers. This output ensures that each layer has independent information and positional relationships, facilitating subsequent layer editing and compositing operations. For example, during image editing, the layer of a specific element can be quickly located based on the layer metadata, and the display order and position of the layers can be adjusted according to the spatial relationship mapping table, enabling more flexible image processing.

[0067] The integrated data is used as the layer structure data.

[0068] In this embodiment, the identified elements are transformed into actual editable layers. By generating layer files, establishing spatial relationship mapping tables and layer metadata, each layer has independent information and positional relationships, facilitating subsequent layer editing and compositing operations.

[0069] Based on the above processing flow, this application generates layer files, storing the pixel data of each element independently, facilitating the editing and modification of individual elements. A spatial relationship mapping table records the original coordinates, stacking order, and transparency information of the layers, providing a foundation for layer compositing and ensuring that layers are displayed in the correct position and order. Generating layer metadata supports subsequent precise retrieval and editing of layers, such as quickly finding specific layers based on element type, improving the efficiency and accuracy of image editing.

[0070] In some optional implementations, the editing instruction information includes layer operation information, text instruction information, and element adjustment information; step S205 includes the following steps: Based on the layered editing engine, the original image is subjected to layer editing operations corresponding to the layer operation information to generate the corresponding first image.

[0071] In this embodiment, the layer editing operations described above may include layer list view operations, basic layer transformation operations, and layer stacking order adjustments.

[0072] Specifically, layer list view operations include: 1) Controlling layer display and hiding: In the layer list view, each layer corresponds to an option box. When the user checks the option box, the system sends a display command to the rendering module, which then draws and displays the pixel data corresponding to that layer on the canvas according to this command. When the user unchecks the option box, the system sends a hide command, and the rendering module skips the pixel data of that layer when drawing the canvas, thus hiding the layer. For example, in an image editing process containing multiple layers such as a person and a background, if the person layer option is unchecked, only the background and other layer content will be displayed on the canvas. 2) Locking layers: Each layer has a lock button next to it. When the user clicks the lock button, the system marks the layer as locked and records it in the layer status information. In subsequent editing operations, the system will first check the layer status. If the layer is found to be locked, all editing commands for that layer, such as moving, scaling, and modifying pixels, will be rejected to prevent the user from accidentally modifying the layer content. 3) Delete Layer: When the user selects the delete button, the system first removes the layer's relevant information from the layer list, including the layer name and metadata. Then, the system deletes the corresponding pixel file from memory or disk that stores the layer's pixel data. Simultaneously, the layer's pixel data is no longer considered during canvas rendering, achieving the effect of removing the layer from the canvas. 4) Copy Layer: When the user clicks the copy button, the system reads all information about the selected layer, including pixel data, metadata, and layer state. Then, the system creates a new layer object, copies the read information into the new layer object, and generates a unique identifier for the new layer. Finally, the new layer is added to the layer list, and both the original layer and the newly copied layer are displayed simultaneously during canvas rendering.

[0073] Basic layer transformation operations include: dragging and dropping layers to any position on the canvas; scaling layers by dragging the control points on the layer's edges (drag inwards to shrink the layer, drag outwards to enlarge the layer); rotating layers around their center point by dragging the rotation control points outside the layer to adjust the angle; horizontal flip mirrors the layer along the horizontal axis, and vertical flip mirrors it along the vertical axis; and adjusting the layer's opacity by sliding the transparency bar, with real-time feedback on the canvas.

[0074] Layer stacking order adjustment includes: 1) Changing layer order: When a user drags layers in the layer list, the system monitors the layer position changes in real time. When a layer is dragged to a new position, the system records the new layer order information. For example, if the original layer list order was Layer 1, Layer 2, Layer 3, and the user drags Layer 3 above Layer 1, the new order becomes Layer 3, Layer 1, Layer 2. 2) Updating the spatial location map: The system automatically updates the spatial location map according to the new layer order. The spatial location map records the stacking order, original coordinates, and other information of each layer. During the update, the system rearranges the records in the map according to the new layer order to ensure that the layers can be displayed in the correct order during subsequent compositing. For example, when compositing an image, the system will draw the pixel data of each layer onto the canvas in sequence according to the order in the spatial location map, with the first drawn layer on the bottom layer and the last drawn layer on the top layer.

[0075] The layer editing features provide users with basic layer management functions, enabling them to easily control the display, editing, and position of layers. Through layer list view operations, basic transformation operations, and stacking order adjustments, it meets users' various basic layer editing needs.

[0076] Perform a layer regeneration operation on the first image corresponding to the text instruction information to generate a corresponding second image.

[0077] In this embodiment, the above-mentioned layer regeneration operation includes local prompt word generation, local content generation, and layer content replacement.

[0078] Specifically, local prompt generation includes: 1) Extracting metadata: The system first obtains information such as the original prompt and element type from the metadata of the target layer. For example, if the target layer is a person layer, its metadata may contain the original prompt "Draw a person walking in the park" and the element type "person". 2) Generating local prompts by combining user instructions: The system analyzes the user's new input instructions and integrates them with the original prompt and element type. Through a specific algorithm, key information in the user's instructions is identified and added to the original prompt to generate more accurate local prompts. For example, if the original prompt is "Draw a beautiful garden", the target layer is flowers in the garden, and the user's new instruction is "Make the flowers red", the algorithm will analyze the key information "flowers" and "make red", and combine them with the original prompt to generate the local prompt "In the garden, make the flowers red".

[0079] Local content generation includes: 1) Determining the generation range: The system uses the pixel mask of the target layer as a reference to determine the range where new content needs to be generated. The pixel mask is a matrix of the same size as the image, with each element having a value of 0 or 1, where 1 indicates that the pixel belongs to the target layer. Based on the pixel positions with a value of 1 in the pixel mask, the system defines a rectangular area as the generation range. 2) Calling the image generation model (such as the AIGC raw image model): The system calls a pre-trained image generation model, passing the generated local prompts and the determined generation range as input parameters to the model. The model calculates and generates new pixel content within the defined generation range based on the description in the local prompts. For example, the model only calculates the area where the flower is located, generating pixel data for a red flower based on the prompt "turn the flower red".

[0080] Layer content replacement includes: 1) Obtaining new pixel content: After the image generation model generates new pixel content, the system reads it and stores it in memory. 2) Replacing the original layer content: Based on the pixel mask of the target layer, the system overwrites the corresponding pixel positions of the original layer with the newly generated pixel content. The pixel data of other layers remains unchanged. For example, if the flower layer is replaced by newly generated red flower pixel content, the pixel data of other layers such as the background are not affected, thus achieving local redrawing of the target layer without destroying the global image.

[0081] The layer regeneration function allows users to regenerate parts of a layer by generating local prompts, generating local content, and replacing layer content, enabling modifications to specific elements without regenerating the entire image, thus improving editing efficiency and flexibility.

[0082] Perform element attribute editing operations on the second image corresponding to the element adjustment information to generate the corresponding third image.

[0083] In this embodiment, the above-mentioned element attribute editing operations include color adjustment, style transfer, and detail repair.

[0084] Specifically, color adjustments include: 1) Hue adjustment: The system provides a hue adjustment slider. When the user slides the slider, the system obtains the current hue value. The hue value range is usually 0-360 degrees, representing different color hues. The system converts the RGB color value of each pixel on the layer according to the hue value. For example, it converts the RGB color space to the HSV color space (H represents hue, S represents saturation, V represents brightness), modifies the H value, and then converts it back to the RGB color space, thereby changing the basic hue of the layer color. 2) Saturation adjustment: The user sets the saturation value through the saturation adjustment slider. The saturation value range is usually 0-100%. The system converts the layer pixels from the RGB color space to the HSV color space, modifies the S value according to the saturation value set by the user, and then converts it back to the RGB color space. Increasing the saturation will make the color more vibrant, while decreasing the saturation will make the color more muted. 3) Brightness adjustment: The brightness adjustment slider allows the user to set the brightness value, which is also usually 0-100%. The system converts pixels to the HSV color space, modifies the V value, and then converts them back to the RGB color space. Increasing brightness makes the layer brighter, while decreasing brightness makes it darker. These adjustments are independent and do not affect the colors of other layers.

[0085] Style transfer includes: 1) Selecting an art style model: The system has built-in various art style models, such as realistic, cartoon, and oil painting. After the user selects an art style on the interface, the system loads the corresponding style model. 2) Applying style transfer: The system inputs the pixel data of the target layer into the selected art style model. The model analyzes and processes the pixel's color and texture features, and transforms the pixels according to the characteristics of the selected style. For example, when selecting the cartoon style, the model simplifies pixel details, enhances color contrast, and gives the layer a cartoonish effect, while keeping the styles of other layers unchanged.

[0086] Detailed repair includes: 1) Identifying defective areas: The system integrates an AI repair model that automatically analyzes the pixel data of the layer to identify defective areas, such as blurriness and noise. The model determines which areas need repair by comparing the characteristics of surrounding normal pixels. 2) Targeted repair: Once a defective area is identified, the AI ​​repair model uses interpolation, filtering, and other algorithms to fill and repair the defective area based on information from surrounding normal pixels. For example, for blurry areas, the model can analyze the edge and texture information of surrounding clear pixels to regenerate clear pixel data, improving the quality of the layer.

[0087] Among them, the element attribute editing operation provides users with fine-grained editing functions for layer elements. Through color adjustment, style transfer and detail repair, users can accurately control the color, style and quality of the layer to meet users' needs for high-quality image editing.

[0088] The third image is used as the edited image result.

[0089] In this example, after basic layer operations, partial layer regeneration, and fine-tuning of element attributes, the layer data has been modified and improved according to user requirements. This edited layer data, including pixel data, spatial location mapping tables, and metadata for each layer, is used to pass to subsequent layer compositing processing.

[0090] Based on the above processing flow, this application performs layer editing operations on the original image corresponding to the editing instructions input by the user by using a layered editing engine. This can meet the user's various editing needs for layers, thereby enabling intelligent editing of layers according to user needs. This effectively improves image editing efficiency and flexibility, enhances the user experience, and helps improve the intelligence and adaptability of image generation.

[0091] In some alternative implementations, step S206 includes the following steps: Obtain a spatial location mapping table corresponding to the image result.

[0092] In this embodiment, the aforementioned designated spatial location mapping table refers to a pre-constructed spatial location mapping table corresponding to all the aforementioned designated layers. The construction process of the aforementioned designated spatial location mapping table can refer to the aforementioned process of establishing a spatial relationship mapping table based on the element recognition results to obtain the corresponding spatial relationship mapping table, which will not be elaborated on here.

[0093] Based on the specified spatial location mapping table, pixel-level compositing processing is performed on all specified layers in the image result to obtain the corresponding preliminary composite image.

[0094] In this embodiment, the specific implementation process of performing pixel-level compositing on all specified layers in the image result based on the specified spatial location mapping table to obtain the corresponding preliminary composite image will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0095] The initial synthesized image is subjected to edge blending processing to obtain the corresponding first synthesized image.

[0096] In this embodiment, edge blending primarily addresses potential mismatches at the junctions between layers to smooth out imperfections. First, the system detects edge regions between adjacent layers. This is achieved by analyzing the gradient information of layer pixels, for example, using the Sobel operator to calculate the gradient value of each pixel in the horizontal and vertical directions; regions with larger gradient values ​​are often edge regions. For the detected edge regions, the edge blending algorithm analyzes information such as the color and brightness of the edge pixels of adjacent layers. Taking two adjacent layers a and b as an example, in their edge transition region, the algorithm selects a pixel band of a certain width (this width can be set according to actual needs, such as 10 pixels). For each pixel (x, y) in this transition region, the algorithm obtains the pixel color values ​​Ca and Cb of layers a and b near that position (according to certain sampling rules, such as bilinear interpolation), as well as the brightness values ​​La and Lb. Then, a blending weight w is calculated based on the pixel's position in the transition region. This weight is usually related to the pixel's distance from the edge; the closer to the edge, the more the weight may favor the pixel value of one of the layers. For example, a linear blending method can be used. Assuming the transition region extends from the edge of layer a to the edge of layer b, and for a pixel within the transition region, its distance from the edge of layer a is d, and the total width of the transition region is D, then the blending weight w = Dd. The color value C of the pixel in the transition region can then be calculated using the following formula: C = w × Cb + (1 The brightness value L can also be calculated in a similar way: L = w × Lb + (1) × Ca. w)×La. In this way, all pixels in the transition area are calculated and processed to eliminate harsh stitching marks and make the transition between layers more natural.

[0097] In this process, an edge blending algorithm is applied to smooth the transitions between layers. Since the edges of different layers may not match, the edge blending algorithm eliminates harsh stitching marks, making the transitions between layers more natural.

[0098] The first synthesized image is subjected to color consistency correction processing to obtain the corresponding second synthesized image.

[0099] In this embodiment, the specific implementation process of performing color consistency correction on the first synthesized image to obtain the corresponding second synthesized image will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0100] The second synthesized image is used as the target image.

[0101] Based on the above processing flow, this application uses pixel-level compositing, edge blending processing, and color consistency correction to synthesize multiple layers into a complete high-quality composite image. This effectively solves problems such as position mismatch, unnatural edges, and color inconsistency that may occur during the layer compositing process, and effectively improves the quality of the generated composite image.

[0102] In some optional implementations of this embodiment, the step of performing pixel-level compositing processing on all specified layers in the image result based on the specified spatial location mapping table to obtain the corresponding preliminary composite image includes the following steps: Retrieve all specified layers corresponding to the image result.

[0103] In this embodiment, the specified layer refers to all layers included in the above image result.

[0104] Obtain the layer coordinates and stacking order corresponding to all specified layers from the specified spatial location mapping table.

[0105] In this embodiment, the aforementioned designated spatial location mapping table refers to a pre-constructed spatial location mapping table corresponding to all the specified layers. This designated spatial location mapping table records in detail the coordinate position (layer coordinates) of each layer in the final composite image and the stacking order between layers. For example, assuming that an image containing three layers—background, person, and decorative elements—is to be composited, the spatial location mapping table will clearly indicate that the background layer is located at the bottom layer, and its coordinate range covers the entire composite image area; the person layer is in the middle layer, with a specific coordinate range representing its position in the image; and the decorative element layer is at the top layer, also with a corresponding coordinate range.

[0106] Based on the layer coordinates and the stacking order, all the specified layers in the image result are pixel-overlayed to obtain the corresponding processed image.

[0107] In this embodiment, the system processes each layer sequentially from the bottom layer to the top layer according to the specified spatial location mapping table. For each layer, the pixel value at the corresponding coordinate position is read from the original pixel data of the image result. Taking the background layer as an example, the system reads the pixel value at coordinates (x, y) from the background image file. This pixel value contains the color information of that position (usually represented by the RGB color model, i.e., the values ​​of the three components of red, green, and blue, generally ranging from 0 to 255). Then, this pixel value is placed at the corresponding position (x, y) in the composite image. Next, the character layer is processed. The pixel value at coordinates (x, y) in the character layer (where (x, y) is relative to the character layer itself, and will be converted to the corresponding coordinates in the composite image in the mapping table) is read and superimposed on the corresponding position in the composite image. If the position already has the pixel value of the background layer, for image formats with an alpha channel (such as PNG), the system determines how to mix the two pixel values ​​based on the alpha value (0-1, where 0 represents complete transparency and 1 represents complete opacity) of the character layer pixel. For example, using a simple blending formula: the composite pixel value C = α × P_person + (1 The image is calculated as α) × P_background, where α is the alpha value of a pixel in the character layer, P_character is the color value of a pixel in the character layer, and P_background is the color value of a pixel at the corresponding position in the background layer. By processing all layers sequentially in this way, a preliminary composite image is formed.

[0108] The processed image is used as the initial synthesized image.

[0109] Based on the above processing flow, this application can automatically and accurately superimpose the pixel data of each layer to form a preliminary composite image by using the layer coordinates and stacking order recorded in the specified spatial location mapping table corresponding to the image result, effectively ensuring the accuracy of the generated preliminary composite image.

[0110] In some optional implementations of this embodiment, the step of performing color consistency correction processing on the first synthesized image to obtain the corresponding second synthesized image includes the following steps: A global color analysis is performed on the first synthesized image to obtain the corresponding color distribution information.

[0111] In this embodiment, the purpose of color consistency correction is to ensure that the overall color tone of the synthesized image is uniform. First, the system performs a global color analysis on the first synthesized image, statistically analyzing the color distribution of all pixels in the first synthesized image, including the mean, variance, and other statistical quantities of each color channel (such as RGB channels), and uses these as corresponding color distribution information. For example, the average value R of all pixels in the red channel, the average value G of the green channel, and the average value B of the blue channel are calculated. These average values ​​can reflect the overall color cast of the image in each color channel.

[0112] Based on the color distribution information, the target color parameters in the first synthesized image that meet the adjustment conditions are determined.

[0113] In this embodiment, the color parameters that need to be adjusted, such as hue, saturation, and brightness, can be determined based on the color distribution information obtained from the analysis.

[0114] Obtain the parameter adjustment strategy corresponding to the target color parameter.

[0115] In this embodiment, the above parameter adjustment strategy includes: Hue adjustment mainly changes the basic hue of colors in the image, such as adjusting a reddish image to a more neutral hue. Specifically, this can be achieved by converting the image from the RGB color space to the HSV (Hue, Saturation, Brightness) color space, directly adjusting the hue value H in the HSV space, and then converting it back to the RGB space. Saturation adjustment changes the vividness of colors in the image. If the overall saturation of the image is too high or too low, it can be adjusted using a saturation adjustment coefficient ks (typically between 0 and 2; less than 1 decreases saturation, and greater than 1 increases saturation). For the saturation value S of each pixel, the adjusted saturation value S′ = ks × S. Brightness adjustment changes the overall brightness of the image. This can also be adjusted using a brightness adjustment coefficient kl (the range is set according to requirements, generally greater than 0). For the brightness value L of each pixel (calculated in HSV space or through other methods), the adjusted brightness value L′ = kl × L. During the adjustment process, the system will make targeted adjustments to the color parameters of different layers based on the contribution of each layer to the composite image. For example, if a layer occupies a large area in an image and has significant color deviation, the color parameters of that layer will be adjusted in detail to ensure color consistency across the entire image. By comprehensively adjusting color parameters such as hue, saturation, and brightness, the entire image appears more harmonious, resulting in a high-quality composite image.

[0116] The target color parameters in the first synthesized image are adjusted based on the parameter adjustment strategy to obtain the adjusted specified synthesized image.

[0117] In this embodiment, the target color parameters in the first composite image can be adjusted based on the above parameter adjustment strategy, and the generated specified composite image can be used as the corresponding second composite image.

[0118] The specified composite image is used as the second composite image.

[0119] Based on the above processing flow, this application performs color consistency correction on the first composite image to effectively ensure the overall color tone uniformity of the image. Since different layers may have color differences due to different generation methods or editing operations, this application adjusts the colors of each layer using a parameter adjustment strategy to generate a high-quality composite image. This makes the entire composite image look more harmonious, thereby effectively improving the generation quality of the second composite image.

[0120] In some alternative implementations, the user information obtained is subject to user consent and complies with relevant laws and policies.

[0121] Furthermore, any software tools or components not belonging to our company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] It should be emphasized that, to further ensure the privacy and security of the target images, they can also be stored in a blockchain node.

[0124] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0125] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0126] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of an image generation device based on artificial intelligence, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0127] like Figure 3 As shown, the AI-based image generation device 300 described in this embodiment includes: a first receiving module 301, a processing module 302, a generation module 303, a second receiving module 304, an operation module 305, a synthesis module 306, and an output module 307. Wherein: The first receiving module 301 is used to receive the original image generated by the preset image generation model, and the text prompt words input by the user corresponding to the original image; Processing module 302 is used to perform semantic segmentation and element recognition processing on the original image based on the text prompt words to obtain the corresponding element recognition results; The generation module 303 is used to perform layer structure generation processing based on the element recognition results to obtain the corresponding layer structure data. The second receiving module 304 is used to receive editing instruction information corresponding to the layer structure data issued by the user based on a preset interactive interface; The operation module 305 is used to perform layer editing operations on the original image based on a preset layered editing engine, corresponding to the editing instruction information, to obtain the edited image result; The compositing module 306 is used to perform layer compositing processing based on a preset intelligent compositing algorithm and the image results to obtain the corresponding target image; The output module 307 is used to output the target image based on a preset output format.

[0128] In some optional implementations of this embodiment, the processing module 302 includes: The segmentation submodule is used to perform visual semantic segmentation processing on the original image based on a preset visual segmentation model to obtain the corresponding visual elements; Calling submodules is used to invoke the preset large language model; The processing submodule is used to perform semantic association verification and result correction on the visual elements based on the text prompt words and the large language model to obtain the corresponding output results. The first determining submodule is used to use the output result as the element identification result.

[0129] In some optional implementations of this embodiment, the generation module 303 includes: The first generation submodule is used to perform layer file generation processing based on the element recognition results to obtain the corresponding editable layer file; A submodule is established to perform spatial relationship mapping table creation based on the element recognition results, thereby obtaining the corresponding spatial relationship mapping table. The second generation submodule is used to perform layer metadata generation processing based on the element recognition results to obtain the corresponding layer metadata. The integration submodule is used to integrate the editable layer file, the spatial relationship mapping table, and the layer metadata to obtain the corresponding integrated data; The second determining submodule is used to use the integrated data as the layer structure data.

[0130] In some optional implementations of this embodiment, the editing instruction information includes layer operation information, text instruction information, and element adjustment information; the operation module 305 includes: The first operation submodule is used to perform layer editing operations on the original image based on the layered editing engine, corresponding to the layer operation information, to generate the corresponding first image; The second operation submodule is used to perform a layer regeneration operation on the first image corresponding to the text instruction information to generate a corresponding second image. The third operation submodule is used to perform element attribute editing operations on the second image corresponding to the element adjustment information, and generate the corresponding third image. The third determining submodule is used to use the third image as the edited image result.

[0131] In some optional implementations of this embodiment, the synthesis module 306 includes: The acquisition submodule is used to acquire a specified spatial location mapping table corresponding to the image result; The compositing submodule is used to perform pixel-level compositing processing on all specified layers in the image result based on the specified spatial location mapping table to obtain the corresponding preliminary composite image; The fusion submodule is used to perform edge fusion processing on the preliminary synthesized image to obtain the corresponding first synthesized image; The correction submodule is used to perform color consistency correction processing on the first synthesized image to obtain the corresponding second synthesized image; The fourth determining submodule is used to use the second synthesized image as the target image.

[0132] In some optional implementations of this embodiment, the synthesis submodule includes: The first acquisition unit is used to acquire all specified layers corresponding to the image result; The second acquisition unit is used to acquire the layer coordinates and stacking order corresponding to all the specified layers from the specified spatial location mapping table; The processing unit is configured to perform pixel overlay processing on all the specified layers in the image result based on the layer coordinates and the stacking order to obtain the corresponding processed image; The first determining unit is used to use the processed image as the preliminary synthesized image.

[0133] In some optional implementations of this embodiment, the correction submodule includes: The analysis unit is used to perform global color analysis on the first synthesized image to obtain the corresponding color distribution information; The second determining unit is used to determine the target color parameters in the first synthesized image that meet the adjustment conditions based on the color distribution information. The third acquisition unit is used to acquire the parameter adjustment strategy corresponding to the target color parameter; An adjustment unit is used to adjust the target color parameters in the first composite image based on the parameter adjustment strategy to obtain the adjusted specified composite image; The third determining unit is used to use the specified synthetic image as the second synthetic image.

[0134] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0135] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0136] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0137] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for image generation methods based on artificial intelligence. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0138] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions for the artificial intelligence-based image generation method.

[0139] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0140] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the artificial intelligence-based image generation method described above.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0142] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

[0143] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

Claims

1. An artificial intelligence-based image generation method, characterized by, Includes the following steps: Receives an original image generated by a preset image generation model, and a text prompt word input by the user corresponding to the original image; Based on the text prompts, semantic segmentation and element recognition are performed on the original image to obtain the corresponding element recognition results. Based on the element recognition results, layer structure generation processing is performed to obtain the corresponding layer structure data; Based on a preset interactive interface, the system receives editing instructions from the user that correspond to the layer structure data. Based on a preset layered editing engine, the original image is subjected to layer editing operations corresponding to the editing instruction information to obtain the edited image result; Based on a preset intelligent synthesis algorithm, the image results are combined with layers to obtain the corresponding target image; The target image is output and processed based on a preset output format. 2.The AI-based image generation method of claim 1, wherein, The step of performing semantic segmentation and element recognition processing on the original image based on the text prompt words to obtain the corresponding element recognition results specifically includes: The original image is subjected to visual semantic segmentation based on a preset visual segmentation model to obtain the corresponding visual elements. Invoke the preset large language model; Based on the text prompts, the large language model is used to perform semantic association verification and result correction on the visual elements to obtain the corresponding output results. The output result is used as the element identification result. 3.The AI-based image generation method of claim 1, wherein, The step of generating layer structure data based on the element recognition results specifically includes: Based on the element recognition results, layer file generation processing is performed to obtain the corresponding editable layer file; Based on the element recognition results, a spatial relationship mapping table is established to obtain the corresponding spatial relationship mapping table. Based on the element recognition results, layer metadata generation processing is performed to obtain the corresponding layer metadata; The editable layer file, the spatial relationship mapping table, and the layer metadata are integrated to obtain the corresponding integrated data. The integrated data is used as the layer structure data. 4.The AI-based image generation method of claim 1, wherein, The editing instruction information includes layer operation information, text instruction information, and element adjustment information; the step of performing layer editing operations corresponding to the editing instruction information on the original image based on a preset layered editing engine to obtain the edited image result specifically includes: Based on the layered editing engine, layer editing operations corresponding to the layer operation information are performed on the original image to generate a corresponding first image; Perform a layer regeneration operation on the first image corresponding to the text instruction information to generate a corresponding second image; Perform element attribute editing operations corresponding to the element adjustment information on the second image to generate the corresponding third image; The third image is used as the edited image result. 5.The AI-based image generation method of claim 1, wherein, The step of performing layer compositing processing based on a preset intelligent synthesis algorithm and the image result to obtain the corresponding target image specifically includes: Obtain a spatial location mapping table corresponding to the image result; Based on the specified spatial location mapping table, pixel-level compositing processing is performed on all specified layers in the image result to obtain the corresponding preliminary composite image; The preliminary synthesized image is subjected to edge blending processing to obtain the corresponding first synthesized image; The first synthesized image is subjected to color consistency correction processing to obtain the corresponding second synthesized image; The second synthesized image is used as the target image. 6.The AI-based image generation method of claim 5, wherein, The step of performing pixel-level compositing processing on all specified layers in the image result based on the specified spatial location mapping table to obtain the corresponding preliminary composite image specifically includes: Retrieve all specified layers corresponding to the image result; Obtain the layer coordinates and stacking order corresponding to all specified layers from the specified spatial location mapping table; Based on the layer coordinates and the stacking order, pixel overlay processing is performed on all the specified layers in the image result to obtain the corresponding processed image; The processed image is used as the initial synthesized image. 7.The AI-based image generation method of claim 5, wherein, The step of performing color consistency correction processing on the first synthesized image to obtain the corresponding second synthesized image specifically includes: Perform global color analysis on the first synthesized image to obtain the corresponding color distribution information; Based on the color distribution information, the target color parameters in the first synthesized image that meet the adjustment conditions are determined; Obtain the parameter adjustment strategy corresponding to the target color parameter; The target color parameters in the first synthesized image are adjusted based on the parameter adjustment strategy to obtain the adjusted specified synthesized image; The specified composite image is used as the second composite image.

8. An image generation device based on artificial intelligence, characterized in that, include: The first receiving module is used to receive the original image generated by the preset image generation model, and the text prompt words input by the user corresponding to the original image; The processing module is used to perform semantic segmentation and element recognition processing on the original image based on the text prompt words to obtain the corresponding element recognition results; The generation module is used to perform layer structure generation processing based on the element recognition results to obtain the corresponding layer structure data. The second receiving module is used to receive editing instruction information issued by the user corresponding to the layer structure data based on a preset interactive interface; The operation module is used to perform layer editing operations on the original image based on a preset layered editing engine, corresponding to the editing instruction information, to obtain the edited image result; The compositing module is used to perform layer compositing processing based on a preset intelligent compositing algorithm and the image results to obtain the corresponding target image; The output module is used to process the target image based on a preset output format.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the artificial intelligence-based image generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the artificial intelligence-based image generation method as described in any one of claims 1 to 7.

Images