A picture creation processing method and device, a storage medium and an electronic device

Abstract

This specification discloses an image creation processing method, apparatus, storage medium, and electronic device. The method includes: acquiring image search text input by a user; displaying at least one reference search image corresponding to the image search text; determining a target area image selected by the user for the target search image; responding to the user's target image creation operation for the target area image; and performing image creation processing on the target area image using an image creation big model based on the target image creation operation.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and electronic device for image creation and processing. Background Technology

[0002] In internet search scenarios, image search has become an important way for users to obtain information and inspiration. By entering keywords, users can quickly obtain reference images related to their needs. However, in practical applications, after finding the target image, users often need to create a secondary work to meet the needs of personalized expression, commercial design, or content reprocessing. Summary of the Invention

[0003] This specification provides an image creation and processing method, apparatus, storage medium, and electronic device, the technical solutions of which are as follows:

[0004] Firstly, embodiments of this specification provide an image creation and processing method, the method comprising:

[0005] Obtain the image search text input by the user, and display at least one reference search image corresponding to the image search text;

[0006] Determine the target area image selected by the user for the target search image;

[0007] In response to the user's target image creation operation on the target area image, the target area image is processed using a large image creation model based on the target image creation operation.

[0008] In one feasible implementation, determining the target region image selected by the user for the target search image includes:

[0009] In response to the user's region selection operation for the target search image, the target region image is determined based on the region selection operation;

[0010] Based on the target search image, at least one image intelligent creation function is determined, and an intelligent creation interface is displayed. The intelligent creation interface includes the target area image and the at least one image intelligent creation function.

[0011] In one feasible implementation, before determining the target region image based on the region selection operation in response to the user's region selection operation for the target search image, the method further includes:

[0012] Display the screenshot creation mode for the reference search image, determine the user's control trigger operation for the screenshot creation mode, and enable the screenshot creation mode;

[0013] In response to the user's screenshot operation targeting a region of the search image, the image cropping region corresponding to the screenshot operation is determined, and the image cropping region is used as the target region image.

[0014] In one feasible implementation, the image creation operation based on the target image employs a large image creation model to perform image creation processing on the target region image, including:

[0015] Based on the target image creation operation, determine the user creation operation description information for the target area image;

[0016] Based on the user creation operation description information and the target area image, the target area image is processed using an image creation model to obtain the target creation image.

[0017] The target created image is displayed to the user.

[0018] In one feasible implementation, determining the user creation operation description information for the target region image based on the target image creation operation includes:

[0019] Collect user creation behavior data corresponding to the target image creation operation, and extract target image features, user operation features and contextual environment information based on the user creation behavior data;

[0020] User creation operation description information is obtained by performing description transformation processing based on the target image features, user operation features, and contextual environment information.

[0021] In one feasible implementation, the step of processing the target region image using an image creation model based on the user creation operation description information and the target region image to obtain the target created image includes:

[0022] Based on the user creation operation description information and the target area image, the target search image is marked with prompts to obtain a target creation labeled image. The target creation labeled image carries a target area image mark and a creation operation description mark.

[0023] The target image and the user's creation operation description are input into the image creation model. The target image is then processed by the image creation model to obtain the target image.

[0024] In one feasible implementation, the method further includes:

[0025] An initial large-scale image creation model for image creation scenarios is created using a basic large-scale language model.

[0026] Obtain sample images to be created for the image creation scenario, and label the sample images to be created with creation image tags. The sample images to be created include sample search images, sample region images in the sample search images, and sample creation operation description information.

[0027] The initial image creation model is trained using the sample creation image data. During the model training process, the initial image creation model is used to process the images in the sample area to obtain predicted creation images. Based on the predicted creation images and the creation image labels, the model parameters of the initial image creation model are adjusted until the initial image creation model finishes training, thus obtaining the image creation model.

[0028] Secondly, embodiments of this specification provide an image creation and processing apparatus, the apparatus comprising:

[0029] The search module is used to obtain the image search text input by the user and display at least one reference search image corresponding to the image search text;

[0030] The selection module is used to determine the target area image selected by the user for the target search image;

[0031] The creation module is used to respond to the user's target image creation operation on the target area image, and to perform image creation processing on the target area image using a large image creation model based on the target image creation operation.

[0032] In one feasible implementation, the selection module is configured to:

[0033] In response to the user's region selection operation for the target search image, the target region image is determined based on the region selection operation;

[0034] Based on the target search image, at least one image intelligent creation function is determined, and an intelligent creation interface is displayed. The intelligent creation interface includes the target area image and the at least one image intelligent creation function.

[0035] In one feasible implementation, the selection module is configured to:

[0036] Display the screenshot creation mode for the reference search image, determine the user's control trigger operation for the screenshot creation mode, and enable the screenshot creation mode;

[0037] In response to the user's screenshot operation targeting a region of the search image, the image cropping region corresponding to the screenshot operation is determined, and the image cropping region is used as the target region image.

[0038] In one feasible implementation, the creation module is used for:

[0039] Based on the target image creation operation, determine the user creation operation description information for the target area image;

[0040] Based on the user creation operation description information and the target area image, the target area image is processed using an image creation model to obtain the target creation image.

[0041] The target created image is displayed to the user.

[0042] In one feasible implementation, determining the user creation operation description information for the target region image based on the target image creation operation includes:

[0043] Collect user creation behavior data corresponding to the target image creation operation, and extract target image features, user operation features and contextual environment information based on the user creation behavior data;

[0044] User creation operation description information is obtained by performing description transformation processing based on the target image features, user operation features, and contextual environment information.

[0045] In one feasible implementation, the creation module is used for:

[0046] Based on the user creation operation description information and the target area image, the target search image is marked with prompts to obtain a target creation labeled image. The target creation labeled image carries a target area image mark and a creation operation description mark.

[0047] The target image and the user's creation operation description are input into the image creation model. The target image is then processed by the image creation model to obtain the target image.

[0048] In one feasible implementation, the device is further used for:

[0049] An initial large-scale image creation model for image creation scenarios is created using a basic large-scale language model.

[0050] Obtain sample images to be created for the image creation scenario, and label the sample images to be created with creation image tags. The sample images to be created include sample search images, sample region images in the sample search images, and sample creation operation description information.

[0051] The initial image creation model is trained using the sample creation image data. During the model training process, the initial image creation model is used to process the images in the sample area to obtain predicted creation images. Based on the predicted creation images and the creation image labels, the model parameters of the initial image creation model are adjusted until the initial image creation model finishes training, thus obtaining the image creation model.

[0052] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0053] Fourthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0054] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0055] In one or more embodiments of this specification, the electronic device acquires image search text input by the user, displays at least one reference search image corresponding to the image search text, determines the target area image selected by the user for the target search image, and, in response to the user's target image creation operation for the target area image, performs image creation processing on the target area image using a large image creation model based on the target image creation operation. This achieves an integrated process from image search to target area selection to intelligent creation processing, providing users with an efficient and intelligent creation experience. It simplifies the complex process of editing intended images after image search, while improving creation effects and efficiency. It makes the combination of search and creation more efficient and intelligent. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1This is a scene diagram of an image creation and processing system provided in the embodiments of this specification;

[0058] Figure 2 This is a flowchart illustrating an image creation and processing method provided in the embodiments of this specification;

[0059] Figure 3 This is a schematic diagram of a process for determining a target area image provided in the embodiments of this specification;

[0060] Figure 4 This is a schematic diagram of a screenshot creation process provided in the embodiments of this specification;

[0061] Figure 5 This is a schematic diagram of an image creation process provided in the embodiments of this specification;

[0062] Figure 6 This is a flowchart of the model training process for a large-scale image creation model provided in the embodiments of this specification;

[0063] Figure 7 This is a schematic diagram of the structure of an image creation and processing device provided in the embodiments of this specification;

[0064] Figure 8 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification;

[0065] Figure 9 This is a schematic diagram of the operating system and user space structure provided in the embodiments of this specification;

[0066] Figure 10 yes Figure 9 Architecture diagram of the Android operating system in China;

[0067] Figure 11 yes Figure 9 Architecture diagram of the iOS operating system. Detailed Implementation

[0068] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0069] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0070] In related technologies, image search tools (such as search engines) primarily focus on image display and filtering, but lack support for subsequent editing needs. After completing an image search, users must download the image and switch to other image editing tools for processing, resulting in a cumbersome and fragmented workflow. Traditional image editing tools (such as Photoshop and GIMP) are complex and difficult for non-professional users to master, especially when performing high-quality secondary creation on specific areas of the target image. Users need to manually complete steps such as cutout, background replacement, and style adjustment, which is time-consuming and yields unpredictable results. Therefore, current image creation and processing methods suffer from a disconnect between the search and editing processes, and the high barrier to entry for editing tools leads to inconvenient post-search image creation and processing.

[0071] The present specification will now be described in detail with reference to specific embodiments.

[0072] Please see Figure 1 This is a scene illustration of an image creation and processing system provided in an embodiment of this application. Figure 1 As shown, an image creation and processing system can include at least a client cluster and a service platform 100.

[0073] In some embodiments, the client cluster may include at least one client, such as Figure 1 As shown, it specifically includes client 1 corresponding to user 1, client 2 corresponding to user 2, ..., client n corresponding to user n, where n is an integer greater than 0.

[0074] Each client in a client cluster can be an electronic device with communication capabilities, including but not limited to: wearable devices, handheld devices, personal computers, tablets, in-vehicle devices, smartphones, computing devices, or other processing devices connected to a wireless modem. Electronic devices may have different names in different networks, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), and electronic devices in 5G networks or future evolved networks.

[0075] In some embodiments, the service platform 100 may be a single server device, such as a rack-mounted, blade, tower, or cabinet-type server device, or a workstation, mainframe, or other hardware device with strong computing power; or it may be a server cluster composed of multiple servers, wherein the servers in the service cluster may be composed in a symmetrical manner, wherein each server is functionally and hierarchically equivalent in the transaction link, and each server can provide services to the outside world independently, wherein providing services independently can be understood as not requiring the assistance of other servers.

[0076] In one or more embodiments of this specification, the service platform 100 can establish a communication connection with at least one client in the client cluster, and complete the data interaction during the image creation process based on the communication connection.

[0077] For example, service platform 100 provides search services to the outside world. Users can send image search text to service platform 100 through a client. Service platform 100 can obtain the image search text entered by the user, search the image search text to obtain at least one reference search image, display the reference search image on the search display interface, determine the target area image selected by the user for the target search image, and service platform 100 can respond to the user's target image creation operation on the client for the target area image, and perform image creation processing on the target area image based on the target image creation operation using an image creation big model.

[0078] It should be noted that the service platform 100 establishes a communication connection with at least one client in the client cluster via a network for interactive communication. This network can be a wireless network or a wired network. Wireless networks include, but are not limited to, cellular networks, wireless LANs, infrared networks, or Bluetooth networks. Wired networks include, but are not limited to, Ethernet, universal serial bus (USB), or controller area networks. In one or more embodiments of the specification, technologies and / or formats including HyperText Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network (such as target compressed packets). Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0079] The image creation processing system embodiments provided in this specification and the image creation processing methods described in one or more embodiments belong to the same concept. The execution entity corresponding to the image creation processing methods involved in one or more embodiments of this specification can be an electronic device, which can be the aforementioned service platform 100 or the aforementioned client, depending on the actual application environment. The specific implementation process of the demonstration file generation system embodiment can be found in the following method embodiments, and will not be repeated here.

[0080] In one embodiment, such as Figure 2 As shown, a method for image creation and processing is proposed. This method can be implemented using a computer program and can run on an image creation and processing device based on the von Neumann architecture. The computer program can be integrated into an application or run as a standalone utility application. The image creation and processing device can be an electronic device.

[0081] Specifically, the image creation and processing methods include:

[0082] S102: Obtain the image search text input by the user, and display at least one reference search image corresponding to the image search text;

[0083] User-inputted image search text: refers to the keywords or descriptive text that users enter into the search box of a search engine or electronic device to search for images.

[0084] For example, a user enters search keywords via keyboard, such as "high-resolution images of blue skies." The electronic device receives this text input and triggers subsequent search logic through the search engine. The input text can include various expressions, such as keywords ("landscape") and natural language descriptions ("mountains at sunset").

[0085] Furthermore, after obtaining the image search text input by the user, the image search text is processed to obtain at least one reference search image, and the reference search image is displayed on the search display interface.

[0086] Image search processing can be understood as using the correlation between image search text and image content, and through algorithm and database matching, extracting images that meet the search criteria from the image library.

[0087] Reference search images: These are images selected from the search results and displayed to the user.

[0088] Search results page: The page where users browse search results, including a collection of images found in the search.

[0089] For example, electronic devices parse the image search text entered by the user, extract keywords (such as "blue sky"), and use image retrieval technologies (such as text retrieval, visual retrieval, and multimodal retrieval) to search the image database based on the keywords. After the retrieval is completed, at least one reference search image is obtained. The search display interface sorts the image results that meet the conditions and displays them to the user based on relevance or quality.

[0090] S104: Determine the target area image selected by the user for the target search image;

[0091] Target search image: The specific image selected by the user from the reference search image set.

[0092] Target area image: A local area selected by the user in the target search image using a screenshot or selection tool, which serves as the target for subsequent creative processing.

[0093] As an illustration, a user can select a target image from the reference search image set, and then trigger the screenshot function (such as dragging a box with the mouse or defining an area with the touch screen). The electronic device captures a partial area of ​​the image selected by the user, the system records the specific coordinates of the selected area (such as the start and end coordinates of a rectangle), and crops out the corresponding area image as the target area image, displays a preview of the target area image, and allows the user to confirm or readjust the selected area.

[0094] For example, if a user selects a panoramic image of a "sunset beach" as the reference image, and uses the selection tool to select the sun in the center of the image as the target area, the system will crop out the rectangular area containing the sun and display the cropped image as a preview of the target area image.

[0095] S106: In response to the user's target image creation operation on the target area image, the target area image is processed using an image creation big model based on the target image creation operation.

[0096] Target image creation operations: Specific operations performed by the user in the creation interface, such as background replacement, filter application, style transfer, etc.

[0097] Large-scale image creation model: This model is pre-trained and obtained by transferring the basic large-scale language model to image creation scenarios. It can quickly apply the basic large-scale language model to a new image creation field without retraining a new model. Only fine-tuning of the basic large-scale language model is required. The transfer and transformation of the basic large-scale language model (which can be called an LLM model) to the image creation scenario can realize a multimodal large-scale language model (which can be called MLLM) that is compatible with multimodal data such as text and images. This multimodal large-scale language model can perform image creation processing, and the target image creation operation generates model input data. The multimodal large-scale language model obtained by transfer training can be called the large-scale image creation model.

[0098] For example, based on the target area image selected by the user, the AI ​​creation tool interface is displayed, including AI function options such as background removal, background replacement, and style transfer. The system records the user's creative operation data. For example, if the user selects the "background replacement" function and specifies a new background color or image, the system generates creative description information based on the creative operation data: the user's creative operation is converted into structured description information, which is then input into the image creation model. The image creation model generates a new creative image based on the target area image and the creative description information. The electronic device displays the generated target creative image to the user, who can choose to save, edit further, or retry.

[0099] For example, a user selects the "Background Replacement" function in the creation interface of the target area image ("Sunset Sun") and uploads a blue sky background image. The system describes the user's operation as "replace the background of the sunset sun with a blue sky image". The image creation model generates a sun image with a blue sky background and displays it to the user. The user can save or adjust the generated image, such as refining the halo effect of the sun.

[0100] In the embodiments of this specification, the electronic device acquires the image search text input by the user, displays at least one reference search image corresponding to the image search text, determines the target area image selected by the user for the target search image, and responds to the user's target image creation operation for the target area image, using an image creation model to perform image creation processing on the target area image based on the target image creation operation. This achieves an integrated process from image search to target area selection to intelligent creation processing, providing users with an efficient and intelligent creation experience. It simplifies the complex process of editing intended images after image search, while improving the creation effect and efficiency. It makes the combination of search and creation more efficient and intelligent.

[0101] Optional, please see Figure 3 , Figure 3 This is a flowchart illustrating a method for determining a target area image as described in this specification. Specifically, the process of determining the target area image selected by the user for the target search image can be referenced as follows:

[0102] S202: In response to the user's region selection operation for the target search image, determine the target region image based on the region selection operation;

[0103] Region selection operation: Users can select or define local areas of interest on the target search image by using the mouse, touch screen or other tools.

[0104] Target area image: A specific area image cropped from the target search image based on the user's area selection operation.

[0105] In a demonstrative manner, in response to the user's selection of a target search image in the search results, the image is loaded onto the interactive interface. The user then initiates the region selection function (e.g., by clicking the "Select Region" button or directly dragging a selection box). At this point, the user's region selection information is captured in real time, including the coordinates of the start and end points of the selection (e.g., a rectangle) or complex selection paths (e.g., polygon boundaries). For irregular regions (e.g., freehand selections), edge detection or shape fitting is performed to generate a closed region. Based on the user-selected closed region, the corresponding portion of the target search image is cropped to generate the target region image. The cropped result is optimized, for example, by smoothing edges and adjusting transparency to ensure the quality of the region image. Finally, the cropped target region image is displayed, allowing the user to confirm or reselect the region.

[0106] Example: A user selects coconut trees on a beach within a panoramic image of the search result "tropical beach". The system then crops a rectangular area containing the coconut trees as the target area image and displays the cropped result for the user's confirmation.

[0107] In one feasible implementation, such as Figure 4 As shown, Figure 4 This is an illustration of screenshot creation. The process involves responding to the user's region selection operation for a target search image, and determining the target region image based on the region selection operation. The following methods can be used as a reference:

[0108] S3002: Display the screenshot creation mode for the reference search image, determine the user's control trigger operation for the screenshot creation mode, and start the screenshot creation mode;

[0109] Screenshot Creation Mode: A specific interactive mode that allows users to take screenshots of specific areas of a reference image search, used to select a target area of ​​the image.

[0110] Control-triggered actions: The actions taken by users to initiate screenshot creation mode by clicking buttons, dragging tools, or other interactive methods.

[0111] In a demonstrative manner, after a user selects a reference search image, the electronic device loads an interactive interface for that image. The interface displays an entry point control for the screenshot creation mode (such as a "Screenshot Creation" button or tool icon), prompting the user to enable screenshot creation mode. The electronic device monitors the user's actions in real time. When the user clicks the "Screenshot Creation Mode" button or performs a trigger action (such as double-clicking an image or selecting a menu item), the electronic device recognizes the trigger action, responds to the user's trigger action, and switches to screenshot creation mode.

[0112] Optionally, switching to screenshot creation mode will change the mouse pointer displayed on the electronic device to a selection tool shape (such as a cross or rectangle), prompting the user to perform a screenshot operation on the image. The screenshot mode supports multiple selection methods, such as rectangular selection, freehand path selection, polygon selection, etc.

[0113] Example: The user selects a "city night view" image as a reference. After clicking the "Screenshot Creation" button, the system enters screenshot creation mode, and the prompt text at the top of the interface reads: "Please drag the mouse to select the image area."

[0114] S3004: In response to the user's screenshot operation on the target search image, determine the image cropping area corresponding to the screenshot operation, and use the image cropping area as the target area image.

[0115] Screenshotting: Users can select a specific area of ​​interest on an image by dragging the mouse, drawing a path, or using a selection box.

[0116] Image cropping area: A portion of the image cropped based on user actions.

[0117] As an illustration, in screenshot creation mode, when a user drags the mouse or draws a path to select a target area, the system can capture the specific parameters of the area screenshot operation, for example:

[0118] 1) Rectangular selection: Record the coordinates of the starting and ending points.

[0119] 2) Freehand drawing: Record the coordinate sequence of path points to generate a closed region.

[0120] Furthermore, the electronic device crops the corresponding local content from the original image based on the user-selected region coordinates, generating a target region image. The cropped target region image is then processed, such as:

[0121] Smooth edges: Prevents area edges from being too harsh.

[0122] Resolution adjustment: Ensure that the area image is compatible with subsequent creative processing.

[0123] Transparency handling: If the area contains a transparent background, retain the transparency effect.

[0124] Then, the cropped target area image is displayed, allowing the user to confirm or adjust it. After the user confirms, the target area image will be used as input for subsequent creation steps.

[0125] Example: A user selects the lights on the top of a skyscraper from the reference search images for "City Night View". The system records the starting coordinates (100, 150) and ending coordinates (300, 400) of the rectangle selection. The system then crops the corresponding image area, generating an image containing only the target area of ​​the skyscraper's top lights. The cropping result is displayed on the interface, and after user confirmation, the user proceeds to the next step of the creative process.

[0126] In this specification, S3002 and S3004 achieve an efficient connection between the user selecting a reference search image and generating the target area image, achieving the following advantages:

[0127] 1) User experience optimization: Provides an intuitive screenshot creation mode, enabling users to quickly enter the area selection operation without complicated settings.

[0128] 2) Precise region capture: Based on the user's region selection operation, the system can accurately crop the target region image to ensure the integrity and clarity of the local content.

[0129] 3) Operational flexibility: Supports multiple area selection methods (rectangular selection, freehand drawing, polygonal path, etc.) to meet different creative needs.

[0130] 4) Efficiently connects to subsequent creation: The cropped target area image can be directly used for intelligent creation processing without the need for additional adjustments or editing by the user.

[0131] The S3002 and S3004 workflow designs significantly improve the convenience and efficiency of image creation, providing user-friendly support for quickly capturing and creating images in search scenarios.

[0132] S204: Based on the target search image, determine at least one image intelligent creation function, and display the intelligent creation interface, which includes the target area image and the at least one image intelligent creation function.

[0133] Intelligent image creation function: AI-assisted creation tools based on target area images, including AI background removal, AI background replacement, AI filter addition, AI style transfer and other functions.

[0134] Intelligent Creation Interface: An interactive interface that integrates the target area image and intelligent creation functions, used for user operation and preview of creation results.

[0135] In one feasible implementation, the electronic device analyzes the content characteristics (such as scene, color, and texture) of the target search image and the user's region selection operation to obtain image content and user demand information. Based on the image content and user demand, it dynamically recommends intelligent creation functions. For example:

[0136] If the image in the area depicts people, we recommend using "AI cutout" and "AI background blur".

[0137] If the area image is a landscape, we recommend "AI Style Transfer" and "Color Enhancement".

[0138] Then, the electronic device loads a smart creation interface, which includes, but is not limited to:

[0139] 1) Target area image: The image of the area cropped by the user is displayed in the main position for intuitive operation by the user.

[0140] 2) Smart Creation Function Menu: Lists recommended creation function buttons (such as background removal, filters, background replacement, etc.).

[0141] Furthermore, electronic devices can provide user operation guidance, that is, through text prompts or animations, to help users understand how to use the creation functions and provide options for adjusting operation parameters (such as filter intensity and background color selection). After the user selects the intelligent image creation function, the system calls the large image creation model based on the intelligent image creation function to generate results in real time and previews them on the interface. Subsequently, the user can select other creation functions or combinations of functions (such as applying filters and background replacement simultaneously).

[0142] Example: The user selects coconut trees as the target area in the image. Recommended intelligent content creation features for electronic devices include:

[0143] Background removal: Remove the background and keep only the coconut tree.

[0144] Background replacement: Replace with a blue sky and white clouds background.

[0145] Filter application: Enhance the color saturation of the coconut tree and highlight details.

[0146] The user selects "Background Replacement" as the target intelligent creation function and uploads a new background image (such as a sunset beach). The system then uses a large model to generate the replaced image and provides a real-time preview.

[0147] In the embodiments described in this specification, precise region selection and cropping functions help users focus on local details of the target image. Intelligent function recommendations and an intuitive creation interface design lower the user's operational threshold and provide diverse personalized creation tools. The entire process is efficient and flexible, making it particularly suitable for scenarios involving localized creative processing of images, such as advertising design, content creation, and e-commerce displays.

[0148] In one feasible implementation, such as Figure 5 As shown, Figure 5 This is a schematic diagram of an image creation process. The image creation operation based on the target image uses a large image creation model to perform image creation processing on the target area image. This can be done in the following ways:

[0149] S4002: Determine user creation operation description information for the target area image based on the target image creation operation;

[0150] Target image creation operations: Specific operations that users perform on the target area of ​​the image in the creation interface, such as selecting functions (cutout, background replacement, etc.), adjusting parameters (brightness, contrast, etc.), or entering semantic descriptions ("make the background more dreamy").

[0151] User creation operation description information: Convert user operation requirements into structured description information for the image creation model to understand and execute.

[0152] This method illustratively captures the user's image creation actions and records their operational data within the creation interface. This includes: function selection data (e.g., selecting the "background replacement" function), parameter setting data (e.g., setting the background color to "blue sky"), and natural language input data (e.g., the user inputs "make the overall style fresher"). Simultaneously, it detects the sequence and related functions of the user's actions. Then, it extracts descriptive features of the creation actions from the recorded data, including key features of the user's actions (such as operation type, parameter values, and semantic information) and creative requirement features related to the natural language input. Finally, it converts these descriptive features into structured descriptive information to obtain the user's creative action description information.

[0153] Example: A user selects the "Background Replacement" function for the target area image "Coconut Trees", uploads a background image of "Blue Sky and White Clouds", and sets the edge softening level to 5. The system generates the operation description information as follows: Replace the background of the coconut trees with blue sky, keeping the edges smooth transition.

[0154] In one feasible implementation, determining the user creation operation description information for the target region image based on the target image creation operation can be done in the following way:

[0155] A2: Collect user creation operation behavior data corresponding to the target image creation operation, and extract target image features, user operation features and contextual environment information based on the user creation operation behavior data;

[0156] User-generated action behavior data: Records the actions performed by users in the interface, including clicks, swipes, drags, inputs, and other actions and their parameters.

[0157] Target image features: Content features of the target region image, such as color distribution, texture pattern, object edge information, etc.

[0158] User operation characteristics: user operation trajectory, function selection and parameter settings, such as drag path and adjusted value range.

[0159] Contextual information: Background information of the target area image in the overall creation task, such as image type (people, landscape), overall style (simple, complex) and the user's previous operation records.

[0160] This is illustrative of real-time data collection of user creation behavior on the creation interface. For example:

[0161] The user clicked the "Background Replacement" function and selected a blue gradient background; the user adjusted the background transparency to 50%. The system records the time, location, and parameter information of each action, generating user creative operation data.

[0162] Furthermore, by extracting target image features from user creation behavior data, computer vision technology can be used to analyze the content of the target area image and extract target image features. Target image features include, but are not limited to, the main color tone of the image, texture complexity, the contour information of object edges, and object detection technology to identify the semantic content of the image (such as people, animals, and landscapes) contained in the target area image.

[0163] Furthermore, analyzing user creation behavior data to identify user action patterns and intentions can reveal user action characteristics. For example:

[0164] Detecting whether a user continuously adjusts specific parameters (such as brightness and contrast) may be an attempt to enhance image clarity.

[0165] If the area the user is dragging is small, further refinement may be required (such as edge sharpening or fine cutout).

[0166] Furthermore, extracting contextual information from user creation behavior data can be achieved by combining the position of the target area image within the original search images and its surrounding environment. For example, this can help determine if the target area image matches the overall image style and whether the user has previously made adjustments to the target area image (e.g., applying a filter).

[0167] For example, the user selected the "background replacement" operation on the image of the target area "coconut tree" and set the background to "gradient blue sky".

[0168] The data collected by the system includes:

[0169] User behavior data: The function selection is "Background Replacement", and the parameters are "Background Color: Gradient Blue Sky, Transparency: 50%".

[0170] Target image features: Extract the edge contour, main color tone (green), and texture characteristics (leaf details) of the coconut tree.

[0171] Contextual information: The coconut tree area is based on a panoramic view of a "tropical beach", with an overall style that leans towards nature and sunshine.

[0172] A4: Based on the target image features, user operation features, and contextual information, a description transformation process is performed to obtain user creation operation description information.

[0173] Description transformation processing: The collected feature data is transformed into structured descriptive information that can be understood by large-scale image creation models, including clear operational objectives and parameter requirements.

[0174] User creation operation description information: a structured or semantic description that clearly defines the specific content and intent of the creation operation for the model to execute and process.

[0175] Intuitively, target image features, user operation features, and contextual information are integrated into a logically clear feature set. Then, description generation processing is performed based on this feature set to generate descriptive information. For example, the feature set indicates that if the user selects the background replacement function, the descriptive information should include the boundary information of the target area, the background replacement content, and transition parameters. Natural language generation techniques (such as GPT) or rule-based algorithms are used to transform the feature data into a semantic description, and the semantic description is then converted into a machine-understandable structured data format (such as JSON) to ensure that the large-scale image creation model can parse it.

[0176] For example, the above method can accurately collect user operation behavior data, comprehensively extract target image features, user operation features, and contextual environment information, and generate machine-understandable user creation operation descriptions through descriptive transformation processing. This process achieves efficient transformation from user needs to model input, ensuring accurate expression of creative needs, while improving the adaptability of the large image creation model to complex tasks. The overall method supports personalized, high-quality creative result output, significantly reduces the user's operational threshold, and improves creative efficiency and experience.

[0177] S4004: Based on the user creation operation description information and the target area image, the target area image is processed using an image creation model to obtain the target creation image;

[0178] Target Image: The final image result generated after processing the large image creation model, which includes the effect of realizing the user's creative requirements.

[0179] In illustrative terms, user-generated operation description information and target area images are used as input to generate data in a format acceptable to the large model. Optional input data for the large model includes: original target area images (JPEG, PNG, etc.) and operation description information (structured text or JSON).

[0180] Furthermore, based on the input description information and the target area image, the image creation model first identifies the content features of the target area image (such as edges, textures, and colors), then understands the user's creative needs (such as background replacement and style adjustment), calls the corresponding generation or editing technology tools, generates the target creation image that meets the needs, and displays the generated target creation image to the user. The user can choose to save the image, continue to adjust, or re-execute the creation operation.

[0181] Example: The user's target area image is "coconut tree", and the description requires the background to be replaced with a blue sky with smooth edges.

[0182] The image creation model performs the following tasks based on the input:

[0183] 1) Extract the coconut tree area.

[0184] 2) Replace the background with the uploaded blue sky image.

[0185] 3) Soften the edges of the coconut tree to ensure it blends naturally with the new background.

[0186] After the large-scale image creation model completes the task processing, it generates the target image. Users can see the effect of a coconut tree placed under a blue sky and white clouds, and can further fine-tune or save it.

[0187] In one feasible implementation, the step of processing the target region image using an image creation model based on the user creation operation description information and the target region image to obtain the target created image can be carried out in the following way:

[0188] B2: Based on the user creation operation description information and the target area image, the target search image is marked with prompts to obtain a target creation labeled image. The target creation labeled image carries a target area image mark and a creation operation description mark.

[0189] Target creation annotation images: These are images that clearly indicate the location of the target area and the creation operation requirements on the target search image, guiding the generation and processing of the large model for image creation.

[0190] Target area image marking: Visually mark the target area image selected by the user (such as rectangles or outlines) to clearly indicate the location of the target area.

[0191] Creation operation description tag: Based on the user's creation operation description information, the creation requirements are transformed into visual or text tags (such as annotations and labels) and attached to the target creation annotation image.

[0192] In a schematic way, the location of the target area image selected by the user is highlighted on the target search image in the form of a mark, and a creation operation description mark is generated at the same time. Finally, the target area image mark and the creation operation description mark are integrated into the target search image to generate a target creation annotation image, so as to synthesize the target creation annotation image.

[0193] Marking methods can include: Rectangular border: Enclosing the target area with a border; Edge highlighting: Highlighting the outline of the target area with color; Masking: Covering the target area with a semi-transparent color to emphasize its uniqueness.

[0194] Creative operation description tags can include: attaching text or graphic tags to the target search image based on the user's creative operation description information, describing the content and requirements of the creative operation. For example, adding a note near the target area: "Replace the background with a gradient blue sky, 50% transparency." Using icons to represent specific operations (such as "filter" or "cutout").

[0195] Example: A user selects the search image "city night view" and chooses a tall building in the center of the image as the target area. The user's action is "replace background with blue sky, transparency 50%". The system generates a target image with annotations: the tall building area is marked with a rectangle. An annotation is added outside the rectangle: "Background replacement: blue sky, transparency 50%". The entire target search image has highlighted areas and annotation information, used as a reference for subsequent models.

[0196] B4: Input the target creation annotation image and the user creation operation description information into the image creation model, and use the image creation model to perform image creation processing on the target area image to obtain the target creation image.

[0197] Target Image: The final image output from the large-scale image creation model, including the editing effects and creation requirements for the target area image.

[0198] The target creation labeled image and the user creation operation description information are integrated into the model input data, and then input into the image creation big model for parsing: target creation labeled image (which can provide the location of the target area and preliminary visual cues) and user creation operation description information (which clarifies the creation needs through structured data (such as JSON) or semantic text).

[0199] The image creation model analyzes the input data to identify target region markers and creation description markers in the target image. Combined with user creation operation descriptions, it understands the specific creation task and then uses task-related generation techniques (such as background replacement, style transfer, and effect addition) to edit the target region image.

[0200] After editing, the target image is generated and output. The electronic device displays the target image and allows users to view, save, or adjust the creation results.

[0201] This manual demonstrates how the above method clearly expresses user creative requirements in the form of target creation annotation images and operation descriptions. This provides clear visual and semantic input for the large-scale image creation model, significantly improving the accuracy and efficiency of the creation task. The target creation annotation images intuitively mark the target area and operation requirements, facilitating precise model positioning and execution. The description information further supplements the details of complex requirements, ensuring that the generated results meet user expectations. This overall process not only improves creation quality and user experience but also reduces repeated adjustments caused by unclear requirements, providing strong technical support for rapid and efficient image creation.

[0202] S4006: Display the target created image to the user.

[0203] In one or more embodiments of this specification, the execution flow of S4002-S4006 automates the process from user creation requirements to final image generation, transforming complex user creation operations into standardized descriptive information to ensure the large model accurately understands user intent. Leveraging the powerful capabilities of the image creation large model, it enables rapid and intelligent creation based on user needs, generating natural and high-quality target images. Users do not need complex editing skills; simple operations are sufficient to complete personalized creation tasks, lowering the barrier to entry. Real-time preview of creation results allows users to further fine-tune creation parameters or try other styles, enhancing user experience and satisfaction. The entire process is efficient and intelligent, particularly suitable for scenarios requiring rapid generation of personalized image creation results, such as social media content creation, advertising design, and visual effects production.

[0204] Optionally, the following illustrates the model training process for a large-scale image creation model, such as... Figure 5 As shown, Figure 5 This is a flowchart of the model training process for a large-scale image creation model, specifically:

[0205] S5002: Uses a basic large language model to create an initial large model for image creation scenarios;

[0206] Pre-acquire basic large language models, such as the Tongyi Qianwen Large Model, Vicuna-7B Model, ChatGPT Large Model, Wenxin Yiyan Large Model, and Angedis Large Model.

[0207] Then, determine the image creation module for the image creation scenario and the large language generative module corresponding to the basic large language model, and create an initial image creation large model that includes at least the image creation module and the large language generative module;

[0208] S5004: Obtain sample image data to be created for the image creation scenario, and label the sample image data to be created with creation image tags. The sample image data to be created includes sample search images, sample region images in the sample search images, and sample creation operation description information.

[0209] Sample image data acquisition: Acquire a large sample image data to be created. The sample data includes sample search images, sample region images in the sample search images, and sample creation operation description information.

[0210] Sample data annotation: Based on the needs of the image creation scenario, an expert service is introduced to manually annotate the sample images to be created with corresponding creation image tags;

[0211] S5006: The initial image creation model is trained using the sample creation image data. During the model training process, the initial image creation model is used to perform image creation processing on the sample area images to obtain predicted creation images. Based on the predicted creation images and the creation image labels, the model parameters of the initial image creation model are adjusted until the initial image creation model finishes model training, thus obtaining the image creation model.

[0212] Model training process: Input sample data into the initial image creation model for at least one round of model training. Use the initial image creation model to process the sample area images to obtain predicted creation images. Determine the model loss value based on the predicted creation images and the creation image labels. Adjust the model parameters of the initial image creation model based on the model loss value until the model training termination condition is met to obtain the image creation model.

[0213] Optionally, during subsequent model training, the model loss value is determined using a model loss function based on the predicted created image and the created image label. The model recognition loss is used to adjust the model parameters only for the image creation module in the initial image creation model, while keeping the model module parameters of the large language generative module unchanged.

[0214] In one or more embodiments of this specification, the image creation model trained in the above manner realizes an integrated process from image search to target region selection and intelligent creation processing, providing users with an efficient and intelligent creation experience. It simplifies the complex process of editing intended images after image search, while improving creation effects and efficiency. It makes the combination of search and creation more efficient and intelligent.

[0215] The following will combine Figure 7This specification provides a detailed description of the image creation and processing apparatus provided in the embodiments. It should be noted that... Figure 7 The image creation and processing device shown is used to execute this instruction manual. Figures 1-6 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-6 The example shown.

[0216] Please see Figure 7 This diagram illustrates the structure of an image creation processing device according to an embodiment of this specification. The image creation processing device 1 can be implemented as all or part of a user terminal through software, hardware, or a combination of both. According to some embodiments, the image creation processing device 1 includes a search module 11, a selection module 12, and a creation module 13, specifically used for:

[0217] The search module 11 is used to obtain the image search text input by the user and display at least one reference search image corresponding to the image search text;

[0218] Selection module 12 is used to determine the target area image selected by the user for the target search image;

[0219] The creation module 13 is used to respond to the user's target image creation operation on the target area image, and to perform image creation processing on the target area image using an image creation big model based on the target image creation operation.

[0220] In one feasible implementation, the selection module 12 is used to:

[0221] In response to the user's region selection operation for the target search image, the target region image is determined based on the region selection operation;

[0222] Based on the target search image, at least one image intelligent creation function is determined, and an intelligent creation interface is displayed. The intelligent creation interface includes the target area image and the at least one image intelligent creation function.

[0223] In one feasible implementation, the selection module 12 is used to:

[0224] Display the screenshot creation mode for the reference search image, determine the user's control trigger operation for the screenshot creation mode, and enable the screenshot creation mode;

[0225] In response to the user's screenshot operation targeting a region of the search image, the image cropping region corresponding to the screenshot operation is determined, and the image cropping region is used as the target region image.

[0226] In one feasible implementation, the creation module 13 is used for:

[0227] Based on the target image creation operation, determine the user creation operation description information for the target area image;

[0228] Based on the user creation operation description information and the target area image, the target area image is processed using an image creation model to obtain the target creation image.

[0229] The target created image is displayed to the user.

[0230] In one feasible implementation, determining the user creation operation description information for the target region image based on the target image creation operation includes:

[0231] Collect user creation behavior data corresponding to the target image creation operation, and extract target image features, user operation features and contextual environment information based on the user creation behavior data;

[0232] User creation operation description information is obtained by performing description transformation processing based on the target image features, user operation features, and contextual environment information.

[0233] In one feasible implementation, the creation module 13 is used for:

[0234] Based on the user creation operation description information and the target area image, the target search image is marked with prompts to obtain a target creation labeled image. The target creation labeled image carries a target area image mark and a creation operation description mark.

[0235] The target image and the user's creation operation description are input into the image creation model. The target image is then processed by the image creation model to obtain the target image.

[0236] In one feasible implementation, the device 1 is further used for:

[0237] An initial large-scale image creation model for image creation scenarios is created using a basic large-scale language model.

[0238] Obtain sample images to be created for the image creation scenario, and label the sample images to be created with creation image tags. The sample images to be created include sample search images, sample region images in the sample search images, and sample creation operation description information.

[0239] The initial image creation model is trained using the sample creation image data. During the model training process, the initial image creation model is used to process the images in the sample area to obtain predicted creation images. Based on the predicted creation images and the creation image labels, the model parameters of the initial image creation model are adjusted until the initial image creation model finishes training, thus obtaining the image creation model.

[0240] It should be noted that the image creation processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when executing the image creation processing method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image creation processing apparatus and the image creation processing method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0241] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.

[0242] In the embodiments of this specification, the electronic device acquires the image search text input by the user, displays at least one reference search image corresponding to the image search text, determines the target area image selected by the user for the target search image, and responds to the user's target image creation operation for the target area image, using an image creation model to perform image creation processing on the target area image based on the target image creation operation. This achieves an integrated process from image search to target area selection to intelligent creation processing, providing users with an efficient and intelligent creation experience. It simplifies the complex process of editing intended images after image search, while improving the creation effect and efficiency. It makes the combination of search and creation more efficient and intelligent.

[0243] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-6 The image creation processing method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-6 The specific details of the illustrated embodiments will not be elaborated here.

[0244] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1-6 The image creation processing method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-6 The specific details of the illustrated embodiments will not be elaborated here.

[0245] Please refer to Figure 8 This diagram illustrates a structural block diagram of an electronic device provided in an exemplary embodiment of this specification. The electronic device in this specification may include one or more components such as a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 may be connected via the bus 150.

[0246] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the electronic device via various interfaces and lines, and performs various functions and processes data of electronic device 100 by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0247] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include a non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 120 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described below, etc. The operating system may be the Android system, including systems deeply developed based on the Android system, the iOS system developed by Apple Inc., including systems deeply developed based on the iOS system, or other systems. The data storage area may also store data created by the electronic device during use, such as phonebook data, audio and video data, chat log data, etc.

[0248] See Figure 9 As shown, the memory 120 can be divided into operating system space and user space. The operating system runs in the operating system space, while native and third-party applications run in the user space. To ensure that different third-party applications can achieve good running performance, the operating system allocates corresponding system resources for each application. However, different application scenarios within the same third-party application have different requirements for system resources. For example, in local resource loading scenarios, third-party applications have high requirements for disk read speed; in animation rendering scenarios, third-party applications have high requirements for GPU performance. Since the operating system and third-party applications are independent of each other, the operating system often cannot promptly perceive the current application scenario of a third-party application, resulting in the operating system's inability to adapt system resources accordingly to the specific application scenario of the third-party application.

[0249] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to establish data communication between the third-party applications and the operating system. This would allow the operating system to obtain the current scenario information of the third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0250] Taking the Android operating system as an example, the programs and data stored in memory 120 are as follows: Figure 10As shown, the memory 120 can store the Linux kernel layer 320, the system runtime library layer 340, the application framework layer 360, and the application layer 380. The Linux kernel layer 320, system runtime library layer 340, and application framework layer 360 belong to the operating system space, while the application layer 380 belongs to the user space. The Linux kernel layer 320 provides low-level drivers for various hardware components of the electronic device, such as display drivers, audio drivers, camera drivers, Bluetooth drivers, Wi-Fi drivers, and power management. The system runtime library layer 340 provides support for key features of the Android system through several C / C++ libraries. For example, the SQLite library provides database support, the OpenGL / ES library provides 3D graphics support, and the Webkit library provides browser kernel support. The system runtime library layer 340 also provides the Android runtime library, which mainly provides core libraries that allow developers to write Android applications using the Java language. The Application Framework Layer 360 provides various APIs that may be used when building applications. Developers can also use these APIs to build their own applications, such as activity management, window management, view management, notification management, content provider, package management, call management, resource management, and location management. At least one application runs in the Application Layer 380. These applications can be native applications that come with the operating system, such as contacts, SMS, clock, and camera apps; or third-party applications developed by third-party developers, such as games, instant messaging, and photo editing apps.

[0251] Taking the operating system as an example (iOS), the programs and data stored in memory 120 are as follows: Figure 11As shown, the iOS system includes: Core OS layer 420, Core Services layer 440, Media layer 460, and Cocoa Touch layer 480. Core OS layer 420 includes the operating system kernel, drivers, and low-level program frameworks. These low-level program frameworks provide hardware-level functionality for use by the program frameworks located in Core Services layer 440. Core Services layer 440 provides system services and / or program frameworks required by applications, such as Foundation framework, account framework, advertising framework, data storage framework, network connectivity framework, geolocation framework, motion framework, etc. Media layer 460 provides applications with audiovisual interfaces, such as interfaces related to graphics and images, audio technology, video technology, and AirPlay (wireless playback of audio and video transmission technologies). Cocoa Touch layer 480 provides various commonly used interface-related frameworks for application development and is responsible for user touch interaction on electronic devices. Examples include local notification services, remote push services, advertising frameworks, game tool frameworks, message user interface (UI) frameworks, UIKit frameworks, map frameworks, and so on.

[0252] exist Figure 11 The framework shown includes, but is not limited to, the base framework in the core service layer 440 and the UIKit framework in the touchable layer 480. The base framework provides many basic object classes and data types, offering the most basic system services to all applications, and is independent of the UI. The UIKit framework, on the other hand, provides a basic UI class library for creating touch-based user interfaces. iOS applications can use the UIKit framework to provide their UI, thus providing the application's infrastructure for building user interfaces, drawing, handling user interaction events, responding to gestures, and so on.

[0253] The methods and principles for implementing data communication between third-party applications and the operating system in the iOS system can be found in the Android system, and will not be repeated here.

[0254] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined into a touch screen, which is used to receive touch operations from the user using a finger, stylus, or any suitable object on or near it, and to display the user interface of various applications. The touch screen is usually located on the front panel of the electronic device. The touch screen can be designed as a full-screen, curved screen, or irregularly shaped screen. The touch screen can also be designed as a combination of a full-screen and a curved screen, or a combination of an irregularly shaped screen and a curved screen; this specification does not limit this aspect.

[0255] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WiFi) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0256] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0257] The electronic device described in this specification can also be equipped with a display device. This display device can be any device capable of displaying information, such as a cathode ray tube display (CR), a light-emitting diode display (LED), an e-ink screen, a liquid crystal display (LCD), or a plasma display panel (PDP). Users can use the display device on electronic device 101 to view displayed text, images, videos, and other information. The electronic device can be a smartphone, tablet computer, gaming device, AR (Augmented Reality) device, automobile, data storage device, audio playback device, video playback device, laptop, desktop computing device, or wearable device such as an electronic watch, electronic glasses, electronic helmet, electronic bracelet, electronic necklace, or electronic clothing.

[0258] exist Figure 8 In the illustrated electronic device, the processor 110 can be used to call the application program stored in the memory 120 and specifically perform the following operations:

[0259] Obtain the image search text input by the user, and display at least one reference search image corresponding to the image search text;

[0260] Determine the target area image selected by the user for the target search image;

[0261] In response to the user's target image creation operation on the target area image, the target area image is processed using a large image creation model based on the target image creation operation.

[0262] In one embodiment, when the processor 110 determines the target region image selected by the user for the target search image, it performs the following operations:

[0263] In response to the user's region selection operation for the target search image, the target region image is determined based on the region selection operation;

[0264] Based on the target search image, at least one image intelligent creation function is determined, and an intelligent creation interface is displayed. The intelligent creation interface includes the target area image and the at least one image intelligent creation function.

[0265] In one embodiment, when the processor 110 performs the region selection operation in response to the user's search for a target image, and determines the target region image based on the region selection operation, it performs the following operations:

[0266] Display the screenshot creation mode for the reference search image, determine the user's control trigger operation for the screenshot creation mode, and enable the screenshot creation mode;

[0267] In response to the user's screenshot operation targeting a region of the search image, the image cropping region corresponding to the screenshot operation is determined, and the image cropping region is used as the target region image.

[0268] In one embodiment, when the processor 110 performs the image creation operation based on the target image and uses an image creation model to process the target area image, it performs the following operations:

[0269] Based on the target image creation operation, determine the user creation operation description information for the target area image;

[0270] Based on the user creation operation description information and the target area image, the target area image is processed using an image creation model to obtain the target creation image.

[0271] The target created image is displayed to the user.

[0272] In one embodiment, when the processor 110 performs the user creation operation description information for the target region image determined based on the target image creation operation, it performs the following operations:

[0273] Collect user creation behavior data corresponding to the target image creation operation, and extract target image features, user operation features and contextual environment information based on the user creation behavior data;

[0274] User creation operation description information is obtained by performing description transformation processing based on the target image features, user operation features, and contextual environment information.

[0275] In one embodiment, the processor 110 performs image creation processing on the target area image based on the user creation operation description information and the target area image using an image creation big model to obtain a target created image, including:

[0276] Based on the user creation operation description information and the target area image, the target search image is marked with prompts to obtain a target creation labeled image. The target creation labeled image carries a target area image mark and a creation operation description mark.

[0277] The target image and the user's creation operation description are input into the image creation model. The target image is then processed by the image creation model to obtain the target image.

[0278] In one embodiment, the processor 110, while executing the image creation processing method, also performs the following operations:

[0279] An initial large-scale image creation model for image creation scenarios is created using a basic large-scale language model.

[0280] Obtain sample images to be created for the image creation scenario, and label the sample images to be created with creation image tags. The sample images to be created include sample search images, sample region images in the sample search images, and sample creation operation description information.

[0281] The initial image creation model is trained using the sample creation image data. During the model training process, the initial image creation model is used to process the images in the sample area to obtain predicted creation images. Based on the predicted creation images and the creation image labels, the model parameters of the initial image creation model are adjusted until the initial image creation model finishes training, thus obtaining the image creation model.

[0282] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0283] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A method for image creation and processing, characterized in that, The method includes: Obtain the image search text input by the user, and display at least one reference search image corresponding to the image search text; Determine the target area image selected by the user for the target search image; In response to the user's target image creation operation on the target area image, the target area image is processed using a large image creation model based on the target image creation operation.

2. The method according to claim 1, characterized in that, Determining the target region image selected by the user for the target search image includes: In response to the user's region selection operation for the target search image, the target region image is determined based on the region selection operation; Based on the target search image, at least one image intelligent creation function is determined, and an intelligent creation interface is displayed. The intelligent creation interface includes the target area image and the at least one image intelligent creation function.

3. The method according to claim 2, characterized in that, The step of responding to the user's region selection operation for the target search image and determining the target region image based on the region selection operation includes: Display the screenshot creation mode for the reference search image, determine the user's control trigger operation for the screenshot creation mode, and enable the screenshot creation mode; In response to the user's screenshot operation targeting a region of the search image, the image cropping region corresponding to the screenshot operation is determined, and the image cropping region is used as the target region image.

4. The method according to claim 1, characterized in that, The image creation operation based on the target image uses a large image creation model to perform image creation processing on the target area image, including: Based on the target image creation operation, determine the user creation operation description information for the target area image; Based on the user creation operation description information and the target area image, the target area image is processed using an image creation model to obtain the target creation image. The target created image is displayed to the user.

5. The method according to claim 4, characterized in that, The step of determining the user creation operation description information for the target area image based on the target image creation operation includes: Collect user creation behavior data corresponding to the target image creation operation, and extract target image features, user operation features and contextual environment information based on the user creation behavior data; User creation operation description information is obtained by performing description transformation processing based on the target image features, user operation features, and contextual environment information.

6. The method according to claim 4, characterized in that, The step of processing the target region image using an image creation model based on the user creation operation description information and the target region image to obtain the target created image includes: Based on the user creation operation description information and the target area image, the target search image is marked with prompts to obtain a target creation labeled image. The target creation labeled image carries a target area image mark and a creation operation description mark. The target image and the user's creation operation description are input into the image creation model. The target image is then processed by the image creation model to obtain the target image.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: An initial large-scale image creation model for image creation scenarios is created using a basic large-scale language model. Obtain sample images to be created for the image creation scenario, and label the sample images to be created with creation image tags. The sample images to be created include sample search images, sample region images in the sample search images, and sample creation operation description information. The initial image creation model is trained using the sample creation image data. During the model training process, the initial image creation model is used to process the images in the sample area to obtain predicted creation images. Based on the predicted creation images and the creation image labels, the model parameters of the initial image creation model are adjusted until the initial image creation model finishes training, thus obtaining the image creation model.

8. An image creation and processing device, characterized in that, The device includes: The search module is used to obtain the image search text input by the user and display at least one reference search image corresponding to the image search text; The selection module is used to determine the target area image selected by the user for the target search image; The creation module is used to respond to the user's target image creation operation on the target area image, and to perform image creation processing on the target area image using a large image creation model based on the target image creation operation.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, which are adapted to be loaded by a processor and executed as method steps as claimed in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed the method steps as claimed in any one of claims 1 to 7.

Images