Image processing method and system, electronic equipment, storage medium and program product

Abstract

The invention provides an image processing method and system, electronic equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence, and the method comprises the steps: inputting user demand information related to a to-be-processed target image into a first preset model, and obtaining at least one first parameter, the at least one first parameter comprises description information of a target object in the target image; generating a segmentation mask corresponding to the target object based on the description information of the target object; based on the segmentation mask, generating an alpha channel corresponding to the target object in the target image; processing a background region in the target image according to the alpha channel and the depth information of the target object to obtain a target background; and synthesizing the target object and the target background to obtain a processed target image.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image processing method, system, electronic device, storage medium and program product. Background Art

[0002] Image editing and enhancement is the process of processing images through various technical means to improve the quality, appearance or meet specific needs of the image. In related technologies, users are usually required to manually select the area to be edited, select the editing or enhancement type, or the software automatically or semi-automatically creates a mask layer to separate the content to be edited. However, the user interface of traditional image processing is relatively complex, and users usually need professional knowledge to effectively operate the selection area, adjust parameters and apply tools. At the same time, if automatic or semi-automatic software is used, it is difficult to adjust the editing and enhancement effects according to the specific needs of the user. It can be seen that the technologies related to image editing and enhancement have the problems of high difficulty for users and low intelligence. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, system, electronic device, storage medium and program product to solve the problems of high user difficulty and low intelligence level existing in the technologies related to image editing and enhancement.

[0004] To solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of the present application provides an image processing method, the method specifically comprising:

[0006] Inputting user demand information related to the target image to be processed into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in the target image;

[0007] Based on the description information of the target object, generating a segmentation mask corresponding to the target object;

[0008] Based on the segmentation mask, generating an alpha channel corresponding to the target object in the target image;

[0009] Processing a background area in the target image according to the alpha channel and the depth information of the target object to obtain a target background;

[0010] The target object and the target background are synthesized to obtain a processed target image.

[0011] Optionally, inputting user demand information related to the target image to be processed into a first preset model to obtain at least one first parameter includes:

[0012] Acquire the user demand information uploaded by the user through the user interface;

[0013] Inputting the user demand information into the first preset model to obtain the at least one first parameter;

[0014] Among them, the first preset model is a large language model trained based on user demand prompts.

[0015] Optionally, the generating a segmentation mask corresponding to the target object based on the description information of the target object includes:

[0016] Inputting the description information of the target object into a second preset model to obtain the position information of the target object in the target image;

[0017] The position information is input into a third preset model to obtain the segmentation mask.

[0018] Optionally, the processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background includes:

[0019] Determining depth information of the target image based on the target object;

[0020] Processing the background area in the target image by a preset algorithm according to the depth information and the alpha channel to obtain the target background;

[0021] Wherein, the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter.

[0022] Optionally, the method further comprises:

[0023] When receiving user feedback information on the processed target image, input the feedback information into the first preset model to obtain at least one second parameter, wherein the at least one second parameter includes secondary description information of the target object;

[0024] Execute a target operation based on the secondary description information;

[0025] Among them, the target operation includes generating a segmentation mask corresponding to the target object based on the secondary description information, generating an alpha channel corresponding to the target object in the target image based on the segmentation mask, processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background, synthesizing the target object and the target background to obtain a processed target image.

[0026] In a second aspect, the embodiment of the present application further provides an image processing system, including:

[0027] A language dialogue agent, used for obtaining user demand information uploaded by a user through a user interface, and inputting the user demand information into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in a target image to be processed;

[0028] A mask and cutout agent, comprising a second preset model module, a visual segmentation module and a mask and cutout module, wherein the second preset model module is used to input the description information of the target object into the second preset model to obtain the position information of the target object in the target image, the visual segmentation module is used to input the description information of the target object into the second preset model to obtain the position information of the target object in the target image, and the mask and cutout module is used to input the position information into a third preset model to obtain a segmentation mask;

[0029] a post-processing agent, comprising an image processing module and an image synthesis module, wherein the image processing module is used to process the background area in the target image by a preset algorithm to obtain a target background, wherein the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter, and the image processing module is used to synthesize the target object and the target background to obtain a processed target image;

[0030] A user interface is used to obtain the user demand information uploaded by the user and to display the processed target image.

[0031] Optionally, the user interface includes:

[0032] An image upload area, used to obtain the target image to be processed;

[0033] A dialogue input box, an area for obtaining the user demand information of the user;

[0034] An image display area, used to display the target image to be processed and the processed target image;

[0035] The history record area is used to display the user demand information uploaded by the user in the past;

[0036] The case display area is used to display historical images to be processed and processed historical images.

[0037] In a third aspect, the present application further provides an image processing device, specifically comprising:

[0038] A first processing module, configured to input user demand information related to a target image to be processed into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in the target image;

[0039] A first generating module, configured to generate a segmentation mask corresponding to the target object based on the description information of the target object;

[0040] A second generating module, configured to generate an alpha channel corresponding to the target object in the target image based on the segmentation mask;

[0041] A second processing module, configured to process a background area in the target image according to the alpha channel and the depth information of the target object to obtain a target background;

[0042] The image synthesis module is used to synthesize the target object and the target background to obtain a processed target image.

[0043] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the image processing method described in the first aspect above.

[0044] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method as described in any one of the above-mentioned first aspects are implemented.

[0045] In a sixth aspect, an embodiment of the present application further provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps of the image processing method as described in any one of the first aspects.

[0046] In an embodiment of the present application, at least one first parameter is obtained by inputting user demand information related to the target image to be processed into the first preset model, and the user demand information can be directly converted into a structured parameter for image processing through the first preset model, and a segmentation mask corresponding to the target object is generated based on the description information of the target object in the at least one first parameter, thereby generating an alpha channel corresponding to the target object in the target image according to the segmentation mask. Finally, the background area in the target image is processed according to the alpha channel and the depth information of the target object to obtain the target background, and the synthesis of the target object and the target background is achieved to obtain the processed target image.

[0047] In this way, the embodiments of the present application can perform intent analysis and interpretation on the user demand information uploaded by users, so that users who do not have professional knowledge of images can also process images through simple language information, thereby improving the user experience, and performing intelligent processing of images based on user needs, thereby improving the quality of image editing and enhancement effects, and effectively improving the intelligence level of image editing and enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0049] Figure 1 is a flowchart of an image processing method in an embodiment of the present application;

[0050] Figure 2 is a schematic diagram of a processing process of an image processing method in an embodiment of the present application;

[0051] Figure 3 is a structural schematic diagram of an image processing system in an embodiment of the present application;

[0052] Figure 4 yes Figure 3 Schematic diagram of the user interface in;

[0053] Figure 5 is a schematic diagram of an image processing device in an embodiment of the present application;

[0054] Figure 6 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] In order to make the embodiments of the present application clearer, the relevant technical knowledge involved in the embodiments of the present application is first introduced as follows:

[0057] Common image editing and enhancement operations include image cropping (used to remove unwanted parts of an image and adjust the composition), image scaling (used to change the image size to fit different devices or platforms, or to create special visual effects), image rotation and flipping (used to adjust the image direction or create mirror-symmetrical images), color adjustment (used to change the image's hue, saturation, brightness, or convert it to special colors such as black and white images), applying filters (used to improve the image's visual effects, such as sharpening, blurring, adding special effects, etc.), image denoising (removing noise from an image through image denoising technology to improve the image's clarity and purity), image sharpening and edge detection (using image sharpening and edge detection technology to highlight the image's edges and details, making the image clearer and sharper), Image color correction and enhancement (used to adjust the color balance, contrast and brightness of the image to make the colors more vivid and natural and enhance the visual effect of the image), image restoration (for degraded images, image restoration techniques such as back-projection restoration, iterative restoration and regularized restoration are used to restore the original image from the degraded image; or, image interpolation algorithms are used to estimate missing or damaged pixel values ​​to restore or reconstruct the complete image; low-resolution images can also be converted into high-resolution images through super-resolution reconstruction algorithms to enhance the details and clarity of the image), and image combination (used to combine two or more images into a new image, which can be used to create creative images, synthesize real scenes or simulate special effects, and can also be used to repair damaged images or create panoramic images), etc.

[0058] Traditional image processing tools, such as Photoshop and image applications, provide general functions for image editing and enhancement, which are usually implemented in the following ways: users manually select the area to be edited; the software (for example, the work order processing system in industrial applications can provide preset templates or batch processing functions for quick editing and enhancement) automatically or semi-automatically creates a mask layer to separate the content to be edited; the user selects the type of editing or enhancement; the software post-processes the editing and enhancement effects. However, the user interface of traditional image processing tools is complex, the learning cost is high, and users are usually required to have professional image knowledge to effectively operate various parameters and tools. Alternatively, traditional image processing tools lack intelligent and automated settings and cannot automatically complete tasks based on simple language descriptions by users. At the same time, their editing and enhancement effects often do not conform to the physical characteristics of real lenses, such as failing to consider the distance between the object and the camera.

[0059] In addition, some automatic image editing and enhancement technologies based on deep learning have emerged, which usually process images through the following steps: using image analysis networks to perform comprehensive analysis of images such as object detection, segmentation, and depth estimation; performing specific editing or enhancement operations on specific areas or objects based on user selections or prompts; and post-processing edited and enhanced images based on automated post-processing algorithms. Although the above methods can automatically process images, there is a lack of interaction with users, and users cannot specify image editing parameters or image enhancement strength, making it difficult to adjust editing and enhancement effects according to the specific needs of users. In addition, the depth estimation and image segmentation processes of these technologies are also automatically implemented, and their accuracy directly affects the final effect, and errors are prone to occur in complex scenes.

[0060] Therefore, the image processing method, system, electronic device, storage medium and program product proposed in this application can solve the problems of high user difficulty and low intelligence level in image editing and enhancement related technologies.

[0061] See also Figure 1 and Figure 2 , an image processing method in an embodiment of the present application specifically comprises the following steps:

[0062] Step 101: input user demand information related to a target image to be processed into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in the target image.

[0063] It should be noted that the user demand information may be language uploaded by the user through the terminal device to describe the image processing effect expected for the target image. It may be simple and non-professional, and can convey the user's expectations for the image processing effect of the target image without requiring the user to master complex image processing terms or technical details. For example, the user demand information includes "make the color of this photo more vivid", which means that color enhancement (ColorEnhancement) or color saturation adjustment (Saturation Adjustment) processing can be performed on the target image. In some embodiments, such as Figure 2 As shown in S201, the user can upload the target image to be processed and the user demand information through the terminal device. The terminal device or the server can receive and obtain the target image to be processed and the user demand information together, and input the user demand information into the first preset model, and identify the user's demand elements for processing the target image in the user demand information through the first preset model.

[0064] Specifically, the first preset model can be an artificial intelligence model that can parse user demand information and generate a first parameter. The first parameter can be a structured parameter. The structured parameter is a parameter that can be directly applied by a subsequent image processing process. For example, it may include descriptive information about an object and area in the target image, a numerical value of the enhancement intensity, and an editing type (such as cropping, color adjustment, object removal, etc.). It may also include parameters corresponding to other special image effects indicated by the user, which are not specifically limited in this application.

[0065] The target object may be an object or person in the target image indicated by the user that needs to be segmented from the image background, so as to process the target object and other areas separately and improve the image processing quality and accuracy.

[0066] Step 102: Generate a segmentation mask corresponding to the target object based on the description information of the target object.

[0067] In the specific implementation, Figure 2 As shown in S202, the target image can be visually segmented (Visual Grounding) by the structured parameters corresponding to the user demand information, and the pre-trained model of the Grounding DINO model can be used to identify the focus object through the description information of the target object, locate the target object in the target image, and realize the function of identifying the target object through the description information. Among them, the description information of the target object can be the vector of the target object in the target image. Subsequently, the recognition result of the target object can be input into another artificial intelligence model (Segment Anything Model, SAM). In order to segment the target object from the target image, the SegmentAnything model can generate a more accurate segmentation mask to improve the naturalness of the edge transition of the target object in the target image.

[0068] Step 103: Generate an alpha channel corresponding to the target object in the target image based on the segmentation mask.

[0069] In the specific embodiments of the present application, Figure 2 As shown in S203, after completing the visual segmentation of the target image, the target image can be subjected to mask generation processing. Specifically, a mask-to-matte (M2M) technology or model can be used to generate a smooth alpha channel through the segmentation mask for subsequent image rendering processing, thereby achieving a more natural editing and enhancement transition processing, effectively solving the problem of unnatural edges caused by image processing using related technologies, and improving the effect of image editing and enhancement processing and image quality.

[0070] Step 104: Process the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background.

[0071] It should be understood that Figure 2 As shown in S204, after completing the visual rendering and mask generation processing of the target image, the target image can be post-processed and rendered (BokehRenderer). Specifically, by using the above-mentioned alpha channel and depth information, the background part of the target image can be edited or modified in some form, which may include applying blur effects, adjusting colors, replacing backgrounds, or other types of image enhancement techniques. In this way, an optimized background image can be generated, which can be either an adjusted original background or a newly produced background, so that the processed target background should better match the foreground object (target object), making the overall picture more natural and coordinated. In this process, a preset algorithm can be used for processing so that the image processing process can support multiple types of editing and enhancement, and can also be adaptively edited and enhanced based on image deep learning, so that the image editing and enhancement effects are more natural and realistic, consistent with the bokeh characteristics of physical lenses, and provide users with more choices.

[0072] Step 105: synthesize the target object and the target background to obtain a processed target image.

[0073] In the present application embodiment, Figure 2 As shown in S205, by synthesizing the clear foreground image (i.e., the target object) obtained by visual segmentation and the target background obtained by visual segmentation, mask generation and post-processing rendering, editing and enhancement can be performed on a specific subject (target object or target background) to optimize the edge details of the target object in the target background, without being limited to a single editing and enhancement type.

[0074] It is worth mentioning that the embodiment of the present application can also perform image editing and enhancement processing on the target object through structured parameters related to target object processing corresponding to user demand information, and the present application does not make specific restrictions.

[0075] In this way, the embodiments of the present application can fully combine the user's user demand information to perform intent analysis, can understand the user's specific needs through natural language dialogue, provide more accurate and more user-intended editing and enhancement effects, generate structured parameters related to image editing and enhancement effects that meet user needs, lower the threshold for the use of user image processing technology, and improve image processing efficiency. The embodiments of the present application can also use the Grounding DINO model, SAM, M2M technology, etc. to separate the foreground and background of the target image, edit and enhance specific subjects, optimize edge details, and are not limited to a single editing and enhancement type. Subsequently, the target background can be post-processed and rendered, and the corresponding post-processing rendering effect can be applied according to the post-processing type required by the user to meet the user's image processing needs, intelligently execute the user's intentions, and improve the image processing quality and intelligence level.

[0076] Optionally, inputting user demand information related to the target image to be processed into a first preset model to obtain at least one first parameter includes:

[0077] Acquire the user demand information uploaded by the user through the user interface;

[0078] Inputting the user demand information into the first preset model to obtain the at least one first parameter;

[0079] Among them, the first preset model is a large language model trained based on user demand prompts.

[0080] In some specific embodiments, the first preset model can parse the user demand information and generate structured parameters related to the image editing and enhancement tasks. Specifically, the first preset model can be pre-trained with specific user demand prompts to improve the first preset model's ability to parse user demand information and improve its ability to convert user demands.

[0081] Among them, the specific user demand prompt can be a training set including user demand information and corresponding structured parameters, a structured parameter in the training set can correspond to one or more demand keywords in the user demand information, or a user demand keyword in the training set can correspond to one or more structured parameters. For example, a user sentence included in the user demand information is "I hope the background of this image is blurred and the characters are clearer", then the structured parameter corresponding to the user demand keyword background blur can be background blur (Background Blur) or depth of field effect (Depthof Field Effect), and the structured parameter corresponding to the demand keyword character clarity can be subject highlight (SubjectIsolation) or foreground sharpness (Foreground Sharpness). In this way, a training set containing a large number of user demands and corresponding structured parameters is set, and the first preset model is trained through the training set to improve the first preset model's ability to parse user demand information, accurately obtain user demands, facilitate subsequent image processing results to meet user expectations, and improve user experience.

[0082] It is worth mentioning that the first preset model in the embodiment of the present application can adopt a large language model (Large Language Model, LLM), such as a GPT-4o model, a bidirectional encoder representation method (Bidirectional Encoder Representations from Transformers, BERT) model, etc. The type of large language model used in this application is not specifically limited.

[0083] Optionally, the generating a segmentation mask corresponding to the target object based on the description information of the target object includes:

[0084] Inputting the description information of the target object into a second preset model to obtain the position information of the target object in the target image;

[0085] The position information is input into a third preset model to obtain the segmentation mask.

[0086] In some embodiments, the second preset model (e.g., Grounding DINO model), the third preset model (e.g., SAM), and the M2M model may be used successively for the target image to separate the target object from the background area in the target image. Specifically, the Grounding DINO model can accurately locate the target object in the target image according to the description information of the target object, and the SAM model can generate a segmentation mask according to the positioning result of the target object obtained by the Grounding DINO model, so that the boundary of the target object is clearer and more natural, thereby improving the separation quality of the target object and the background area.

[0087] It is worth mentioning that the Segment Anything model used in the embodiment of the present application can use point prompts to optimize the segmentation results, especially in complex edge areas. Among them, the point prompt can be a training set for training SAM, which includes the positions of a series of points specified in the image used for model training, which can be used to guide or optimize the Segment Anything model to recognize and segment the target object. The points in the training set can be located inside or near the edge of the target object, and this application does not make specific restrictions.

[0088] Optionally, the processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background includes:

[0089] Determining depth information of the target image based on the target object;

[0090] Processing the background area in the target image by a preset algorithm according to the depth information and the alpha channel to obtain the target background;

[0091] Wherein, the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter.

[0092] In some embodiments, the depth information of the target image is determined by the target object, and its depth can be inferred by analyzing the visual content in the image (for example, by using the distance of the target object in the image from the camera or the relative position between different objects in the scene where the image is located). Alternatively, the depth information is obtained based on the reflected light by calculating the light reflected from the scene where the target image is located. Alternatively, a deep learning method such as a convolutional neural network (CNN) is used. CNN learns mapping from a large amount of labeled image data in advance to obtain deep learning of the target image, thereby achieving effective depth prediction.

[0093] In a specific embodiment, in combination with depth information and alpha channel, a preset algorithm can be used to perform progressive processing on the background area in the target image. Progressive processing can be the gradual application of different effects, which may include background blur, color adjustment, brightness change, etc., rather than applying all effects at once, which helps to maintain the details of the image and ensure the naturalness of the image processing results.

[0094] It is worth mentioning that the preset algorithm in the present application can be a bokeh blur algorithm (such as the Dr.Bokeh algorithm), or Deeplens, MPIB and other algorithms can also be used as preset algorithms. Specifically, the preset algorithm in the present application can be an algorithm that has been improved in advance through structured parameters of various image processing. For example, the Dr.Bokeh algorithm has added a variety of editing and enhancement post-processing options in advance. Other editing and enhancement tools can be added on the basis of the original background blur, such as adjusting brightness, contrast, saturation, color temperature, sharpening, etc. It can also support different types of editing effects, such as applying filters, texture overlays or edge enhancements, to provide users with more creative space. In this way, the preset algorithm can not only focus on background blur, but also accept more comprehensive image adjustments to meet the personalized needs of users, which will enhance the practicality and flexibility of the preset algorithm and improve the quality and artistic expression of the image.

[0095] Based on the depth information, editing and enhancement can target areas of specific depth in the image to create a more realistic representation of the scene.

[0096] Optionally, the method further comprises:

[0097] When receiving user feedback information on the processed target image, input the feedback information into the first preset model to obtain at least one second parameter, wherein the at least one second parameter includes secondary description information of the target object;

[0098] Execute a target operation based on the secondary description information;

[0099] Among them, the target operation includes generating a segmentation mask corresponding to the target object based on the secondary description information, generating an alpha channel corresponding to the target object in the target image based on the segmentation mask, processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background, synthesizing the target object and the target background to obtain a processed target image.

[0100] In some other embodiments of the present application, Figure 2As shown, after synthesizing the target background and the target object, a processed target image is obtained, and the processed target image is fed back to the user. If the user's expectations are not met, the user can feedback the corresponding information and adjust the processed target image again. It can be understood that after receiving the user's feedback information, the first preset model can be used again, that is, BokehInterpreter can be used to analyze the user's new needs or adjustment requirements in the feedback information, and extract and generate the second parameter. The second parameter can also be a structured parameter, a parameter that can be directly applied by a subsequent image processing process. The second parameter can also include descriptive information of an object and area in the target image, a value of the enhancement intensity, an editing type (such as cropping, color adjustment, object removal, etc.), and can also include parameters corresponding to other special image effects indicated by the user, which are not specifically limited in this application.

[0101] Subsequently, the target object in the target image is located through the secondary description information of the target object in the second parameter, so as to generate the segmentation mask again through Segment Anything. Furthermore, according to the segmentation mask generated again, the mask optimization processing is performed to generate the alpha channel. Finally, according to the depth of the target image and the alpha channel generated again, the background area in the target image except the target object is processed accordingly to obtain the target background. In this way, the target background and target object obtained again are different from the target background and target object obtained last time. The target background and target object obtained by this target operation are synthesized to obtain the reprocessed target image. This target image is processed again based on the user feedback information and can be closer to user needs.

[0102] It is worth mentioning that in the embodiment of the present application, before the synthesized processed target image does not meet the user's expectations for image processing, the terminal device or the server can adjust the parameters according to the information fed back by the user, and the subsequent image processing process is performed based on this parameter until it meets the user's image processing requirements. Therefore, the embodiment of the present application can perform image editing and enhancement processing on the image to be processed according to the user's demand information, improve the image processing accuracy and processing quality, and the user does not need to manually select the processing area, simplify the image processing operation process, and improve the convenience and intelligence of image processing.

[0103] See also Figure 3 , Figure 3 It is an image processing system 300 in the embodiment of the present application, specifically comprising:

[0104] The language dialogue agent 301 is used to obtain user demand information uploaded by the user through the user interface, and input the user demand information into the first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in a target image to be processed;

[0105] The mask and cutout agent 302 includes a second preset model module, a visual segmentation module and a mask and cutout module, wherein the second preset model module is used to input the description information of the target object into the second preset model to obtain the position information of the target object in the target image, the visual segmentation module is used to input the description information of the target object into the second preset model to obtain the position information of the target object in the target image, and the mask and cutout module is used to input the position information into a third preset model to obtain a segmentation mask;

[0106] A post-processing agent 303, comprising an image processing module and an image synthesis module, wherein the image processing module is used to process the background area in the target image by a preset algorithm to obtain a target background, wherein the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter, and the image processing module is used to synthesize the target object and the target background to obtain a processed target image;

[0107] The user interface 304 is used to obtain the user demand information uploaded by the user and display the processed target image.

[0108] It should be noted that the image processing system provided in the embodiment of the present application can achieve the above Figure 1 All the method steps implemented in the method embodiment shown can achieve the same technical effect, and the parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0109] Optionally, the user interface includes:

[0110] An image upload area, used to obtain the target image to be processed;

[0111] A dialogue input box, an area for obtaining the user demand information of the user;

[0112] An image display area, used to display the target image to be processed and the processed target image;

[0113] The history record area is used to display the user demand information uploaded by the user in the past;

[0114] The case display area is used to display historical images to be processed and processed historical images.

[0115] In some specific embodiments, Figure 4 As shown, the image upload area set in the user interface can facilitate users to upload the target image to be processed, so that the terminal device or server can process the specific target image in time. The dialogue input box set in the user interface, that is, the user can send the image processing requirements here, that is, the image editing and enhancement effects can be specified by using simple dialogue statements, so that non-professional users can also easily create complex editing and enhancement effects, which fills the gap between user professional knowledge and technical implementation, and facilitates the image processing system to intelligently interpret and execute user intentions. After the image processing system completes the image processing, the processing results can be displayed to the user in the image display area.

[0116] In this way, by setting up a user interface, the image processing system can implement an intuitive and interactive editing and enhancement process, allowing users to easily achieve complex editing and enhancement effects through natural language dialogue.

[0117] Please refer to Figure 5 In another embodiment of the present application, an image processing device 400 is also disclosed, the device comprising:

[0118] A first processing module 401 is used to input user demand information related to a target image to be processed into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in the target image;

[0119] A first generating module 402, configured to generate a segmentation mask corresponding to the target object based on the description information of the target object;

[0120] A second generating module 403 is used to generate an alpha channel corresponding to the target object in the target image based on the segmentation mask;

[0121] A second processing module 404 is used to process the background area in the target image according to the alpha channel and the depth information of the target object to obtain a target background;

[0122] The image synthesis module 405 is used to synthesize the target object and the target background to obtain a processed target image.

[0123] Optionally, the first processing module 401 includes:

[0124] An information acquisition unit, used to acquire the user demand information uploaded by the user through the user interface;

[0125] A first processing unit, configured to input the user demand information into the first preset model to obtain the at least one first parameter;

[0126] Among them, the first preset model is a large language model trained based on user demand prompts.

[0127] Optionally, the first generating module 402 includes:

[0128] A second processing unit, configured to input the description information of the target object into a second preset model to obtain the position information of the target object in the target image;

[0129] The third processing unit is used to input the position information into a third preset model to obtain the segmentation mask.

[0130] Optionally, the second processing module 404 includes:

[0131] An information determining unit, configured to determine depth information of the target image based on the target object;

[0132] a fourth processing unit, configured to process the background area in the target image by a preset algorithm according to the depth information and the alpha channel to obtain the target background;

[0133] Wherein, the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter.

[0134] Optionally, the image processing device 400 further includes:

[0135] A third processing module is used for, when receiving feedback information from a user on the processed target image, inputting the feedback information into the first preset model to obtain at least one second parameter, wherein the at least one second parameter includes secondary description information of the target object;

[0136] An execution loop module, used for executing a target operation based on the secondary description information;

[0137] Among them, the target operation includes generating a segmentation mask corresponding to the target object based on the secondary description information, generating an alpha channel corresponding to the target object in the target image based on the segmentation mask, processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background, synthesizing the target object and the target background to obtain a processed target image.

[0138] It should be noted that the image processing device 400 provided in the embodiment of the present application is capable of performing the above Figure 1If the apparatus of the image processing method shown in the above-mentioned embodiment is used, all implementations of the above-mentioned image processing method are applicable to the image processing apparatus and can achieve the same or similar beneficial effects. To avoid repeated description, this embodiment will not be described in detail.

[0139] like Figure 6 As shown, the electronic device of the embodiment of the present application includes: a processor 510, which is used to read the program in the memory 520 and execute the following process:

[0140] Inputting user demand information related to the target image to be processed into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in the target image;

[0141] Based on the description information of the target object, generating a segmentation mask corresponding to the target object;

[0142] Based on the segmentation mask, generating an alpha channel corresponding to the target object in the target image;

[0143] Processing a background area in the target image according to the alpha channel and the depth information of the target object to obtain a target background;

[0144] The target object and the target background are synthesized to obtain a processed target image.

[0145] Among them, Figure 6 In the embodiment of the present invention, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 510 and memory represented by memory 520. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface.

[0146] Optionally, the processor 510 is configured to read a program or instruction in the memory 520 and perform the following steps:

[0147] Acquire the user demand information uploaded by the user through the user interface;

[0148] Inputting the user demand information into the first preset model to obtain the at least one first parameter;

[0149] Among them, the first preset model is a large language model trained based on user demand prompts.

[0150] Optionally, the processor 510 is configured to read a program or instruction in the memory 520 and perform the following steps:

[0151] Inputting the description information of the target object into a second preset model to obtain the position information of the target object in the target image;

[0152] The position information is input into a third preset model to obtain the segmentation mask.

[0153] Optionally, the processor 510 is configured to read a program or instruction in the memory 520 and perform the following steps:

[0154] Determining depth information of the target image based on the target object;

[0155] Processing the background area in the target image by a preset algorithm according to the depth information and the alpha channel to obtain the target background;

[0156] Wherein, the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter.

[0157] Optionally, the processor 510 is configured to read a program or instruction in the memory 520, and further execute the following steps:

[0158] When receiving user feedback information on the processed target image, input the feedback information into the first preset model to obtain at least one second parameter, wherein the at least one second parameter includes secondary description information of the target object;

[0159] Execute a target operation based on the secondary description information;

[0160] Among them, the target operation includes generating a segmentation mask corresponding to the target object based on the secondary description information, generating an alpha channel corresponding to the target object in the target image based on the segmentation mask, processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background, synthesizing the target object and the target background to obtain a processed target image.

[0161] The electronic device provided in the embodiment of the present application can perform the above Figure 1 or Figure 2 The implementation principle and technical effect of the method embodiment shown are similar, and this embodiment will not be repeated here.

[0162] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned Figure 1 or Figure 2The various processes of the image processing method embodiment in the embodiment can achieve the same technical effect, and are not repeated here to avoid repetition. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0163] The present application embodiment further provides a computer program / program product, wherein the computer program / program product is stored in a storage medium, and the computer program / program product is executed by at least one processor to implement the above Figure 1 or Figure 2 The various processes of the image processing method embodiment can achieve the same technical effect, and will not be described again here to avoid repetition.

[0164] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0165] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0166] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: include: Inputting user demand information related to the target image to be processed into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in the target image; Based on the description information of the target object, generating a segmentation mask corresponding to the target object; Based on the segmentation mask, generating an alpha channel corresponding to the target object in the target image; Processing a background area in the target image according to the alpha channel and the depth information of the target object to obtain a target background; The target object and the target background are synthesized to obtain a processed target image.

2. The method according to claim 1, characterized in that The step of inputting user demand information related to the target image to be processed into the first preset model to obtain at least one first parameter includes: Acquire the user demand information uploaded by the user through the user interface; Inputting the user demand information into the first preset model to obtain the at least one first parameter; Among them, the first preset model is a large language model trained based on user demand prompts.

3. The method according to claim 1, characterized in that The step of generating a segmentation mask corresponding to the target object based on the description information of the target object includes: Inputting the description information of the target object into a second preset model to obtain the position information of the target object in the target image; The position information is input into a third preset model to obtain the segmentation mask.

4. The method according to any one of claims 1 to 3, characterized in that The step of processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background includes: Determining depth information of the target image based on the target object; According to the depth information and the alpha channel, a background area in the target image is processed by a preset algorithm to obtain the target background; Wherein, the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter.

5. The method according to claim 2, characterized in that: The method further comprises: When receiving user feedback information on the processed target image, input the feedback information into the first preset model to obtain at least one second parameter, wherein the at least one second parameter includes secondary description information of the target object; Execute a target operation based on the secondary description information; Among them, the target operation includes generating a segmentation mask corresponding to the target object based on the secondary description information, generating an alpha channel corresponding to the target object in the target image based on the segmentation mask, processing the background area in the target image according to the alpha channel and the depth information of the target object to obtain the target background, synthesizing the target object and the target background to obtain a processed target image.

6. An image processing system, characterized in that: include: A language dialogue agent, used for obtaining user demand information uploaded by a user through a user interface, and inputting the user demand information into a first preset model to obtain at least one first parameter, wherein the at least one first parameter includes description information of a target object in a target image to be processed; A mask and cutout agent, comprising a second preset model module, a visual segmentation module and a mask and cutout module, wherein the second preset model module is used to input the description information of the target object into the second preset model to obtain the position information of the target object in the target image, the visual segmentation module is used to input the description information of the target object into the second preset model to obtain the position information of the target object in the target image, and the mask and cutout module is used to input the position information into a third preset model to obtain a segmentation mask; a post-processing agent, comprising an image processing module and an image synthesis module, wherein the image processing module is used to process the background area in the target image by a preset algorithm to obtain a target background, wherein the preset algorithm is a bokeh blur algorithm provided with the at least one first parameter, and the image processing module is used to synthesize the target object and the target background to obtain a processed target image; A user interface is used to obtain the user demand information uploaded by the user and to display the processed target image.

7. The system according to claim 6, characterized in that The user interface comprises: An image upload area, used to obtain the target image to be processed; A dialogue input box, an area for obtaining the user demand information of the user; An image display area, used to display the target image to be processed and the processed target image; The history record area is used to display the user demand information uploaded by the user in the past; The case display area is used to display historical images to be processed and processed historical images.

8. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.

10. A computer program product, characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps of the image processing method according to any one of claims 1 to 5.

Images