Image processing method and device, equipment and medium

Abstract

The invention provides an image processing method and device, equipment and a medium, and relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, deep learning and the like. The method comprises the steps of identifying a main body object corresponding to a target area on an original image in response to a detected input target operation for the original image; performing expansion operation on the main body object to obtain an adjacent area of the main body object; inputting a region identification graph for identifying the positions of the main body object and the adjacent region in the original image and the original image into an image matting model to obtain a foreground mask; separating the original image into a foreground image and a background image based on the foreground mask; performing blurring operation on the background image to obtain a blurred background image; combining the fuzzy background image and the foreground image into a preview image; and outputting the preview image.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the fields of computer vision and deep learning, specifically to an image processing method, an image processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies mainly include natural language processing, computer vision, speech recognition, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] This disclosure provides an image processing method, an image processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of this disclosure, an image processing method is provided, comprising: in response to detecting an input target operation on an original image, identifying a subject object corresponding to a target region in the original image; performing a dilation operation on the subject object to obtain an adjacent region of the subject object; inputting a region identifier map that identifies the positions of the subject object and the adjacent region in the original image into a matting model to obtain a foreground mask; based on the foreground mask, separating the original image into a foreground image and a background image; performing a blurring operation on the background image to obtain a blurred background image; merging the blurred background image and the foreground image into a preview image; and outputting the preview image.

[0006] According to another aspect of this disclosure, an image processing apparatus is provided, comprising: a recognition unit configured to recognize a subject object corresponding to a target region on the original image in response to detecting an input target operation on the original image; a dilation unit configured to perform a dilation operation on the subject object to obtain an adjacent region of the subject object; a matting unit configured to input a region identifier map identifying the positions of the subject object and the adjacent region in the original image into a matting model of the original image to obtain a foreground mask; an image separation unit configured to separate the original image into a foreground image and a background image based on the foreground mask; a blurring unit configured to perform a blurring operation on the background image to obtain a blurred background image; a preview unit configured to merge the blurred background image and the foreground image into a preview image; and an output unit configured to output the preview image.

[0007] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the methods described above.

[0008] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform the above-described method.

[0009] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program implements the above-described method when executed by a processor.

[0010] According to one or more embodiments of this disclosure, after detecting an input target operation on the original image, the disclosure first identifies the main object corresponding to the target region, and obtains the adjacent regions of the main object through a dilation operation, i.e., the edge regions that the matting model needs to focus on. Then, the region identifier map, which identifies the positions of the main object and its adjacent regions in the original image, is input together with the original image into the matting model, enabling the matting model to generate a matting result with better edge processing, i.e., a foreground mask. Subsequently, by separating the foreground from the background, and then blurring the background and merging it with the clear foreground image to form a preview image, the interference of complex backgrounds on the user can be quickly and accurately eliminated, allowing the user to intuitively judge the edge quality of the matting result, thereby improving the overall efficiency and accuracy of the matting operation and reducing the user's operational complexity.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings exemplify embodiments and form part of the specification, serving together with the textual description to explain exemplary implementations of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0013] Figure 1 A schematic diagram of an exemplary system in which the various methods described herein may be implemented according to embodiments of the present disclosure is shown; Figure 2 A flowchart of an image processing method according to an embodiment of the present disclosure is shown; Figure 3 A flowchart is shown illustrating an expansion operation on a subject object according to an embodiment of the present disclosure to obtain the adjacent region of the subject object; Figure 4 A flowchart illustrating a process of blurring a background image according to an embodiment of the present disclosure to obtain a blurred background image is shown. Figure 5 A flowchart of an image processing method according to an embodiment of the present disclosure is shown; Figures 6A-6C A schematic diagram of an image processing method according to an embodiment of the present disclosure is shown; Figure 7 A structural block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown; and Figure 8 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0014] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0016] The terminology used in the description of the various examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. Furthermore, the term "and / or" as used in this disclosure covers any one of the listed items and all possible combinations thereof.

[0017] In related technologies, existing image matting methods often suffer from poor matting results and a poor human-computer interaction experience. In particular, when processing images with complex edges, existing matting methods often produce poor-quality results, and users find it difficult to quickly judge the edge quality of the matted results against complex backgrounds.

[0018] To address the aforementioned issues, this disclosure, upon detecting the input target operation on the original image, first identifies the main object corresponding to the target region and obtains the adjacent regions of the main object through a dilation operation, i.e., the edge regions that the matting model needs to focus on. Then, the region identifier maps, which identify the positions of the main object and its adjacent regions in the original image, are input together with the original image into the matting model, enabling the model to generate a matting result with better edge processing, i.e., a foreground mask. Subsequently, by separating the foreground from the background, blurring the background, and merging it with the clear foreground image to form a preview image, the interference of complex backgrounds on the user can be quickly and accurately eliminated. This allows the user to intuitively judge the edge quality of the matting result, thereby improving the overall efficiency and accuracy of the matting operation and reducing the user's operational complexity.

[0019] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0020] Figure 1 A schematic diagram of an exemplary system 100 in which the various methods and apparatus described herein can be implemented according to embodiments of this disclosure is shown. Reference Figure 1 The system 100 includes one or more client devices 101, 102, 103, 104, 105 and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105 and 106 can be configured to execute one or more applications.

[0021] In embodiments of this disclosure, server 120 may run one or more services or software applications that enable the execution of the methods of this disclosure.

[0022] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtual and virtual environments. In some embodiments, these services may be provided as web-based services or cloud services, such as to users of client devices 101, 102, 103, 104, 105 and / or 106 under a Software as a Service (SaaS) model.

[0023] exist Figure 1 In the configuration shown, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or combinations thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 can sequentially interact with server 120 using one or more client applications to utilize the services provided by these components. It should be understood that various different system configurations are possible and may differ from system 100. Therefore, Figure 1 This is an example of a system used to implement the various methods described herein, and is not intended to be limiting.

[0024] Users can use client devices 101, 102, 103, 104, 105, and / or 106 for human-computer interaction. The client devices provide interfaces that enable users to interact with them. The client devices can also output information to the user through these interfaces. Although... Figure 1 Only six client devices are described, but those skilled in the art will understand that this disclosure can support any number of client devices.

[0025] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors, or other sensing devices. These computer devices can run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablets, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing various applications, such as various internet-related applications, communication applications (such as email applications), short message service (SMS) applications, and can use various communication protocols.

[0026] Network 110 can be any type of network well known to those skilled in the art, and can support data communication using any of a variety of available protocols (including but not limited to TCP / IP, SNA, IPX, etc.). By way of example only, one or more networks 110 can be a local area network (LAN), an Ethernet-based network, a token ring network, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0027] Server 120 may include one or more general-purpose computers, special-purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running a virtual operating system, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that can be virtualized to maintain virtual storage devices for servers). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0028] The computing unit in server 120 can run one or more operating systems, including any of the aforementioned operating systems and any commercially available server operating system. Server 120 can also run any of a variety of additional server applications and / or middleware applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, etc.

[0029] In some implementations, server 120 may include one or more applications to analyze and merge data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105 and / or 106. Server 120 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105 and / or 106.

[0030] In some implementations, server 120 can be a server for a distributed system or a server integrated with blockchain. Server 120 can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. A cloud server is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0031] System 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. Databases 130 may reside in various locations. For example, a database used by server 120 may be local to server 120, or it may be located away from server 120 and may communicate with server 120 via a network-based or dedicated connection. Databases 130 may be of different types. In some embodiments, the database used by server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data from and from the databases in response to commands.

[0032] In some embodiments, one or more of the databases 130 may also be used by an application to store application data. The databases used by the application may be of different types, such as key-value stores, object stores, or regular stores supported by a file system.

[0033] Figure 1The system 100 can be configured and operated in various ways to enable the application of the various methods and apparatus described in this disclosure.

[0034] According to one aspect of this disclosure, an image processing method is provided. For example... Figure 2 As shown, the method includes: step S201, in response to detecting an input target operation on the original image, identifying the subject object corresponding to the target region on the original image; step S202, performing a dilation operation on the subject object to obtain the adjacent region of the subject object; step S203, inputting a region identifier map that identifies the positions of the subject object and the adjacent region in the original image into a matting model with the original image to obtain a foreground mask; step S204, based on the foreground mask, separating the original image into a foreground image and a background image; step S205, performing a blur operation on the background image to obtain a blurred background image; step S206, merging the blurred background image and the foreground image into a preview image; and step S207, outputting the preview image.

[0035] Therefore, upon detecting the input target operation on the original image, this disclosure first identifies the main object corresponding to the target region and obtains the adjacent regions of the main object through a dilation operation, i.e., the edge regions that the matting model needs to focus on. Then, the region identifier maps, which identify the positions of the main object and its adjacent regions in the original image, are input together with the original image into the matting model, enabling the model to generate a matting result with better edge processing, i.e., a foreground mask. Subsequently, by separating the foreground from the background, blurring the background, and merging it with the clear foreground image to form a preview image, the interference of complex backgrounds on the user can be quickly and accurately eliminated. This allows the user to intuitively judge the edge quality of the matting result, thereby improving the overall efficiency and accuracy of the matting operation and reducing the user's operational complexity.

[0036] In some embodiments, the above image processing method can be applied to an image processing system or software with matting functionality. In step S201, the system can call a deep learning model (e.g., an image semantic segmentation model) to process the original image, thereby extracting the main object corresponding to the target region, or it can identify the main object on the original image through other means. It should be noted that the main object is not the matting result, but rather the initially identified main content of the image.

[0037] In some embodiments, the system can provide two matting modes: an intelligent selection mode jointly driven by the user and the image recognition method, and a fully user-driven manual mode. In intelligent selection mode, the target operation detected in step S201 can be a user's smearing operation on the original image, and the target area can be the smeared area. Through the smearing operation, the user can express the matting intention and indicate the rough location and general range of the object to be matted. It is understood that the target operation can also be other operations that define a specific area (i.e., the target area) on the original image, which is not limited here.

[0038] After obtaining the main object, the foreground mask is obtained as the result of the cutout through the dilation operation in step S202 and the cutout operation in step S203.

[0039] The adjacent region of the main object is the area surrounding the main object, the area close to the main object, and / or the area above or near the edge of the main object. This part is the key area of ​​focus for image cutout.

[0040] In some embodiments, in step S202, the main object can be expanded with a fixed expansion intensity. The adjacent region of the main object can be obtained by subtracting the corresponding region of the main object from the expanded region; alternatively, the adjacent region of the main object can be obtained by subtracting the region obtained after eroding the main object from the expanded region; or other methods can be used to obtain the adjacent region.

[0041] The region identification map has the same dimensions as the original image, and the value of each pixel in it indicates whether that pixel belongs to the subject object, adjacent regions, or other regions. The region identification map can indicate the location of the subject object and adjacent regions in the original image.

[0042] In some embodiments, in step S203, the region identifier map and the original image can be jointly input into the matting model (e.g., a deep learning model such as ModNet) to obtain a foreground mask. The foreground mask can be, for example, an alpha mask.

[0043] After obtaining the foreground mask, in step S204, the original image is separated into a foreground image and a background image based on the foreground mask.

[0044] In some embodiments, the foreground mask can be a grayscale image of the same size as the original image, where the value of each pixel (e.g., between 0 and 255) represents the transparency of the corresponding pixel in the original image belonging to the foreground. The foreground image can be obtained by multiplying the color value of each pixel in the original image by the transparency value of the corresponding position in the foreground mask (e.g., a normalized value between 0 and 1). Similarly, the background image can be obtained by multiplying the color value of each pixel in the original image by (1 - transparency value).

[0045] In some embodiments, in step S205, the background image is blurred to obtain a blurred background image while retaining a clear foreground image.

[0046] In some embodiments, the blurring operation may include, but is not limited to, Gaussian blur, mean blur, median blur, mosaic, or pixelation. By blurring the background image, the interference of high-frequency details such as complex textures, highlights, or noise that may exist in the background image on the user's vision can be effectively reduced, providing a low-interference background for subsequent preview images.

[0047] In some embodiments, in step S206, the blurred background image and the foreground image are merged into a preview image. The preview image can be displayed on the interactive interface where the user performs the smearing operation. Compared with the method of directly displaying the foreground on a transparent "checkerboard" background in related technologies, the preview image of this disclosure can effectively eliminate visual interference from the background, allowing the user to intuitively judge the edge quality of the foreground mask generated by the cutout model, thereby improving the overall efficiency and accuracy of the cutout operation and reducing the complexity of the user's operation.

[0048] In some embodiments, in step S207, a preview image may be output for display to the user.

[0049] According to some embodiments, such as Figure 3 As shown, step S202, performing an expansion operation on the main object to obtain the adjacent region of the main object, may include: step S301, performing edge analysis on the main object to obtain multiple edge segments of the main object; step S302, determining the expansion intensity of each of the multiple edge segments; and step S303, performing an expansion operation on the multiple edge segments based on their respective expansion intensities to obtain the adjacent region.

[0050] Therefore, by applying different expansion intensities to different edge segments of the main object, this disclosure can generate a dynamic asymmetric adjacent region for the subsequent matting model, allowing the model to concentrate more computational resources on complex edges that are difficult to handle, thereby improving the accuracy and efficiency of matting.

[0051] In some embodiments, in step S301, edge analysis may include: firstly, extracting the outer boundary contour of the subject object; then, traversing along the contour and dividing the contour into multiple continuous edge segments based on a preset contour division rule. The preset contour division rule may include fixed pixel length, angle variation, semantic boundary, or other rules.

[0052] In some embodiments, in step S302, the system can determine the dilation intensity of each of the multiple edge segments based on their respective characteristics. For example, a larger dilation intensity can be set for edge segments with higher edge complexity. The dilation intensity can be a numerical value and can be a pixel value of the morphological dilation operation radius.

[0053] In some embodiments, in step S303, the system can independently perform an expansion operation for each edge segment using the expansion intensity determined in step S302. Finally, the expansion results of all edge segments can be merged (e.g., through a logical OR operation) to form a complete adjacent region.

[0054] According to some embodiments, step S302, determining the expansion intensity of each of the multiple edge segments, may include: calculating the contour curvature of each of the multiple edge segments, wherein the expansion intensity of each of the multiple edge segments is positively correlated with the contour curvature of the corresponding edge segment.

[0055] Therefore, by correlating the expansion intensity with the contour curvature, the complexity of the edge can be quantified using its geometric features, thereby allocating a larger adjacency area to fine edges with high curvature such as hair strands and wrinkles, guiding the matting model to focus on processing them.

[0056] In some embodiments, multiple sampling points can be selected on each edge segment, and the local curvature or rate of change of curvature at these sampling points can be calculated. The average curvature or maximum curvature of an edge segment can be used as the contour curvature of that segment. The positive correlation can be implemented as a linear or nonlinear mapping function that maps the calculated curvature value to a preset range of dilation intensity (e.g., 5 pixels to 50 pixels).

[0057] According to some embodiments, edge analysis includes extracting semantic labels for multiple edge segments, wherein the dilation intensity of each edge segment is determined based on a preset dilation intensity of the corresponding semantic label of the edge segment.

[0058] Therefore, by utilizing advanced semantic prior knowledge, it is possible to more accurately identify edge regions that are inherently difficult to process, thereby achieving a more intelligent and robust inflation intensity allocation.

[0059] In some embodiments, semantic labels can be generated by a lightweight image segmentation or classification model during the edge analysis step. The system can pre-configure a mapping table or rule base, in which different dilation intensities are preset for different semantic labels (e.g., hair, fur, skin, clothing outline, etc.). For example, the preset dilation intensity corresponding to the "hair" label can be much greater than the preset dilation intensity corresponding to the "skin" label.

[0060] According to some embodiments, step S302, determining the dilation intensity of each of the multiple edge segments, may include: calculating the texture complexity of each of the multiple edge segments, wherein the dilation intensity of each of the multiple edge segments is positively correlated with the texture complexity of the corresponding edge segment.

[0061] Therefore, by analyzing the texture complexity of the region adjacent to the edge, complex edges caused by material can be effectively handled, thereby further improving the accuracy of edge analysis.

[0062] In some embodiments, calculating texture complexity may include analyzing the pixel value changes of neighboring original image regions for each edge segment. In an exemplary embodiment, the gradient magnitude, gray-level co-occurrence matrix features, or local binary patterns of the region may be calculated. Regions with higher texture complexity can be assigned a greater dilation intensity.

[0063] According to some embodiments, the adjacent region includes multiple adjacent sub-regions obtained by performing expansion operations on multiple edge segments respectively. In the region identification map, each of the multiple adjacent sub-regions has a corresponding identification value, and the identification value of each of the multiple adjacent sub-regions is increasing or decreasing in relation to the expansion intensity of the corresponding edge segment.

[0064] Therefore, since the dilation intensity characterizes the edge complexity of each edge segment, by encoding the label value associated with the dilation intensity into the region label map, the matting model can be guided to process edges of different complexities differently, thereby improving the quality of the obtained foreground mask.

[0065] In some embodiments, the identifier value can be a set of preset discrete integer values ​​or a continuous range of floating-point values. In an exemplary embodiment, the system can map the calculated dilation intensity (e.g., a radius value of 5 to 50 pixels) to the pixel values ​​(e.g., grayscale values ​​of 50 to 250) of the "adjacent region" in the region identifier map using a linear or non-linear function. In this way, the pixel values ​​of the adjacent region in the region identifier map can carry the complexity information of their corresponding edge segments.

[0066] According to some embodiments, such as Figure 4 As shown, step S205, blurring the background image to obtain a blurred background image, may include: step S401, determining the blur intensity corresponding to each of the multiple adjacent sub-regions based on the identifier values ​​of each of the multiple adjacent sub-regions; and step S402, applying the blur intensity corresponding to each of the multiple adjacent sub-regions to the background regions in the background image that correspond to the multiple adjacent sub-regions respectively.

[0067] Therefore, based on the identifier values ​​determined in the upstream steps, different blur intensities can be applied to different regions of the background image, achieving differentiated background processing and allowing users to better focus on complex edge regions.

[0068] In some embodiments, in step S401, the system may pre-construct a lookup table or function to map the identifier value (e.g., a grayscale value of 50 to 250) to the blur intensity. The blur intensity may include, but is not limited to, blur parameters such as the radius of the Gaussian blur, the kernel size of the mean blur, or the downsampling factor of the subsequent downsampling operation.

[0069] In some embodiments, in step S402, the background region corresponding to the plurality of adjacent sub-regions can be a background pixel region in the background image that is spatially adjacent to each adjacent sub-region.

[0070] In one exemplary embodiment, the aforementioned "proximity" can be quantified as a fixed distance. For example, the corresponding background region may include all background pixels in the background image that are within a preset distance threshold range from the boundary of each adjacent sub-region.

[0071] In other embodiments, the aforementioned "proximity" range can be dynamically determined, for example, it can be associated with the characteristics of adjacent sub-regions (their dilation intensity or identifier value). In an exemplary embodiment, for each adjacent sub-region, a morphological dilation operation is performed on that sub-region within the range of the background image, using its corresponding dilation intensity or identifier value as a parameter (e.g., as the radius of the dilation kernel). The background pixel region covered by this morphological dilation operation can be determined as the corresponding background region.

[0072] According to some embodiments, the blur intensity may include a downsampling factor. Step S402, applying the blur intensity corresponding to each of the multiple adjacent sub-regions to the background region in the background image, may include: for each adjacent sub-region, downsampling the background region corresponding to that adjacent sub-region based on the downsampling factor of that adjacent sub-region to obtain an intermediate result; and upsampling the intermediate result to obtain the blurred result of the background region.

[0073] Therefore, by using downsampling followed by upsampling to achieve fuzzing, the computational overhead can be reduced and the processing speed improved while ensuring the fuzzing effect.

[0074] In some embodiments, the downsampling factor can be positively correlated with the blur intensity. A higher blur intensity can correspond to a larger downsampling factor. In step S402, the system can apply its corresponding downsampling factor to each neighboring background region for image scaling to obtain an intermediate result; then, the intermediate result is upsampled back to its original size, thereby forming a pixelated blur effect in that region.

[0075] According to some embodiments, the fuzziness intensity of each of the multiple adjacent sub-regions is positively correlated with the label value of the corresponding adjacent sub-region.

[0076] This allows the background area corresponding to the high label value to be subjected to a higher blur intensity, thereby visually highlighting the adjacent foreground edge and making it easier for users to judge the quality of the cutout edge.

[0077] In one exemplary embodiment, a highly complex edge (e.g., a high contour curvature) corresponds to a high dilation intensity (as defined in claim 2), and consequently, a high label value (as defined in claim 6). This embodiment further defines the blur intensity as positively correlated with the label value. In this way, the background region adjacent to the highly complex edge is subjected to the highest level of blurring, thereby minimizing visual interference and facilitating accurate judgment of the highly complex edge by the user.

[0078] According to some embodiments, such as Figure 5 As shown, the image processing method may further include: step S508, adjusting the foreground mask in response to the detected input correction operation for the foreground mask; and step S509, re-performing the image separation, blurring and merging operations based on the adjusted foreground mask to obtain an updated preview image.

[0079] Understandable Figure 5 The operations of steps S501-S507 can be referred to the above text. Figure 2 The descriptions of steps S201-S207 are not repeated here.

[0080] After the preview image is output in step S507, the user can intuitively judge the quality of the cutout based on the preview image, and then correct the cutout through steps S508 and S509. Thus, these steps provide the user with a closed-loop workflow from automatic cutout processing to manual fine-tuning, enabling real-time feedback and iterative optimization of the cutout results.

[0081] In some embodiments, after the preview image is output in step S507, a correction tool for manually adjusting the foreground mask can be automatically displayed in the cutout interface. The correction tool may include a "smudge" tool (for increasing the foreground area) and an "erase" tool (for reducing the foreground area). Accordingly, the correction operation may include operations to increase the foreground area and operations to reduce the foreground area. In step S508, the user's operations using the correction tool are used in real time to update the foreground mask. In step S509, whenever the foreground mask is adjusted, the system can automatically repeat the operations of steps S504 to S507 in real time, i.e., re-separate the image, blur the background, merge, and output an instantly updated preview image.

[0082] Figures 6A-6C A schematic diagram of an image processing method according to an exemplary embodiment of the present disclosure is shown. Figure 6A As shown, the original image to be processed may include multiple people and a complex background. Figure 6B As shown, users can perform rough smearing operations on the original image (as indicated by the black brush strokes in the image) to indicate the subject of interest, namely the person on the far left. Figure 6C As shown, the image processing method disclosed herein responds to the smearing operation by automatically recognizing and completing the cutout, extracting the complete and clear foreground image (i.e., the person on the far left), blurring the background image, and finally merging the clear foreground image and the blurred background image into a preview image and outputting it.

[0083] According to another aspect of this disclosure, an image processing apparatus is provided. For example... Figure 7 As shown, the apparatus 700 includes: a recognition unit 710 configured to recognize a subject object corresponding to a target region on the original image in response to detecting an input target operation on the original image; a dilation unit 720 configured to perform a dilation operation on the subject object to obtain an adjacent region of the subject object; a matting unit 730 configured to input a region identifier map that identifies the positions of the subject object and the adjacent region in the original image into a matting model to obtain a foreground mask; an image separation unit 740 configured to separate the original image into a foreground image and a background image based on the foreground mask; a blurring unit 750 configured to perform a blurring operation on the background image to obtain a blurred background image; a preview unit 760 configured to merge the blurred background image and the foreground image into a preview image; and an output unit 770 configured to output the preview image.

[0084] It is understood that the operation and effects of units 710-770 in device 700 can be referred to the description of steps S201-S207 above, and will not be repeated here.

[0085] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0086] According to embodiments of this disclosure, an electronic device, a readable storage medium, and a computer program product are also provided.

[0087] refer to Figure 8 The present invention describes a structural block diagram of an electronic device 800 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0088] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0089] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, output unit 807, storage unit 808, and communication unit 809. Input unit 806 can be any type of device capable of inputting information to electronic device 800. Input unit 806 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device, and can include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 807 can be any type of device capable of presenting information, and can include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 808 can include, but is not limited to, disk and optical disk. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and can include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0090] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods, processes, and / or processing described above. For example, in some embodiments, these methods, processes, and / or processing may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the methods, processes, and / or processing described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform these methods, processes, and / or processing by any other suitable means (e.g., by means of firmware).

[0091] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0095] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0096] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is established by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0097] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0098] While embodiments or examples of this disclosure have been described with reference to the accompanying drawings, it should be understood that the methods, systems, and devices described above are merely exemplary embodiments or examples, and the scope of the invention is not limited by these embodiments or examples, but only by the granted claims and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. Furthermore, the steps may be performed in a different order than that described in this disclosure. Further, various elements in the embodiments or examples may be combined in various ways. Importantly, as the technology evolves, many elements described herein can be replaced by equivalents that appear after this disclosure.

Claims

1. An image processing method comprising: in response to detecting an input target operation on an original image, identifying a subject object corresponding to a target region on the original image; performing a dilation operation on the subject object to obtain a neighboring region of the subject object; inputting, into a cutout model, a region identification map identifying locations of the subject object and the neighboring region in the original image, to obtain a foreground mask; based on the foreground mask, separating the original image into a foreground image and a background image; performing a blur operation on the background image to obtain a blurred background image; merging the blurred background image and the foreground image into a preview image; and outputting the preview image. The performing a dilation operation on the subject object to obtain a neighboring region of the subject object comprises:

2. The method of claim 1, wherein, performing edge analysis on the subject object to obtain a plurality of edge segments of the subject object; determining a dilation intensity of each of the plurality of edge segments; and based on the dilation intensity of each of the plurality of edge segments, performing a dilation operation on the plurality of edge segments to obtain the neighboring region. The determining a dilation intensity of each of the plurality of edge segments comprises:

3. The method of claim 2, wherein, calculating a contour curvature of each of the plurality of edge segments, wherein the dilation intensity of each of the plurality of edge segments is positively correlated with the contour curvature of the corresponding edge segment. The edge analysis comprises extracting a semantic label of the plurality of edge segments, and the dilation intensity of each of the plurality of edge segments is determined based on a preset dilation intensity of the semantic label of the corresponding edge segment.

4. The method of claim 2, wherein, The determining a dilation intensity of each of the plurality of edge segments comprises:

5. The method of claim 2, wherein, calculating a texture complexity of each of the plurality of edge segments, wherein the dilation intensity of each of the plurality of edge segments is positively correlated with the texture complexity of the corresponding edge segment. The neighboring region comprises a plurality of neighboring sub-regions obtained by performing a dilation operation on the plurality of edge segments respectively, and in the region identification map, each of the plurality of neighboring sub-regions has a corresponding identification value, and the identification value of each of the plurality of neighboring sub-regions is in an increasing or decreasing relationship with the dilation intensity of the corresponding edge segment.

6. The method of claim 2, wherein, The performing a blur operation on the background image to obtain a blurred background image comprises:

7. The method of claim 6, wherein, based on the identification value of each of the plurality of neighboring sub-regions, determining a blur intensity corresponding to each of the plurality of neighboring sub-regions; and applying the blur intensity corresponding to each of the plurality of neighboring sub-regions to a background region corresponding to the neighboring sub-region in the background image to perform a blur operation. The blur intensity comprises a downsampling multiple, and the applying the blur intensity corresponding to each of the plurality of neighboring sub-regions to a background region corresponding to the neighboring sub-region in the background image to perform a blur operation comprises:

8. The method of claim 7, wherein, for each neighboring sub-region, based on the downsampling multiple of the neighboring sub-region, downsampling the background region corresponding to the neighboring sub-region to obtain an intermediate result; and upsampling the intermediate result to obtain a blur result of the background region. ​ 9. The method of claim 7, wherein, The blur intensity of each of the plurality of adjacent sub-regions is positively correlated with the identification value of the corresponding adjacent sub-region.

10. The method of any one of claims 1-9, further comprising: adjusting the foreground mask in response to detecting an inputted modification operation for the foreground mask; and re-performing image separation, blurring and merging operations based on the adjusted foreground mask to obtain an updated preview image.

11. An image processing apparatus, comprising: an identification unit configured to identify a subject object corresponding to a target region on an original image in response to detecting an inputted target operation for the original image; an expansion unit configured to perform an expansion operation on the subject object to obtain an adjacent region of the subject object; a matting unit configured to input a region identification map identifying locations of the subject object and the adjacent region in the original image and the original image into a matting model to obtain a foreground mask; an image separation unit configured to separate the original image into a foreground image and a background image based on the foreground mask; a blurring unit configured to perform a blurring operation on the background image to obtain a blurred background image; a preview unit configured to merge the blurred background image and the foreground image into a preview image; and an output unit configured to output the preview image.

12. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

13. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable a computer to perform the method of any one of claims 1-10.

14. A computer program product comprising a computer program, wherein, The computer program, when executed by a processor, implements the method of any one of claims 1-10. The computer program, when executed by a processor, implements the method of any one of claims 1-10.

Images