Method and system for content-adaptive tutorials for graphics editing tools in a graphics editing system

Abstract

Embodiments of the present application relate to content-adaptive tutorials for graphics editing tools in a graphics editing system. A method, system, and computer storage medium for providing a tool tutorial using a graphics editing system in the graphics editing system to operate based on tutorial information dynamically integrated into a tool tutorial shell. In operation, an image is received in association with a graphics editing application. Tool parameters (e.g., image-specific tool parameters) are generated based on processing the image. The tool parameters are generated for a graphics editing tool of the graphics editing application. The graphics editing tool (e.g., an object removal tool or a spot healing tool) can be an advanced version of a simplified version of a graphics editing tool in a freemium application service. Based on the tool parameters and the image, a tool tutorial data file is generated by merging the tool parameters and the image into the tool tutorial shell. The tool tutorial data file can be selectively drawn in an integrated interface of the graphics editing application.

Description

Technical Field

[0001] Embodiments of the present application relate to content - adaptive tutorials for graphics editing tools in a graphics editing system. Background Art

[0002] Users typically rely on graphics editing systems (e.g., desktop and cloud - based photo - editing applications) to edit and create digital graphics (or images). Such graphics editing systems include editing tools that support various image - editing functions. In particular, users use the editing tools to edit images and improve aspects of the images. For example, the healing brush is a tool for removing nuisances such as skin blemishes or unwanted stains from a photo. However, while powerful, graphics editing systems with a large number of complex editing tools create a steep learning curve for beginners of the graphics editing system. Thus, users need a set of editing tools that can quickly learn how to use the graphics editing system.

[0003] In context, the graphics editing system can specifically be a freemium - based application or service with free editing tools and premium editing tools. In particular, after a user pays for a subscription to the application or service, the premium editing tools can be accessed. Unfortunately, due to the additional complexity of the premium editing tools, it is difficult to effectively onboard new users to the premium editing tools. Generally, providing tutorials to help users understand the premium editing tools or editing tools in general greatly impacts the productivity and efficiency of creating and editing digital graphics. Conventional systems are limited in providing relevant guides (e.g., tutorials and workflows) to support educating users on the use of editing tools because current guides include statically - defined guide features. With the increasing use of digital graphics, improved tool tutorials for the graphics editing system can be utilized to efficiently and effectively use the editing tools in the graphics editing system. Summary of the Invention

[0004] Embodiments of the present disclosure are directed to providing tool tutorials based on tutorial information that is dynamically integrated into a tool - tutorial shell. For example, a tool tutorial for a spot - healing tool (i.e., an object - removal tool) is generated using: an image selected by a user, spot - healing tool parameters generated based on the user - selected image, and a spot - healing tool tutorial shell (e.g., a portable tutorial - document shell). Automatically generating the tool tutorial can be in the context of exposing a user to premium editing tools of a freemium - based application or service. The user is exposed to the premium editing tools as well as a content - adaptive tutorial for the premium editing tools (the "tool tutorial"). The tool tutorial includes the user's content (e.g., tutorial information containing image data) and tool parameters generated for the user's content, which are automatically incorporated into the tool - tutorial shell to improve the user's understanding and adoption of the premium tools.

[0005] In operation, a content-adaptive tutorial "tool tutorial" for a graphics editing tool is generated using graphics editing system operations in a graphics editing system. The graphics editing system includes a graphics editing engine and a graphics editing client that supports graphics editing applications. The graphics editing engine processes input data (e.g., image data), tool parameters, and a tool tutorial shell to automatically generate the tool tutorial. The tool tutorial is specifically generated based on the tool parameters of the editing tool, where the tool parameters are derived for an image using the tool workflow of the editing tool (i.e., a set of processing steps). Based on the input data, the graphics editing engine generates a tool tutorial data file (e.g., a portable tutorial document). The tool data file is caused to be drawn in a tool tutorial interface integrated with the graphics editing application. The tool tutorial data file can also be selectively drawn when the user meets a set of predefined conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1A and Figure 1B FIG. shows a block diagram of a graphics editing system for providing graphics editing system operations in which embodiments described herein can be employed;

[0007] Figures 2A to 2F is an illustration of example graphics editing system processing data for providing graphics editing system operations in accordance with aspects of the techniques described herein;

[0008] Figure 3 is an illustration of an example graphics editing system tool tutorial data file for providing graphics editing system operations in accordance with aspects of the techniques described herein;

[0009] Figure 4 provides a first example method for providing graphics editing system operations in accordance with aspects of the techniques described herein;

[0010] Figure 5 provides a second example method for providing graphics editing system operations in accordance with aspects of the techniques described herein;

[0011] Figure 6 provides a third example method for providing graphics editing system operations in accordance with aspects of the techniques described herein; and

[0012] Figure 7 is a block diagram of an example computing environment suitable for implementing aspects of the techniques described herein. DETAILED DESCRIPTION

[0013] TERM DEFINITIONS

[0014] Various terms and phrases are used herein to describe embodiments of the present invention. Some of the terms and phrases used herein are described here, but more details are included throughout the description.

[0015] As used herein, the term "graphics editing system" includes a graphics editing application or service (e.g., a cloud service application, a mobile application, and a desktop application) and functional components. An application or service generally refers to software downloaded to a client or software running on a server accessed using the client (e.g., a web application). In this regard, an application and a service can include client-side deployment and server-side software. For example, a graphics editing system can be a set of applications or services that enables subscribers to access a collection of software. The software can specifically support graphic design and photography as well as a set of mobile applications and some optional cloud services. A cloud application or service can be based on a freemium application service model, which is a pricing strategy through which an application or service can be provided, but a fee (premium) is charged for additional features. The graphics editing system or application in this model can support a set of free editing tools and another set of advanced editing tools. The provider of a cloud application or service can track usage data of the cloud application or service.

[0016] As used herein, a "graphics editing tool" or "tool" of a graphics editing system is a feature of the graphics editing system that supports image editing functions. In particular, a user will use an editing tool to edit an image and improve various aspects of the image. For example, a healing brush is a tool for removing nuisances such as skin blemishes or unwanted stains from a photograph. However, although powerful, a graphics editing system with a large number of complex editing tools creates a steep learning curve for beginners of the graphics editing system.

[0017] As used herein, the term "tutorial information" refers to information that can be used to generate or draw a tool tutorial. Tutorial information can be received directly from a user (e.g., image data) or indirectly (e.g., subscription information or user usage data). Tutorial information can be dynamically incorporated into a tool tutorial to customize and personalize the tool tutorial for a specific user. Tutorial information can be processed through a tool workflow (i.e., a set of processing steps of a tool) to generate tool parameters associated with the tutorial information. For example, a user's name, location, and other user data, as well as a photograph selected by the user, photograph metadata, and other metadata can be processed through the processing steps of an editing tool (e.g., a spot healing tool) to generate tool parameters, such that the tool parameters can be used to develop a tool tutorial.

[0018] As used herein, the term "tool parameter" refers to a measurable identifier having a quantifiable value for a tool workflow step, where the tool parameter can have other variable quantities. For example, if an object removal tool includes a first step for detecting a face, a second step for trimming the face, and a third step for detecting blemishes on the face, etc., the corresponding tool parameters (e.g., size, offsets on the x-axis and y-axis, perimeter, opacity, Gaussian, etc.) can have values associated with the steps for a particular image. The tool parameters and corresponding values can have placeholders in the tool tutorial shell, such that the tool parameters are incorporated into the tool shell to adaptively generate a tool tutorial data file.

[0019] As used herein, "tool tutorial" can refer to a computer program that supports helping a user learn how to use a tool of a graphics editing system. The tool tutorial supports the transfer of knowledge of using the editing tool and can be used as part of the learning process of the graphics editing system. There are different types of tool tutorials, such as video tutorials, interactive tutorials, and webinars.

[0020] As used herein, "tool workflow" refers to a set of processing steps for implementing a tool. For example, a blemish repair tool can have a blemish repair tool workflow that includes processing steps (e.g., face detection, face landmark identification, skin area trimming, etc.). The tool workflow is a set of processing steps for generating tool tutorial parameters that are dynamically incorporated into the tool tutorial along with tutorial information. The tool workflow processing steps can be incremental editing steps that support the final editing operation of executing the tool. It is contemplated that the tool workflow can be implemented based on a machine learning model. In this regard, a machine learning model (e.g., a deep learning model for identifying blemishes on a face in an image) that is part of the tool workflow itself can be used to generate parameters.

[0021] As used herein, the term "tool tutorial data file" refers to an output file generated based on dynamically merging tutorial information and tool parameters into a tool tutorial. The tutorial information and tool parameters can be specifically merged into a tool tutorial shell. A tool tutorial shell refers to a structured document or representation of a tool tutorial data file before the tutorial information is merged into the tool parameters (i.e., adaptive tutorial content). The tool tutorial data file can be in PTF (Portable Tutorial Format), which can be used for different types of graphics editing systems. The tool tutorial file includes different parts (e.g., chapters and sections), and each different part can be manipulated based on a tool workflow. In particular, tool workflow processing steps can be mapped to specific different parts of the tool tutorial (e.g., a one-to-one mapping between a processing step and a tutorial step including corresponding tool parameters). For example, the tool workflow processing steps for a spot healing tool can include face identification, thus identifying a face in an image selected by a user. The identified face and face identification parameters are tool parameters of the spot healing tool. The tool parameters are used to generate a customized tool tutorial. For example, the tool parameters are merged into the tool tutorial shell to generate a tool tutorial data file. The tool tutorial data file can append additional information (e.g., additional information steps) at the end of the tool tutorial data, which allows a user to continue editing the image.

[0022] Overview

[0023] As background, users often use graphics editing systems (e.g., desktop and cloud-based photo editing applications) to edit and create digital graphics (or images). Such graphics editing systems can include editing tools that support various image editing functions. In particular, users will use the editing tools to edit images to improve various aspects of the images. For example, a healing brush can be a tool for removing distractions such as skin blemishes or unwanted spots from a photo. A user can remove the distraction by painting over the unwanted object and then using information from another part of the photo to clone or eliminate the distraction. A user can adjust brush properties such as size, feathering, and opacity to accurately select and repair the unwanted area.

[0024] The graphics editing system can specifically be a freemium-based application or service with free editing tools and advanced editing tools, where the advanced tools can only be accessed after the user pays to access them. Unfortunately, due to the additional complexity of the advanced editing tools, it may be difficult to effectively guide new users to the advanced editing tools and services. Conventional systems are limited in providing relevant guides (such as tutorials and interfaces) to educate users on using the editing tools because the current guides include statically defined guide features. For example, a graphics editing system can include an interactive tutorial section where the tutorial includes guided help based on a set of pre-identified photos. However, existing tutorials (such as the retouching tool tutorial) cannot dynamically adapt to different images or other tutorial information. For example, existing tutorials demonstrate the retouching process for a pre-selected image. Tool parameters such as unwanted areas, retouching source areas, and brush sizes are pre-made by experts, making the output image perfect. However, using static existing tutorials may not effectively provide guidance to fully transfer knowledge of the editing tools to users or improve users' understanding of how to use the tool.

[0025] In addition, for the spot healing tool, the selection of the source "spot" area (such as a scar or blemish) and the target "healing source" area is driven by the image content, and this is very likely to change across different types of images. For example, when removing blemishes on a face, the healing parameters can vary based on the position, pose, and expression of the face. This means that a tutorial applicable to one image may not produce the desired results on some other images. Usage data from the graphics editing system also shows that the onboarding workflow is most effective when the tutorial includes actual user content.

[0026] In addition, the onboarding workflow may not be well integrated into the graphics editing system or strategically presented to users because conventional graphical user interfaces cannot sufficiently highlight the tutorials or other assistive content of the advanced editing tools in a way that is prominent to users. As a result, users may have to manually trigger the tool tutorials and sometimes even turn to online searches for tutorials. For example, user feedback shows that even though there are already tutorials in the graphics editing system, users still often spend hours on the internet looking for ways to fix defects in photos. Therefore, a dynamic onboarding experience personalized or customized for user content can improve the current user experience of the graphics editing system. For example, the existing onboarding experience of the spot healing tool tutorial based on a set of predefined static images can be improved to provide a more desirable user experience.

[0027] Custom tool tutorials can be based on tool workflows that improve existing tool workflows. In particular, a tool workflow is a set of processing steps that improve existing or conventional processing steps for performing graphical editing using an editing tool. For example, conventional methods implement an automatic detection method for detecting smudges based on feature extraction and thresholding to distinguish smudges from other facial features. However, using conventional methods, the color, shape, and lighting conditions of smudges in an image can have a negative impact on the detection results. Other conventional methods include heat-mapping and adaptive thresholding; however, these conventional methods may be sensitive to noise and may confuse the area around the mouth with the skin. At a higher level, conventional methods implement smudge detection based on spatial color spaces and thresholds; however, the drawback of conventional methods is that they are highly dependent on thresholds and thus lack applicability to various image data. Therefore, an integrated graphical editing system with an alternative basis for performing graphical editing operations (e.g., generating custom tool tutorials based on tool workflows) can improve the computational operations and interfaces that support providing tutorials to help users understand complex editing tools.

[0028] Embodiments of the technical solution can be illustrated with reference to a graphical editing system, and additional details of the graphical editing system are provided in the following specification and with reference to the corresponding diagrams. At a higher level, a graphical editing system (e.g., a photo editing application) can include a large number of complex graphical editing tools. Different graphical editing tools can be used for different graphical editing functions supported by the graphical editing system. The use of different editing tools can be tracked within the graphical editing system. For example, the graphical editing system can be a freemium application or service in which both free and premium editing tools exist. For example, a smudge repair editing tool can be a paid feature of a graphical editing application, where the graphical editing system also includes free graphical editing tools.

[0029] Usage data can reveal that a large number of users purchase premium editing tools (or enter a purchase workflow) based on the use of a specific editing tool (e.g., a smudge repair tool). Therefore, in an example implementation of the present technical solution, the smudge repair tool is a tool for describing a novel technical solution because the smudge repair tool is popular and helps convert users into premium users. Additionally, it is generally expected that the tracked usage data of editing tools in a freemium application or service can be used to identify user behavior with reference to the editing tools and other operations, and these other operations can support automatically generating tool tutorials as described in more detail herein.

[0030] The technical solution of the present disclosure can specifically provide a novel tool tutorial presentation workflow, so that users can immediately understand the tools, tutorials, and the differences between free tools and advanced tools, thereby obtaining a more user-friendly entry experience. For example, the "Let me try" feature can be part of the demonstration workflow and serve as an entry point for users to evaluate advanced editing tools. By demonstrating the workflow, users can choose to try out advanced editing tools specifically for their own images. In particular, users can try out advanced editing tools before purchasing a subscription to the advanced editing tool or advanced service. For example, interface elements can be generated that identify a graphic editing tool (e.g., the healing brush) as an advanced graphic editing tool with selectable buttons (e.g., "Let me try" or "Learn how"), and these selectable buttons trigger additional actions associated with the functions described herein. The interface elements can also include additional information or content to enhance the user's perception of different aspects of the editing tool (e.g., tutorial details, purchase details, or information content). In this regard, it is easier to correctly convey the positive impact of different graphic editing tools.

[0031] The presentation workflow can also operate without user intervention. For example, when it is determined that the user is using a free editing tool, the user's content (i.e., tutorial information including image data) can be automatically used to generate a tool tutorial for another tool of the graphic editing system (e.g., a free tool or an advanced tool). Based on the generated tool tutorial, the tool tutorial can be selectively drawn for the user. The tool tutorial can be selectively drawn based on identified heuristics, such as heuristics from usage data. Heuristic-based methods can be used to automatically generate and draw the tool tutorial and can be configured as needed. For example, a tool tutorial can be generated or presented when certain conditions are determined (e.g., determining that the user is a freemium user and the user clicks on a specific editing tool; determining that the user has not tried a specific tool after a certain duration; or determining that the user is facing challenges when using a specific tool). Embodiments of the present disclosure contemplate other variations and combinations of automatically generating and presenting tool tutorials based on heuristic methods.

[0032] As discussed, simply using predefined sample images for tool tutorials has several drawbacks. Thus, current technical solutions include providing content-adaptive tool tutorials. The technical solutions may include analyzing a user image and identifying whether a tool workflow (i.e., a set of processing steps for a tool) can be applied to the image. The tool workflow can be used to generate tool parameters for an editing tool based on the specific image and other tutorial information. For example, a spot healing tool may have a spot healing tool workflow that can be applied to an image to enhance the image. The spot healing tool workflow may include processing steps different from the conventionally known processing steps for the spot healing tool, as discussed in more detail below. When the spot healing tool workflow is applied to the image, tool parameters (i.e., repair parameters) can be generated. Typically, after determining that the tool workflow can be applied to the image, a tool tutorial is dynamically generated using the computed tool parameters. The tool tutorial can be provided to the user via various channels, such as an onboarding workflow, contextual help, and in-app messaging. To simplify the description of the technical solutions, an example of the technical solutions will be described using the spot healing tool or the healing tool; however, it is expected that the technical solution functionality can be applied to different types of editing tools.

[0033] At a higher level, for example, a graphics editing system operation may include receiving tutorial information, including an image selected by the user that contains spots that should be corrected using the spot healing tool. If the spot healing tool is applicable to the image, the graphics editing system operation may also include automatically identifying the tool parameters for the image. Identifying the tool parameters may include identifying a spot mask, a repair source, and other tool parameters from the image based on the spot healing tool workflow, and dynamically generating a tool tutorial using the tool parameters. For example, the spot healing tool workflow may include detecting faces in the image and identifying facial landmark points. Based on the face and the facial landmark points, a skin area trimming operation is performed. Spots (e.g., scars and blemishes) can be identified on the face, thereby generating a mask layer for the spots. A corresponding repair source region can be identified for the spots, where the repair source region supports repairing the spots. The spot healing tool parameters are derived based on the spots and the repair source region. The spot healing tool parameters and the image are used to generate a tutorial data file (e.g., a tutorial document). The tutorial data file can be generated in different workflows associated with the graphics editing system.

[0034] Accordingly, the embodiments described herein address the above problems of conventional graphics editing systems by providing tool tutorials based on tutorial information that is dynamically integrated into a tool tutorial shell. Advantageously, content-adaptive graphics editing can be provided based on tool tutorials (e.g., a spot healing tool tutorial) across multiple different types of graphics editing systems or software. Specifically, the workflow of a content-adaptive graphics editing tutorial can be applied to different types of graphics editing systems and corresponding tool tutorials. As described herein, the generated tool tutorials can be presented in a graphics editing system interface to provide guidance to users for editing their own images or photos. In this way, users can interactively use the tool tutorials in different types of graphics editing systems, which will improve the users' overall understanding of different editing tools and also improve the users' graphics editing experience.

[0035] Graphics editing system

[0036] Reference Figure 1A and Figure 1B , Figure 1A and Figure 1B illustrate an example graphics editing system 100 in which the methods of the present disclosure may be employed. In particular, Figure 1A and Figure 1B illustrate the high-level architecture and operation of a graphics editing system 100 according to an implementation of the present disclosure. In addition to other engines, managers, generators, selectors, or components (collectively referred to herein as "components") not shown, the technical solution environment of the graphics editing system 100 further includes a graphics editing engine 110 (including a graphics editing system operation 112 and a graphics editing system interface 114), a graphics editing engine input data 120 (including tutorial information 122 and a tool tutorial shell 124), a graphics editing engine output data 130 (including graphics editing system operation data 132 and a tool tutorial data file 134), a tool parameter engine 140 (including a tool parameter model 142, a tool workflow 144, and a tool parameter generator 144), a graphics editing client 180 (including a graphics editing interface 182 and a graphics editing system operation 184), and a network 190.

[0037] The components of the graphics editing system 100 may communicate with each other via one or more networks (e.g., a public network or a virtual private network "VPN") as shown by the network 190. The network 190 may include, but is not limited to, one or more local area networks (LANs) and / or wide area networks (WANs). The graphics editing client 180 may be a client computing device corresponding to the computing device described herein with reference to Figure 7 The graphics editing system operations (e.g., the graphics editing system operation 112 and the graphics editing system operation 184) may be performed by a processor executing instructions stored in a memory, as further described with reference to Figure 7 ​

[0038] At a higher level, the graphics editing system 100 includes a graphics editing engine 110 and a graphics editing client 180. The graphics editing client 180 receives and transmits tutorial information 122, enabling the graphics editing engine to process graphics editing engine input data 120, including a tool tutorial shell 124. Based on the graphics editing engine 110 processing the input data 120, a tool parameter engine 140 can generate tool parameters, enabling the graphics editing engine 110 to generate graphics engine output data (e.g., graphics editing system operation data 132, tool tutorial data file 134) based on the tutorial information 122, the tool tutorial shell 124, and the tool parameters.

[0039] Accordingly, components of the graphics editing system can be used to perform graphics editing system operations that provide a tool tutorial based on tutorial information dynamically integrated into a tool tutorial shell. In operation, an image is received in association with a graphics editing application. Tool parameters are generated based on the processing of the image. The tool parameters are generated for an editing tool of the graphics editing application. The editing tool (e.g., an object removal tool or a spot healing tool) can be an advanced version of a simplified version of a graphics editing tool in a freemium application service. Based on the tool parameters and the image, a tool tutorial data file is generated by incorporating the tool parameters and the image into the tool tutorial shell. The tool tutorial data file can be selectively rendered in an integrated interface of the graphics editing application.

[0040] Reference Figure 1B , the graphics editing system 100, the graphics editing engine 10, and the graphics editing client 20 with reference Figure 1Aand the described components correspond. At a higher level, in step 52, the graphics editing client accesses and transmits input data (e.g., tutorial information, image data, metadata, and other user content) via an application interface. The input data can be transmitted from the graphics editing client 20 to the graphics editing engine 10 such that the input data is stored in the graphics editing engine. The graphics editing client can be a client device that uses an application or service to access the graphics editing application, and some or all of the functions of the graphics editing engine 10 can be executed locally on the client device or remotely on a server that supports the application or service associated with the input data. The input data can also include a tool tutorial shell (e.g., tool tutorial shell 124) for generating a tool tutorial data file (e.g., tool tutorial data file 134). In step 54, the graphics editing engine 10 accesses tutorial information (e.g., an image) and initializes a tool workflow. The tool workflow can be used for an object removal tool (e.g., a spot healing tool). The tool workflow is a set of processing steps for implementing a tool. The tool workflow can be associated with a machine learning model that supports generating tool parameters for the tool and further supports the manner of generating refined tool parameters.

[0041] In steps 56, 58, and 60, the graphics editing engine 10 performs operations (e.g., using the tool parameter engine 140) that support generating tool parameters for the tool workflow. Specifically, generating tool parameters based on executing the set of processing steps includes accessing a measurable identifier of the tool parameter. The measurable identifier corresponds to one or more steps in the set of processing steps.

[0042] For example, the measurable identifier of the tool parameter can be the x-axis and y-axis offsets of a spot area or a repair area. A quantization value for the measurable identifier is determined based on performing one or more steps on the image data of the image. For example, the offset values on the x-axis and y-axis of the spot area or the repair area are identified and stored as tool parameters. If other steps in the tool workflow are associated with the tool parameter, the tool parameter can be incrementally updated as each step of the tool workflow is executed. It is expected that different types of variations and combinations of the tool parameter can be defined for one or more processing steps of the tool workflow such that when one or more steps are performed on the image data of the image, the resulting quantization value corresponds to the tool parameter that is the measurable identifier.

[0043] The tool workflow is initially described at a higher level and is discussed in more detail below with reference to the corresponding drawings. At step 56, face detection, identification of facial landmarks, and skin region trimming operations are performed on the image. At step 58, the graphics editing engine 10 performs a stain detection operation on the image. It is expected that determining that a stain has been detected is based on a stain detection confidence score. The stain detection confidence score is a quantitative value used to determine whether a candidate stain is selected as a stain. Different scoring models can be used to score candidate stains, and a subset of candidate stains is selected based on the stain detection confidence score and a stain detection confidence score threshold.

[0044] At step 60, the graphics editing engine 10 determines a repair source for the stain based on identifying a repair source region in the image. A tool parameter engine (e.g., tool parameter engine 140) that generates tool parameters can be used to perform application-specific tool workflow operations. The tool parameters can include the identified face, facial landmarks, trimmed regions of the skin, etc. It is expected that other editing tools can have different tool parameters derived from their corresponding tool workflows.

[0045] At step 62, the graphics editing engine 10 generates tool parameters based on the tool workflow operations performed on the image. At step 64, based on the tool parameters and the image, the graphics editing engine 10 generates and transmits a tutorial data file to cause the drawing of the tutorial data file. At step 66, the graphics editing client receives the tool tutorial data file and causes the drawing of the tool tutorial data file based on selective drawing conditions.

[0046] Reference Figures 2A to 2F , the graphics editing system operation can generate tool parameters from the tool workflow, thereby using the tool parameters and tutorial information to generate a tool tutorial data file. For example, Figures 2A to 2E shows graphics editing system operation data (e.g., graphics editing system operation data) as data generated when executing the tool workflow for the stain repair tool workflow. As described above, the stain repair tool workflow includes performing stain tool workflow operations, including face detection and identification of facial landmarks, and skin region trimming based on the identified facial landmarks. Detecting facial landmarks is a subset of the shape prediction problem. In this regard, the graphics system operation includes a two-part process of locating the face in the image and then detecting key facial structures on the face region of interest. Facial landmark points are used to extract the boundaries of the face region and separate the facial landmarks. As Figure 2A and Figure 2B shown, a face (e.g., the face boundary in Figure 2B ) can be detected with multiple facial landmark points (e.g., facial landmark points 202A - 202MM in Figure 2A ).

[0047] The tool workflow operation can include the Graphics Editing Engine 10 using a histogram of oriented gradients (HOG) feature as a face detector in combination with a pre-trained linear classifier to detect face boundaries in an image. The Graphics Editing Engine 10 can include a face analysis machine learning model (e.g., the tool parameter model 142), which supports detecting face features and generating a face detection confidence score based on the predictions of the face analysis machine learning model, regardless of whether a face is detected. In one implementation, to detect face features, the Graphics Editing Engine 10 uses an ensemble of regression tree algorithms to achieve face alignment in one millisecond. In this implementation, a general cross-platform software library (e.g., the Dlib library) is used to perform face feature detection to estimate the positions of 68 (x, y) coordinates of the face structure mapped to the face.

[0048] Facial landmark detection can include identifying key landmarks on the face. Facial landmarks are used to locate and represent prominent regions of the face, such as the eyes, eyebrows, nose, mouth, and jawline. The detected face and feature points are as Figure 2B shown, where the face features correspond to clusters of facial landmark points. For example, the first cluster of facial landmark points corresponds to the first eye, and the second cluster of landmark points corresponds to the second eye. If it is determined that a face has been detected (e.g., based on the face detection confidence score), additional Graphics Editing Engine operations can be performed on the image (including creating an object removal tool tutorial or a spot healing tool tutorial for the image).

[0049] Turning Figure 2C to, the tool workflow operation also includes skin area trimming based on the identified facial landmarks. As context, the graphics editing operation supports identifying face spots and corresponding repair sources. The identification of face spots can be specifically performed to reduce a large number of false positives. Conventionally, in face analysis operations, facial features such as nostrils, eye corners, and mouths may be mis-identified as face spot candidates. In embodiments of the present disclosure, to prevent false positive scenarios, the graphics editing operation includes skin area trimming of the detected face. Skin area trimming includes segmenting the face area, including the eye, nostril, and mouth areas.

[0050] As Figure 2C shown, the trimmed area is enclosed in 302, where the face features include the first eye area 304, the second eye area 306, the bottom area 308 of the nostrils, and the lip area 310 that are trimmed off. Advantageously, removing some face areas helps improve the accuracy of additional graphics editing operations, where graphics editing operations are performed on the trimmed area to predict the spot (e.g., blemish or scar) area and the corresponding repair source area to repair the spot.

[0051] The tool workflow operation also includes detecting spots (e.g., scars and blemishes) and generating an accurate mask for the spots. AsFigure 2A As shown, the facial landmark detection operation supports extracting the boundaries of the facial region, including separating the eye and hair regions of the face. The facial landmark detection operation also supports accurately separating the skin region from the non-skin regions of the face. The tool workflow operation also includes using a deep learning-based classifier to detect the presence of facial stains. In particular, the tool workflow operation can be implemented based on a convolutional neural network (CNN) and a classifier. The CNN can be used as a feature extractor and a classifier to distinguish stains from normal regions in the face and generate a mask layer covering the stains.

[0052] Reference Figure 2D , in operation, stain detection (as shown in Figure 400A) and mask generation (as shown in Figure 400B) can be performed as part of the tool workflow. To detect stains and generate a mask, the tool workflow includes feeding the input image into the CNN to generate a convolutional feature map. From the convolutional feature map, the tool workflow operation supports identifying candidate stain regions (e.g., candidate stain 402 and candidate stain 404). The candidate stain regions can be distorted into a predefined shape (e.g., a square). Rol pooling (also known as region of interest pooling) is an operation that uses a convolutional neural network to support object detection tasks. For example, to detect multiple stains in a single image, Rol pooling can be used to perform max pooling on the input of non-uniform size to obtain a fixed-size feature map. Thus, using the Rol pooling layer, the tool workflow operation supports reshaping the candidate stains into a fixed size so that the candidate stains can be fed into the fully connected layer. From the Rol feature vector, the candidate stains can be predicted and classified as stains or non-stains in the proposed region and the mask region of that point using the softmax layer. It is expected that the workflow operation can be based on Faster R-CNN, which includes two networks (i.e., a region proposal network for generating region proposals and a network for detecting objects using these proposals). Faster R-CNN can replace the selective search algorithm applicable to the feature map to identify region proposals.

[0053] The tool workflow operation also includes generating a feature map. A pre-trained improved CNN (e.g., the VGG16 model) for classification and detection can be used to generate the feature map. An image with the facial region can be used to enhance the feature map. Advantageously, focusing only on the facial region can result in more efficient and accurate prediction of stains. It is expected that the CNN and stain detection can be associated with a stain detection confidence score representing the quantified confidence that a stain has been detected. Based on the stain detection confidence score (e.g., if the probability value is less than a predefined threshold), it can be determined that there are no stains in the image. At this time, remedial operations can be performed, including skipping the generation of the tutorial, prompting the user to input another image, and transmitting an information interface element explaining the current state of the tutorial.

[0054] Reference Figure 2E Based on the stain mask, the repair source region can identify corresponding points (e.g., the repair region 502, the stain region 504, and the stain 506). Thus, the tool workflow operations include identifying the repair source region that will be used to repair the stain (or stain region). Operationally, the identified mask layer or mask region is accessed. A repair brush algorithm (e.g., the repair brush algorithm for the content-aware stain removal tool tutorial) is applied to any identified stain. For each identified stain, the tool workflow operations calculate a set of boundary pixels corresponding to the mask of that stain. Then the tool workflow operations evaluate the set of boundary pixels to identify the repair source region. Additionally, the tool workflow operations further include: automatically based on a random angle and an offset from the stain, and further based on: evaluating candidate repair source regions based on the standard deviation of the boundary pixels between the candidate repair source region and the target stain to find the repair source region.

[0055] The repair brush algorithm can be specifically configured to improve accuracy and reduce false positives. The repair brush algorithm can be configured to exclude any candidate repair source region that meets certain predefined conditions. For example, a first condition can exclude a candidate repair source region if the overlap between the candidate repair source bounding box and the face landmark bounding box (e.g., identified during a skin trimming operation) is non-empty; while a second condition can exclude a candidate repair source region if the candidate repair source region bounding box (e.g., identified during deep learning stain detection and masking) is outside the skin region. Embodiments of the present disclosure contemplate other variations and combinations of exclusion conditions for improving accuracy.

[0056] The repair brush algorithm can also include an increase in the error threshold between the repair source region and the stain boundary pixels. For example, a maximum difference of 1 / 255 can be defined as the error threshold. Any candidate repair source region where the boundary pixel difference exceeds the defined error threshold is rejected. If a candidate repair source region does not meet the defined criteria, the tool workflow operations proceed to the next identified stain. When analyzing all identified stains without identifying a repair source region, the image can be rejected as a candidate image for generating the tool tutorial. If a repair source region is identified for a stain, repair parameters corresponding to the stain are generated and provided for generating the tool tutorial.

[0057] Reference Figure 2F , Figure 2FShows example tool parameters for a spot healing tool. The tool parameters are represented in a human-readable and machine-readable markup language document (e.g., tool parameter 600). A text data format is used to represent different types of tool parameters based on tutorial information and tool workflows. The tool parameter representation can be divided into markup and content, including different tags, elements, and attributes corresponding to the tool parameters. For example, the tool parameter markup and content can include "RetouchAreas" 602, "SourceX" 604, "OffsetY" 606, and mask "SizeX" 608 and "SizeY" 610. Embodiments of the present disclosure contemplate other variations and combinations of the tool parameter data representation of the markup and markup language content.

[0058] Turn to Figure 3 , Figure 3 Shows an example tool tutorial data file based on a tool tutorial shell integrated with tool parameters. The graphics editing engine 10 is configured to generate the tool tutorial data file. The tool tutorial data file is generated based on the tool tutorial shell. The tool tutorial shell can refer to a structured document (e.g., a tutorial structured document for an adaptive tutorial) that can cause to be drawn in a tool tutorial interface integrated into a graphics editing application. The tool tutorial data file can be in a portable format such that the tool tutorial data file can be drawn in different contexts. For example, the tutorial data file can have a PTF file extension, where the tutorial data file includes audio and video capabilities. The tool tutorial data file can include different sections (e.g., chapters and parts), where each different section can include one or more editing steps. In one implementation, there is a one-to-one mapping between tutorial steps across different sections. The tutorial steps and integrated tool parameters are represented in a human-readable and machine-readable markup language document (e.g., tool tutorial data file 700). A text data format is used to represent different types of tutorial steps based on tutorial information and tool workflows. The tool parameter representation can be divided into markup and content, including different tags, elements, and attributes corresponding to the tool parameters. For example, the step statement text 302 includes tool tutorial shell text that can be integrated with mask data (e.g., SizeX 304 corresponding to SizeX 608 of the tool parameter data representation). It is expected that additional information steps can be appended to the generated tool tutorial data file to allow the user to continue editing and further refine the image associated with the tool tutorial data file. Embodiments of the present disclosure contemplate other variations and combinations of the tutorial data file representation.

[0059] Tutorial data files can be transmitted for rendering in a graphics editing application interface. The tutorial data files can be selectively rendered in an integrated tutorial interface or some other interface that supports rendering of the tutorial data files. The tutorial data files can be selectively rendered. Selectively rendering the tutorial data files can be based on a number of different methods. For example, heuristic methods can be used such that the tutorial data files are shown only when certain conditions are met. Embodiments of the present disclosure contemplate a number of different conditions. Example conditions include determining that the user is a freemium user and has clicked on a repair tool icon, determining that the user has not tried the repair tool after a period of time, determining that the user is a beginner and has encountered problems when using the repair tool.

[0060] Accordingly, embodiments of the techniques described herein provide an improvement over conventional systems by providing content-adaptive tutorials for editing tools of a graphics editing system. For example, the editing tool can be an object removal tool or a spot healing tool that supports removing unwanted blemishes in an image. The editing tool can include a tool workflow (such as a spot healing tool workflow that automatically determines a spot mask and a repair source) having processing steps for dynamically generating a tool tutorial (such as a tool tutorial data file). Additionally, the spot healing tool workflow includes unconventional steps for automatically determining spots (such as scars and blemishes) in an image based on deep learning and skin region pruning, and steps for identifying a repair source region to repair the spot region. The generated tool tutorial is caused to be rendered to the user in different user contexts.

[0061] Exemplary Method

[0062] Reference Figure 4 、 Figure 5 and Figure 6 , Figure 4 、 Figure 5 and Figure 6 are flowcharts showing methods for providing a tool tutorial based on operating a graphics editing system in a graphics editing system using tutorial information that is dynamically integrated into a tool tutorial shell. The method can be executed using the graphics editing system described herein. In an embodiment, one or more computer storage media contain computer-executable instructions that, when executed by one or more processors, can cause the one or more processors to execute the method in the graphics editing system.

[0063] Turning Figure 4 , Figure 4It is a flowchart showing a method 400 for generating a content-adaptive tool tutorial based on operations of a graphics editing system. Initially, at step 410, an image is accessed. The image is associated with a graphics editing application. At step 420, tool parameters are generated based on performing a set of processing steps (i.e., tool workflow) of an object removal tool associated with the graphics editing application. The object removal tool is a graphics editing tool of the graphics editing application. At step 430, based on the tool parameters and the image, a tool tutorial data file is automatically generated. The automatic generation of the tutorial is based on integrating the tool parameters and the image data of the image into a tool tutorial shell associated with the object removal tool. At step 440, the tool tutorial data file is transmitted to cause the drawing of the tool tutorial data file.

[0064] Turn to Figure 5 , Figure 5 A flowchart showing a method 500 for providing a content-adaptive tool tutorial based on operations of a graphics editing system is provided. Initially, at step 510, an interface for receiving tutorial information is provided. The tutorial information includes an image associated with a graphics editing application. At step 520, the tutorial information is transmitted to cause the generation of a tool tutorial data file. The tool tutorial data file is generated based on tool parameters and the tutorial information. At step 530, the tool tutorial data file is received. At step 540, the drawing of the tool tutorial data file is caused. The drawing of the tool tutorial data file is selectively performed based on determining that at least one of a plurality of user-based selective drawing conditions has been met. Additionally, the tool tutorial data file is caused to be generated and then drawn on an interface integrated with the graphics editing application based on any of the following: determining that the user is a freemium user and the user clicks on a specific editing tool; determining that the user has not tried a specific tool after a period of time; or determining that the user faces a challenge when using a specific tool.

[0065] Turn to Figure 6 , Figure 6It is a flowchart showing a method 600 for providing a content adaptive tool tutorial based on operations of a graphics editing system. Initially, at step 610, tool parameters are generated based on a set of processing steps for performing a graphics editing tool associated with a graphics editing application. Generating tool parameters based on performing the set of processing steps includes accessing a measurable identifier of the tool parameters, where the measurable identifier corresponds to one or more steps in the set of processing steps, and determining a quantization value for the measurable identifier based on performing one or more steps on the image data of the image. At step 620, a tool tutorial shell is accessed. The tool tutorial shell adopts a portable tutorial format including chapters and sections of tutorial steps. At step 630, a tool tutorial data file is generated based on mapping the tool tutorial steps to the tool parameters and the image data of the image. Generating the tool tutorial data file is based on integrating the tool parameters and the image data of the image into a tool shell associated with the graphics editing tool.

[0066] Exemplary Operating Environment

[0067] An overview of embodiments of the present invention has been briefly described. The following describes an exemplary operating environment in which embodiments of the present invention can be implemented to provide a general context for various aspects of the present invention. First, with particular reference to Figure 7 , an exemplary operating environment for implementing embodiments of the present invention is shown and is generally designated as computing device 700. Computing device 700 is only one example of a suitable computing environment and is not intended to impose any limitation on the scope of use or functionality of the present invention. Computing device 700 should also not be construed as having any dependency or requirement on any one or combination of the illustrated components.

[0068] The present invention can be described in the general context of computer code or machine - available instructions executed by a computer or other machine, such as a personal data assistant or other handheld device, including computer - executable instructions, such as program modules. Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs specific tasks or implements specific abstract data types. The present invention can be practiced in various system configurations, including handheld devices, consumer electronics, general - purpose computers, more specialized computing devices, etc. The present invention can also be practiced in a distributed computing environment where tasks are performed by remote processing devices linked through a communication network.

[0069] Referring to Figure 7 , computing device 700 includes a bus 710 that directly or indirectly couples the following devices: a memory 712, one or more processors 714, one or more presentation components 716, input / output ports 718, input / output components 720, and an illustrative power supply 722. Bus 710 represents what may be one or more buses, such as an address bus, a data bus, or a combination thereof. AlthoughFigure 7 The various boxes are shown with lines for clarity of concept, but other arrangements of the described components and / or component functions are also contemplated. For example, a presentation component such as a display device can be considered an I / O component. Additionally, the processor has a memory. We recognize that this is the nature of the art and reiterate that Figure 7 the figures are merely illustrative of example computing devices that can be used in conjunction with one or more embodiments of the present invention. No distinction is made among categories such as "workstation," "server," "laptop computer," "handheld device," etc., because all of these are within the Figure 7 scope and refer to "computing device."

[0070] Computing device 700 generally includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 700 and includes volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0071] Computer storage media includes volatile and non-volatile removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by computing device 700. Computer storage media does not include the signal itself.

[0072] Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the foregoing should also be included within the scope of computer-readable media.

[0073] The memory 712 includes computer storage media in the form of volatile and / or non-volatile memory. The memory can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. The computing device 700 includes one or more processors that read data from various entities such as the memory 712 or the I / O component 720. The (multiple) presentation components 716 present data indications to the user or other devices. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc.

[0074] The I / O port 718 allows the computing device 700 to be logically coupled to other devices including the I / O component 720, some of which may be built-in. Illustrative components include microphones, joysticks, gamepads, satellite dishes, scanners, printers, wireless devices, etc.

[0075] Referring to the technical solution environment described herein, the embodiments described herein support the technical solutions described herein. The components of the technical solution environment may be integrated components including a hardware architecture and a software framework that support constrained computing and / or constrained query functions within a technical solution system. The hardware architecture refers to the physical components and their interrelationships, while the software framework refers to software that provides functions that can be implemented by the hardware embodied on the device.

[0076] An end-to-end software-based system can operate within system components to operate computer hardware to provide system functions. At a lower level, a hardware processor executes instructions selected from a machine language (also known as machine code or native) instruction set for a given processor. The processor recognizes native instructions and performs corresponding low-level functions related to, for example, logic, control, and memory operations. Low-level software written in machine code can provide more complex functions for higher-level software. As used herein, computer-executable instructions include any software, including low-level software written in machine code, high-level software such as application software, and any combination thereof. In this regard, system components can manage resources and provide services for system functions. Embodiments of the present invention contemplate any other variations and combinations thereof.

[0077] As an example, a technical solution system may include an API library that includes specifications for routines, data structures, object classes, and variables that can support the interaction between the hardware architecture of a device and the software framework of the technical solution system. These APIs include configuration specifications for the technical solution system such that different components therein can communicate with each other within the technical solution system as described herein.

[0078] The technical solution system may further include a machine learning system. The machine learning system may include machine learning tools and training components. The machine learning system may include machine learning tools for performing operations in different types of technical fields. The machine learning system may include pre-trained machine learning tools that can be further trained for specific tasks or technical fields. At a higher level, machine learning is a field of study that enables computers to learn without being explicitly programmed. Machine learning explores the research and construction of machine learning tools, including machine learning algorithms or models that can learn from existing data and make predictions on new data. Such machine learning tools operate by building models from example training data in order to make data-driven predictions or decisions in the form of outputs or evaluations. Although example embodiments are presented with respect to some machine learning tools, the principles presented herein can be applied to other machine learning tools. It is contemplated that different machine learning tools may be used. For example, logistic regression (LR), naive Bayes, random forest (RF), neural network (NN), matrix factorization, and support vector machine (SVM) tools can be used to solve problems in different technical fields.

[0079] Generally, there are two types of problems in machine learning: classification problems and regression problems. Classification problems (also known as categorization problems) aim to classify items into one of several class values (e.g., whether this email is spam). Regression algorithms aim to quantify certain items (e.g., by providing a value as a real number). Machine learning algorithms can provide a score (e.g., a number from 1 to 100) to quantify one or more products as matches for users in an online marketplace.

[0080] Machine learning algorithms utilize training data to find correlations between identifying features (or combinations of features) that affect the outcome. A trained machine learning model can be implemented to perform machine learning operations based on the feature combinations. The administrator of the machine learning system can also determine which of the various feature combinations are relevant (e.g., lead to a desired outcome) and which are irrelevant. Feature combinations that are determined to be (e.g., classified as) successful are input into the machine learning algorithm for the machine learning algorithm to learn which feature combinations (also known as "patterns") are "relevant" and which patterns are "insignificant". Machine learning algorithms utilize features to analyze data to generate outputs or evaluations. A feature can be an individual measurable property of an observed phenomenon. The concept of a feature is related to the concept of an explanatory variable used in statistical techniques such as linear regression. Selecting informative, discriminatory, and independent features is important for the effective operation of a machine learning system in pattern recognition, classification, and regression. Features can be of different types, such as numbers, strings, and graphics.

[0081] Machine learning algorithms use training data to find correlations between identifying features that affect an outcome or an assessment. The training data includes known data for one or more identifying features and one or more outcomes. With the training data and the identifying features, a machine learning tool can be trained. The machine learning tool determines the correlations of the features because they are related to the training data. The result of the training is a trained machine learning model. When the machine learning model is used to perform an assessment, new data is provided as input to the trained machine learning model, and the machine learning model generates an assessment as output.

[0082] Having identified the various components used herein, it should be understood that any number of components and arrangements may be employed within the scope of the present disclosure to achieve the desired functionality. For example, for clarity of concepts, the components in the embodiments depicted in the figures are shown connected by lines. Other arrangements of these and other components may also be implemented. For example, although some components are depicted as single components, many of the elements described herein may be implemented as discrete or distributed components or in combination with other components and may be implemented in any suitable combination and location. Some elements may be entirely omitted. Additionally, the various functions described herein as being performed by one or more entities may be performed by hardware, firmware, and / or software, as described below. For example, the various functions may be performed by a processor executing instructions stored in a memory. Accordingly, other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions) may be used in addition to or in place of those shown.

[0083] The embodiments described in the following paragraphs may be combined with one or more of the specifically described alternatives. In particular, in the alternatives, the claimed embodiments may include references to more than one other embodiment. The claimed embodiments may specify further limitations to the claimed subject matter.

[0084] The subject matter of embodiments of the present invention is specifically described herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter may also be embodied in other ways, to include combinations of different steps or steps similar to those described in this document in conjunction with other existing or future technologies. Additionally, although the terms "step" and / or "block" may be used herein to imply different elements of a method employed, these terms should not be construed to imply any particular order among the individual steps disclosed herein unless the order of individual steps is explicitly described.

[0085] For the purposes of this disclosure, the term "including" has the same broad meaning as the term "comprising", and the term "access" includes "receive", "reference", or "retrieve". Additionally, the term "communicate" has the same broad meaning as the terms "receive" or "transmit", which are facilitated by software- or hardware-based buses, receivers, or transmitters using the communication media described herein. Additionally, unless stated to the contrary, words such as "a" and "an" include both the plural and the singular. Thus, for example, when there is one or more features, the constraint of "feature" is satisfied. Additionally, the term "or" includes conjunctive, disjunctive, and both (thus, a or b includes a or b as well as a and b).

[0086] For the purposes of the detailed discussion above, embodiments of the present invention are described with reference to a distributed computing environment; however, the distributed computing environment described herein is merely exemplary. Components may be configured to perform novel aspects of the embodiments, where the term "configured to" may mean "programmed to" perform a specific task or implement a specific abstract data type using code. Additionally, while embodiments of the present invention may generically refer to the technical solution environments and diagrams described herein, it should be understood that the described technology may be extended to other implementation contexts.

[0087] Embodiments of the present invention have been described with respect to specific embodiments, which are illustrative in all respects and not restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from the scope of the present invention.

[0088] As can be seen from the above, the present invention is well suited to achieve all of the above objectives and purposes as well as other obvious inherent advantages of the structure.

[0089] It should be understood that certain features and subcombinations are useful and may be used without reference to other features or subcombinations. This is contemplated by the claims and within the scope of the claims.

Claims

1. A computer-implemented method, the method comprising: Accessing an image, wherein the image is associated with a graphics editing application; Determining whether a spot healing tool workflow is applicable to the image based on the image, wherein the spot healing tool workflow includes a set of processing steps of a spot healing tool, and wherein when the spot healing tool workflow is not applicable to the image, a content-adaptive tutorial is not generated for the image, and when the spot healing tool workflow is applicable to the image, the content-adaptive tutorial is generated for the image; Wherein determining whether the spot healing tool workflow is applicable to the image is based on analyzing whether tool parameters with placeholders in a tool tutorial shell can be generated for the image, the tool parameters being associated with detecting a face, performing skin area trimming, and detecting one or more spots; Based on determining that the spot healing tool workflow is applicable to the image, generating the tool parameters based on performing the set of processing steps of the spot healing tool workflow; Wherein generating the tool parameters is based on the one or more spots and one or more repair source regions of the image; Automatically generating a tool tutorial data file associated with the spot healing tool based on the tool parameters and the image, wherein automatically generating the tutorial data file is based on integrating the tool parameters and the image data of the image into the tool tutorial shell, the tool tutorial shell supporting the generation of a content-adaptive tool tutorial for specific image data and their corresponding tool parameters; and Transmitting the tool tutorial data file to cause the drawing of the tool tutorial data file.

2. The method according to claim 1, wherein the tool tutorial shell includes different parts corresponding to tutorial steps, the tutorial steps being integrated with the tool parameters and the image data of the image, wherein the tool parameters are measurable identifiers having quantization values for one or more steps in the set of processing steps of the tool workflow for a graphics editing tool.

3. The method according to claim 1, wherein generating the tool parameters based on the set of tool processing steps includes: Accessing a measurable identifier of the tool parameters, wherein the measurable identifier corresponds to one or more steps in the set of processing steps; And Determining a quantization value for the measurable identifier based on performing the one or more steps on the image data of the image.

4. The method according to claim 1, wherein the set of tool processing steps further includes: Detecting the face in the image; Identifying a plurality of facial landmark points of the face; Performing skin area trimming on the image based on the face and the plurality of landmarks; Detecting the one or more spots on the image; Generating a mask for the one or more spots of the image; Identifying the one or more repair source regions of the image, wherein the one or more repair source regions support repairing the corresponding one or more spots of the image; And Generate the tool parameters based on the one or more stains and the one or more repair source regions of the image, where the tool parameters are repair parameters for the stain repair tool.

5. The method according to claim 1, wherein automatically generating the tool tutorial data file is based on integrating the tool parameters and the image data of the image into the tool tutorial shell associated with the graphics editing tool.

6. The method according to claim 1, wherein integrating the tool parameters and the image into the tool tutorial shell further comprises: Accessing the tool tutorial shell, where the tool tutorial shell is a portable tutorial format including chapters or sections of tutorial steps; And Generating the tool tutorial data file based on mapping the tutorial steps to the tool parameters and the image data of the image.

7. The method according to claim 1, wherein causing the rendering of the tool tutorial data file is based on: selectively rendering the tutorial data file based on determining that at least one of a plurality of user-based selective rendering conditions has been met; or wherein the tool tutorial data file is caused to be generated and then rendered on an interface integrated with the graphics editing application based on: Determining that the user is a freemium user and the user clicks on a specific editing tool; Determining that the user has not tried a specific tool after a period of time; or Determining that the user faces challenges when using a specific tool.

8. A computerized system, comprising: Accessing an image, where the image is associated with a graphics editing application; Determining whether a stain repair tool workflow is applicable to the image based on the image, where the stain repair tool workflow includes a set of processing steps of the stain repair tool, and when the stain repair tool workflow is not applicable to the image, no content-adaptive tutorial is generated for the image, and when the stain repair tool workflow is applicable to the image, generating the content-adaptive tutorial for the image, where determining whether the stain repair tool workflow is applicable to the image is based on analyzing whether tool parameters with placeholders in the tool tutorial shell can be generated for the image, the tool parameters being associated with detecting faces, performing skin area trimming, and detecting one or more stains; Based on determining that the stain repair tool workflow is applicable to the image, generating the tool parameters based on performing the set of processing steps of the stain repair tool workflow; where generating the tool parameters is based on the one or more stains and one or more repair source regions of the image; Automatically generating a tool tutorial data file associated with the stain repair tool based on the tool parameters and the image, where automatically generating the tutorial data file is based on integrating the tool parameters and the image data of the image into the tool tutorial shell, the tool tutorial shell supporting the generation of a content-adaptive tool tutorial specific to the image data and their corresponding tool parameters; and Transmit the tool tutorial data file to cause the rendering of the tool tutorial data file.

9. The system according to claim 8, wherein the tool tutorial shell includes different parts corresponding to tutorial steps, the tutorial steps being integrated with the tool parameters and the image data of the image, wherein the tool parameters are measurable identifiers having a quantization value for one or more steps of the set of processing steps of the tool workflow for the object removal tool.

10. The system according to claim 8, wherein generating the tool parameters based on performing the set of processing steps includes: Accessing a measurable identifier of the tool parameters, wherein the measurable identifier corresponds to one or more steps of the set of processing steps; And Determining a quantization value for the measurable identifier based on performing the one or more steps on the image data of the image.

11. The system according to claim 8, wherein automatically generating the tool tutorial data file is based on integrating the tool parameters and the image data of the image into the tool tutorial shell associated with the object removal tool, wherein the tool tutorial shell includes different parts corresponding to tutorial steps integrated with the tool parameters and the image data of the image.

12. The system according to claim 8, wherein integrating the tool parameters and the image into the tool tutorial shell further includes: Accessing the tool tutorial shell, wherein the tool tutorial shell is a portable tutorial format including chapters or parts of tutorial steps; And Generating the tool tutorial data file based on mapping the tutorial steps to the tool parameters and the image data of the image.

13. The system according to claim 8, wherein the rendering of the tool tutorial data file is based on: selectively rendering the tutorial data file based on determining that at least one of a plurality of user-based selective rendering conditions has been satisfied; Or wherein the tool tutorial data file is caused to be generated and then rendered on an interface integrated with the graphics editing application based on: Determining that the user is a freemium user and the user clicks on a specific editing tool; Determining that the user has not tried a specific tool after a period of time; Or Determining that the user faces challenges when using a specific tool.

14. One or more computer storage media having computer-executable instructions embodied thereon that, when executed by a computing system having a processor and a memory, cause the processor to: Access an image, wherein the image is associated with a graphics editing application; Determine whether a spot healing tool workflow is applicable to the image based on the image, wherein the spot healing tool workflow includes a set of processing steps of a spot healing tool, wherein when the spot healing tool workflow is not applicable to the image, a content-adaptive tutorial is not generated for the image, and when the spot healing tool workflow is applicable to the image, the content-adaptive tutorial is generated for the image, wherein determining whether the spot healing tool workflow is applicable to the image is based on analyzing whether tool parameters with placeholders in a tool tutorial shell can be generated for the image, the tool parameters being associated with detecting a face, performing skin area trimming, and detecting one or more spots; Based on determining that the workflow of the spot healing tool is applicable to the image, generating the tool parameters based on performing the set of processing steps of the workflow of the spot healing tool, wherein generating the tool parameters is based on the one or more spots and one or more source regions for healing of the image; automatically generating a tool tutorial data file associated with the spot healing tool based on the tool parameters and the image, wherein automatically generating the tutorial data file is based on integrating the tool parameters and the image data of the image into the tool tutorial shell, and the tool tutorial shell supports generating a content-adaptive tool tutorial for the image-specific image data and their corresponding tool parameters; and transmitting the tool tutorial data file to cause the rendering of the tool tutorial data file.

15. The medium according to claim 14, wherein causing the processor to access the image includes accessing the image based on a user selection to initiate an object removal tool, wherein the object removal tool is an advanced graphics editing tool for the graphics editing application, and the graphics editing application is provided as a freemium service including at least one free graphics editing tool and the advanced graphics editing tool.

16. The medium according to claim 14, wherein the tool tutorial shell includes different parts corresponding to tutorial steps, and the tutorial steps are integrated with the tool parameters and the image data of the image, wherein the tool parameters are measurable identifiers, and the measurable identifiers have quantization values for one or more steps in the set of processing steps of the tool workflow for the object removal tool.

17. The medium according to claim 14, wherein causing the processor to generate the tool parameters based on performing the set of processing steps includes: accessing the measurable identifier of the tool parameter, wherein the measurable identifier corresponds to one or more steps in the set of processing steps; and determining a quantization value for the measurable identifier based on performing the one or more steps on the image data of the image.

18. The medium according to claim 14, wherein the set of processing steps includes: detecting the face in the image; identifying a plurality of facial landmark points of the face; performing skin region trimming on the image based on the face and the plurality of landmarks; detecting the one or more spots on the image; generating a mask for the one or more spots of the image; identifying the one or more source regions for healing of the image, wherein the one or more source regions for healing support healing the corresponding one or more spots of the image; and generating the tool parameters based on the one or more spots and the one or more source regions for healing of the image, wherein the tool parameters are healing parameters for the spot healing tool.

19. The medium according to claim 14, wherein integrating the tool parameters and the image into the tool tutorial shell further includes: Access the tool tutorial shell, where the tool tutorial shell is a portable tutorial format that includes chapters or sections of tutorial steps; And Generate the tool tutorial data file based on mapping the tutorial steps to the tool parameters and the image data of the image.

20. The medium according to claim 14, wherein causing the processor to cause the rendering of the tool tutorial data file is based on: selectively rendering the tutorial data file based on determining that at least one of a plurality of user-based selective rendering conditions has been satisfied; or wherein the tool tutorial data file is caused to be generated and then rendered on an interface integrated with the graphics editing application based on: Determine that the user is a freemium user and the user clicks on a specific editing tool; Determine that the user has not tried a specific tool after a period of time; or Determine that the user faces challenges when using a specific tool.

Images