Service method and device based on matting, electronic device and storage medium

Abstract

Embodiments of the present application provide a service method and device based on matting, electronic equipment and storage medium. In the embodiments of the present application, the matting service is combined with application recommendation, a matting image is obtained by performing matting processing on an original image, and an icon of a recommended application is displayed; and in response to a user operation, an interface of the recommended application can be displayed, and an image corresponding to the matting image is displayed on the application interface; in addition, different precision matting images can be provided for different applications, so that different applications can provide corresponding services for users in a timely manner, improve service efficiency, and the whole process is simple to operate, has high operation efficiency, and improves the user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a service method, apparatus, electronic device and storage medium based on image matting. Background Technology

[0002] With the development of image processing technology, users often need to provide services based on images in their daily activities, such as image-based information search services. In some scenarios, it may not be suitable to provide services directly based on the original image, which requires processing the original image into a target image that meets the requirements. To address this, some user-defined image cutout applications have emerged. Users can upload original images to these applications, which can then automatically extract the regions of interest from the original image to generate the target image. The target image can be saved to a gallery, and when the user needs it, it can be retrieved from the gallery, and the corresponding service can be provided based on it. However, the quality of the target images provided by existing user-defined image cutout applications may not meet the service requirements, and the service process is relatively cumbersome and inefficient. Summary of the Invention

[0003] This application provides a method, apparatus, electronic device, and storage medium for image matting services to meet users' increasingly demanding image matting requirements.

[0004] In a first aspect, embodiments of this application provide a method for providing image matting services, applied to an electronic device. The method includes: displaying an original image; responding to a user inputting a first user operation on the original image, displaying a first matted image, the first matted image being obtained by performing matting processing on the original image; responding to a user inputting a second user operation on the first matted image, displaying icons of a first application and a second application in a target area; responding to a user inputting a third user operation in the target area, displaying a first interface of the first application, the first interface including a first image corresponding to the first matted image; responding to a user inputting a fourth user operation in the target area, displaying a second interface of the second application, the second interface including a second image corresponding to the first matted image; the first image and the second image have different image precision, for example, the image precision of the second matted image is greater than that of the first matted image.

[0005] In the embodiments provided in the first aspect, the image cutout service is combined with application recommendation. By performing image cutout processing on the original image to obtain a cutout image, the icons of different applications are displayed. In response to user operations for different applications, the interfaces of different applications can be displayed, and cutout images of different precision corresponding to the cutout image are displayed on the interfaces of different applications respectively. This makes it convenient for different applications to provide corresponding services to users in a timely manner, improves service efficiency, and the whole process is simple to operate, highly efficient, and enhances the user experience.

[0006] In one possible design, the target area includes a first area and a second area, which are non-overlapping regions within the target area. The third user operation is to drag the first cutout image into the first area and release it; the fourth user operation is to drag the first cutout image into the second area and release it.

[0007] Optionally, the first region is the region where the icon of the first application is located, and the second region is the region where the icon of the second application is located.

[0008] The entire process, from the user dragging the subject cutout image to providing it to the corresponding application, is continuous. "Continuous" means the user doesn't need to copy, paste, or switch applications; their fingers never leave the screen, making the operation simple and quick. In one possible design, the user inputs a second user action on the first cutout image, which involves dragging the first cutout image into a third area; where the target area and the third area partially or completely overlap.

[0009] In one possible design approach, displaying the first cutout image includes: overlaying a first outline onto the original image, wherein the image within the range of the first outline in the original image is the first cutout image. Displaying the first outline serves as a notification to the user that the cutout operation has been performed, allowing the user to perform subsequent user actions on the first cutout image.

[0010] In one possible design approach, the first application requires image precision higher than a set image precision threshold or does not have low latency requirements, while the second application requires image precision lower than or equal to the set image precision threshold or has low latency requirements; wherein, the image precision of the first image is greater than that of the second image. For the application requiring image precision higher than the set image precision threshold, image filtering is performed on the first cutout image to improve image precision and meet the first application's requirements, enabling the first application to display service information to the user more accurately. For the second application, which does not require image precision higher than the set image precision threshold, the first cutout image is directly provided to the second application, facilitating timely display of service information to the user and improving service efficiency. Prioritizing low latency requirements over image precision requirements, directly providing the first cutout image to the second application helps ensure the application's timely service delivery to the user and guarantees a good user experience.

[0011] In one possible design, the electronic device can also respond to a fifth user operation inputting on the original image to display a second cutout image, which is of a different category than the first cutout image; and respond to a second user operation inputting on the second cutout image to display an icon of a third application in the target area; the icon of the third application is different from the icons of the first and second applications. This allows for the recommendation of different types of applications for different categories of cutout images to adapt to different usage needs and improve the user experience.

[0012] Optionally, the first application and the second application are associated with the category of the first cutout image, and the third application is associated with the category of the second cutout image.

[0013] Alternatively, the category information of the first cutout image or the category information of the second cutout image may be displayed on the electronic device.

[0014] In one possible design approach, before displaying the icons of the first and second applications in the target area, the first and second applications can be determined from the candidate applications based on at least one of the following information: attribute information of the first cutout image, usage frequency of the candidate applications, running status, recent usage time, and application type. The candidate applications include applications on electronic devices and / or applications on other devices communicatively connected to the electronic device. This allows for personalized recommendations of different applications for different cutout objects or different users, improving the flexibility of application recommendations and meeting user needs.

[0015] In one possible design, if the first user action is implemented as a click, pinch, or press operation on the original image, the first cutout image is displayed in response to the user's click, pinch, or press operation on the original image.

[0016] Furthermore, in response to a user's click, pinch, or press operation on the original image, the first cutout image is highlighted in the original image to remind the user that the first cutout image can be used for subsequent user operations.

[0017] In one possible design approach, where the original image contains multiple objects, the target cutout object can be the object among the multiple objects that corresponds to the screen position of the click, pinch, or press operation.

[0018] In one possible design, a first image and first service information associated with the first image are displayed on a first interface; a second image and second service information associated with the second image are displayed on a second interface.

[0019] In one possible design, in response to the user's close operation on the first interface, the first interface is closed, and the interface containing the icons of the first application and the second application is returned; in response to the user entering a fourth user operation in the target area, the second interface of the second application is displayed.

[0020] In another possible design, the original image is displayed; in response to a first user operation on the original image, a first cutout image is displayed; and in response to a second user operation on the first cutout image, icons of the first application and the second application are displayed in the target area; and in response to a fourth user operation in the target area, a second interface of the second application is displayed.

[0021] In one possible design, the third user action is implemented as a click, pinch, or press operation by the user on the icon of the first application. The electronic device can respond to the third user action by displaying the first interface of the first application. The fourth user action is implemented as a click, pinch, or press operation by the user on the icon of the second application. The electronic device can respond to the fourth user action by displaying the second interface of the second application.

[0022] Secondly, embodiments of this application provide another image matting-based service method applied to an electronic device. The method includes: displaying an original image; in response to a first user operation input to the original image, performing image matting processing on the original image and displaying a first matted image obtained by performing image matting processing on the original image; in response to a second user operation input to the first matted image, displaying icons of a first application and a second application in a target area; and in response to a third user operation input to the first application, displaying a first interface of the first application, the first interface including a third matted image obtained by performing image filtering processing on the first matted image.

[0023] In one possible design, after the icons of the first application and the second application are displayed in the target area, the electronic device responds to a fourth user operation input by the user for the second application and displays a second interface of the second application, the second interface including the first cutout image.

[0024] In the embodiments provided in the second aspect, the image cutout service is combined with application recommendations. By processing the original image to obtain a cutout image, icons of different applications are displayed. Furthermore, in response to user actions for different applications, the interfaces of different applications can be displayed. For applications with high image precision requirements, the cutout image can be filtered to obtain a higher-precision image and displayed on the application's interface, facilitating the application to provide users with higher-quality services. For applications with lower image precision requirements, the cutout image can be directly displayed on the application's interface, allowing the application to provide timely services to users and improving service efficiency. Regardless of the application, the entire process is simple to operate, highly efficient, and enhances the user experience.

[0025] In one possible design approach, in response to a first user action on the input of the original image, a cutout region in the original image is determined, the cutout region including the target cutout object, for example, the area of ​​the cutout region is greater than a% of the original image, a% can be 1%, to avoid poor cutout effect caused by the cutout region being too small; the cutout region is cropped from the original image as the first cutout image.

[0026] In one possible design approach, in response to a second user operation inputting a first cutout image, a first application and a second application are determined from the candidate applications based on at least one of the following: attribute information of the first cutout image, usage frequency of candidate applications, running status, recent usage time, and application type. Icons of the first application and the second application are then displayed in the target area. The candidate applications include applications on electronic devices and / or applications on other devices communicatively connected to the electronic device. Recommending applications for the target cutout object from multiple dimensions enhances the user experience.

[0027] In one possible design approach, before performing image filtering on a cutout image to obtain a second cutout image, a third user operation input by the user for the first application can be performed to determine whether the first application has low latency requirements. If it is determined that the first application does not have low latency requirements, the image filtering operation on the first cutout image is performed to obtain a third cutout image, thereby improving the quality of the cutout image and meeting the user's usage needs.

[0028] In one possible design approach, the category information of the target object to be cut out in the original image is identified; if it is determined based on the category information of the target object that the target object to be cut out does not contain sensitive information, the first cut-out image is displayed to improve the security of the cut-out.

[0029] Furthermore, when the target object to be matted is the target subject object in the original image, the original image is input into the image segmentation model for image segmentation to obtain the mask region of the target subject object; based on the mask region of the target subject object, the target local image corresponding to the target subject object is cropped from the original image; the target local image is input into the image classification model to perform image classification on the target local image to obtain the category information of the target subject object.

[0030] Thirdly, embodiments of this application provide another image matting-based service method applied to an electronic device. The method includes: displaying an original image; performing image matting processing on the original image in response to a first user operation input to the original image, and displaying a first matted image obtained by performing image matting processing on the original image; displaying a target area in response to a second user operation input to the first matted image; and displaying a first interface of the first application in response to a third user operation in response to a user selecting a first application in the target area. The first interface includes a third matted image obtained by performing image filtering processing on the first matted image.

[0031] In the embodiment provided in the third aspect, the image cutout service is combined with application recommendation. The cutout image is obtained by performing image cutout processing on the original image and the target area is displayed. The target area is used to carry the application icon. In response to the user's operation of selecting an application in the target area, the cutout image is filtered to obtain a higher precision image and displayed on the interface of the application. This makes it easier for the application to provide users with higher quality services. The whole process is simple to operate, has high operating efficiency, and improves the user experience.

[0032] Fourthly, embodiments of this application provide another method for image matting, applied to an electronic device. The method includes: displaying an original image; performing image matting processing on the original image in response to a first user operation input to the original image, and displaying a first matted image obtained by performing image matting processing on the original image; displaying a target area in response to a second user operation input to the first matted image; and displaying a second interface, the second interface including the first matted image, in response to a fourth user operation in which the user selects a second application in the target area.

[0033] In the embodiment provided in the fourth aspect, the image cutout service is combined with application recommendation. The cutout image is obtained by performing image cutout processing on the original image and the target area is displayed. The target area is used to carry the application icon. In response to the user's operation of selecting an application in the target area, the cutout image is directly displayed on the application interface, which makes it convenient for the application to provide services to the user in a timely manner, improves service efficiency, and the whole process is simple to operate, has high operation efficiency, and improves the user experience.

[0034] Fifthly, embodiments of this application also provide an electronic device, including: a memory, a processor, and a display screen; the memory is used to store a computer program, and the display screen is used to display an image or an application interface; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the image matting service method provided in embodiments of this application.

[0035] Sixthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps of the image matting service method provided in embodiments of this application.

[0036] Understandably, the beneficial effects of the electronic device provided in the fifth aspect and the computer-readable storage medium provided in the sixth aspect can be referenced to the beneficial effects of the first aspect and any of its possible design embodiments, which will not be repeated here. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0038] Figures 1A-1B A schematic diagram illustrating the display of an original image in an application interface of a "Gallery" as an exemplary embodiment of this application;

[0039] Figures 2A-2E A schematic diagram of an application interface for displaying the original image after a "camera" takes a picture, provided as an exemplary embodiment of this application;

[0040] Figures 3A-3E This embodiment of the present application provides a schematic diagram of the interface change process of an electronic device when a subject object contained in the original image is cut out and an application is recommended.

[0041] Figures 4A-4D This embodiment of the present application provides a schematic diagram illustrating the interface changes of an electronic device when extracting text objects contained in an original image and recommending applications.

[0042] Figure 5A schematic diagram of a thermal zone is provided for an exemplary embodiment of this application;

[0043] Figure 6 A schematic diagram of a target area provided for an exemplary embodiment of this application;

[0044] Figure 7A A flowchart illustrating a service method based on image matting provided for an exemplary embodiment of this application;

[0045] Figure 7B A flowchart illustrating another image matting-based service method provided for an exemplary embodiment of this application;

[0046] Figure 8 A structural block diagram of an electronic device provided as an exemplary embodiment of this application;

[0047] Figure 9 This is an exemplary embodiment of the present application providing an interactive sequence diagram based on image matting, in which a target image matting object is implemented as a target subject object;

[0048] Figure 10 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] First, the terminology used in the embodiments of this application will be explained. It should be understood that this explanation is for the purpose of making the embodiments of this application clearer and does not necessarily constitute a limitation on the embodiments of this application.

[0051] 1. Target object to be cut out.

[0052] The original image includes at least one object, which can be a main subject such as a person, animal, building, vehicle, or plant, or a text object such as text, letters, or symbols, either alone or in combination. For ease of distinction and description, the object in the original image that can be used for the cutout operation is called the cutout object. The cutout object can be any object in the original image.

[0053] Alternatively, one way to determine the object to be cut out is:

[0054] The cutout object is automatically determined based on the proportion of the object in the original image. For example, an object whose area proportion in the original image is greater than a set area proportion threshold can be used as the cutout object. For example, if the set area proportion threshold is 1%, an object whose area proportion in the original image is greater than 1% can be used as the cutout object. The number of cutout objects determined in this way can be one or more.

[0055] In this embodiment, a matting-based service method can be executed on any matting object. For ease of distinction and description, in subsequent embodiments, the matting object to which the actual matting operation is performed is defined as the "target matting object". The target matting object can be any matting object in the original image, and can be one or more, depending on the application scenario.

[0056] The methods for determining the target matting object include: performing object detection on the original image to obtain the category and location of the matting objects contained in the original image; and selecting one, two, or more matting objects as the target matting object based on the categories and locations of multiple matting objects. For example, matting objects of a specific category (e.g., selecting objects of a unique category existing in the image) and / or matting objects with prominent positions (e.g., targets located in or near the center of the image) can be selected as the target matting object. Alternatively, a first user operation, such as a click, pinch, or press operation, can be performed on the original image, and the matting object corresponding to the screen position of the click, pinch, or press operation can be selected as the target matting object.

[0057] For object detection, various neural network models for object detection can be called from the system library. These can be traditional neural network models or self-developed neural network models, and there are no restrictions on which one is used.

[0058] 2. Hot zone and target area.

[0059] The hot zone and the target area are pre-defined areas on the screen of an electronic device. The target area is mainly used to display the icon of the application involved in the image matting service method; in subsequent embodiments, the target area may also be referred to as the icon area. The hot zone is mainly used to control the display of the target area. For example, based on the user's action of dragging the first matted image into the hot zone, it is determined whether to display the application icon in the target area. If the user drags the first matted image into the hot zone, the electronic device displays the application icon in the target area. Otherwise, the application icon is not displayed in the target area.

[0060] The target area can completely overlap with the hot area, or it can partially overlap with the hot area. For example, the hot area is larger than the target area, so the target area is located inside the hot area; or, the hot area can be a region inside the target area; or, the hot area and the target area have an overlapping area, and the hot area and the target area each have their own independent regions.

[0061] In one optional embodiment, the hot zone includes a right-side hot zone and / or a left-side hot zone. The right-side hot zone can be a touch area defined by the right edge of the screen, and the left-side hot zone can be a touch area defined by the left edge of the screen. The target area includes a right-side target area and a left-side target area. The right-side target area is an area extending from the right edge of the screen to the other side, and the left-side target area is an area extending from the left edge of the screen to the other side. Neither the hot zone nor the target area is limited in shape; for example, it can be a rectangular area, or a circular, elliptical, or other shaped area. Furthermore, the hot zone and the target area can have the same or different shapes.

[0062] For example, Figure 5 An example of a right-side hotspot and / or a left-side hotspot is shown, where the screen width is x, and the right-side or left-side hotspot can be a rectangular area with a width of 1 / 10x. For example, when the right-side hotspot is a rectangular area with a width of 1 / 10x, the target area on the right can be a rectangular area with a width of 1 / 8x, 1 / 10x, or 1 / 12x, etc. Figure 5 The width of the target area on the right and left is 1 / 8x, represented by dashed lines, in the illustration, but it is not limited to this. Additionally, this application exemplarily shows a schematic diagram of the target area, such as... Figure 3C The illustration uses the target area as an icon area as an example. Figure 6 Another target area is illustrated as an example. Figure 3C and Figure 6 The illustration uses a rectangular target area as an example, but it is not limited to this.

[0063] In cases where the hot zone includes the right-side hot zone, the second user operation input by the user on the touch object (e.g., the first cutout image) corresponds to the location of a screen touch point. The electronic device can determine whether the touch object (e.g., the first cutout image) is close to the right-side hot zone by using the distance information between the location of the screen touch point and the right-side hot zone. Furthermore, if the distance information is less than or equal to 0, it indicates that the current touch point is located at the edge or inside the right-side hot zone, and the electronic device displays the application's icon in the target area.

[0064] In one optional embodiment, the target area may include a first area and a second area, wherein the first area and the second area are non-overlapping areas within the target area. Optionally, the first area is the area where the icon of a first application is located, and the second area is the area where the icon of a second application is located. Furthermore, in addition to the first and second areas, the target area may also include more areas, with different areas within the target area used to carry icons for different applications.

[0065] In this embodiment, the first application and the second application refer to applications with different requirements for the precision of the cutout image. Depending on the application requirements, in one example, in a scheme combining cutout service and application recommendation, only the first application can be recommended, in which case the target area can display all icons of the first application, and the number of the first application can be one or more; or, in another example, in a scheme combining cutout service and application recommendation, only the second application can be recommended, in which case the target area can display all icons of the second application, and the number of the second application can be one or more; or, in yet another example, in a scheme combining cutout service and application recommendation, both the first and second applications can be recommended simultaneously, in which case the target area can also display icons of both the first and second applications simultaneously, and the number of the first and second applications is not limited. It should be noted that the number of the first and second areas will also differ depending on the number of the first and second applications, with one application icon corresponding to one area.

[0066] In this embodiment, taking a first application and a second application as an example, the target area includes a first area where the icon of the first application is located and a second area where the icons of the second application are located. Furthermore, for ease of distinction and description, in some embodiments, the hot zone is also referred to as a third area; the third area and the hot zone are different names for the same area.

[0067] 3. Explanation regarding the first and second applications.

[0068] The image matting service method provided in this application involves a first application and a second application. The first application and the second application refer to applications with different requirements for the precision of the matted image. Relatively speaking, the first application is an application with relatively high requirements for the precision of the matted image; the second application is an application with relatively low requirements for the precision of the matted image.

[0069] In an optional embodiment, for any application, it can be distinguished as a first application or a second application by determining whether the image precision required by the application is higher than a set image precision threshold. In this embodiment, the first application is an application with high-precision image cutout requirements, demanding image precision higher than the set image precision threshold. For example, the first application may be a save or share application, such as a third-party instant messaging tool or a photo library. In contrast, the second application is an application without high-precision image cutout requirements, demanding image precision lower than or equal to the set image precision threshold. For example, the second application is an image search application, such as a browser or shopping software.

[0070] In another optional embodiment, for any application, it can be distinguished as either a first application or a second application by determining whether the image precision required by the application is higher than a set image precision threshold. If it is higher than the set image precision threshold, it can be further determined whether the application has latency requirements. In this embodiment, the first application is an application that requires image precision higher than the set image precision threshold and does not have low latency requirements; the second application is an application that requires image precision higher than the set image precision threshold and has low latency requirements, and an application that requires image precision lower than or equal to the set image precision threshold. Whether an application has low latency requirements can be determined by whether the latency required by the application is less than the set latency threshold. If the latency required by the application is less than the set latency threshold, the application is considered to have low latency requirements; otherwise, the application does not have low latency requirements. The set latency threshold can be flexibly determined according to application needs. Applications with low latency requirements can be safety-assisted driving applications, transaction applications, or payment applications, etc.

[0071] In another alternative embodiment, for any application, it can be distinguished as either a first application or a second application by determining whether the application has low latency requirements. In this embodiment, the first application is an application without low latency requirements, and the second application is an application with low latency requirements.

[0072] 4. Explanation of image precision regarding image matting.

[0073] Image precision refers to the fineness of image quality. In various embodiments of this application, both objective and subjective methods can be used to measure the precision of the cutout image. The objective method refers to determining the image precision by comparing the cutout image with the original image from at least one objective indicator dimension.

[0074] An objective approach refers to determining the accuracy of image matting by comparing the quality of the matted image with the original image using at least one objective metric. This objective metric may include, but is not limited to, Mean Squared Error (MSE), Sum of Absolute Differences (SAD), Intersection Over Union (IOU), or Grad. MSE is calculated by summing and averaging the squared differences between the true values ​​(pixel values ​​in the original image) and the observed values ​​(pixel values ​​in the matted image). A smaller MSE value indicates higher matting accuracy. MAE calculates the average of the absolute differences between the observed and true values; a smaller MAE value indicates higher matting accuracy. SAD calculates and sums the absolute differences between corresponding pixels in the matted and original images, determining which part of the original image is most similar to the matted image. In image processing, the direction of the gradient represents the direction of image edges, while the magnitude of the gradient represents the intensity of the edges in the matted image. Therefore, by calculating the gradient of the cutout image, edge information in the cutout image can be extracted, thereby determining the image accuracy of the cutout image.

[0075] Subjective method: This method determines the image precision based on the subjective perception of the cut-out image through visual inspection. For example, the more pixel information at the image edges, the higher the image precision; the more missing information at the image edges, the lower the image precision.

[0076] In this application embodiment, the original image can be displayed on the screen of an electronic device to execute the image matting service method provided in this application embodiment on the original image. The user interface for displaying the original image will vary depending on the application scenario. For example, in a gallery or photo-based application scenario, the user interface for displaying the original image could be the application interface of the "Gallery" app on the electronic device, the application interface after taking a photo using the "Camera" app, or the application interface of other applications; this application does not limit this.

[0077] Figure 1A-1B This example demonstrates how to display the original image in the "Gallery" application interface.

[0078] Figure 1A The user interface 01 shown is the desktop of an electronic device exemplarily provided in this application embodiment. The user interface 01 may include icons for at least one application, such as weather, calendar, email, settings, app store, notes, gallery, phone, SMS, browser, and camera. The positions of the application icons and their corresponding names can be adjusted according to user preferences, and this application does not impose any limitations on this.

[0079] It should be noted that, Figure 1A The schematic diagram of the interface of the electronic device shown is an exemplary demonstration of the embodiments of this application. The schematic diagram of the interface of the electronic device may also be in other styles, and this application does not limit it.

[0080] like Figure 1A As shown, when the electronic device detects a user action to open the gallery 10, it can display the following in response to the user action: Figure 1B The user interface 02 shown is an application interface of the gallery application exemplarily provided in this embodiment of the application. The user interface 02 includes a function control 121 and a display area 122, wherein the function control 121 is used to open settings; and the display area 122 is used to display thumbnails of images or videos in the gallery. It should be understood that the user interface 02 may also include other controls, such as... Figure 1B The photo control, album control, time control, and discovery control shown, as well as controls not shown, are not limited in this application embodiment.

[0081] like Figure 1B As shown, when the electronic device detects a user operation on any image in the display area 122, it responds to the user operation by using that image as the original image and displaying it on the "Gallery" application interface. This facilitates the subsequent execution of the image matting service method provided in this embodiment on the original image. The "Gallery" application interface displaying the original image may be as described in subsequent embodiments. Figure 3A or Figure 4A The user interface.

[0082] Figure 2 illustrates an example of an application interface that displays the original image after taking a photo using a "camera".

[0083] Figure 2A An exemplary diagram of the user interface of an electronic device is shown. The user turns on their electronic device, causing the screen to display the device's desktop, i.e., user interface 03. For example... Figure 2A As shown, the user interface 03 may include icons for at least one application, such as weather, calendar, email, settings, app store, notes, gallery, phone, SMS, browser, and camera 310. The positions of the application icons and their corresponding names can be adjusted according to user preferences, and this application does not impose any limitations on this.

[0084] It should be noted that, Figure 2A The schematic diagram of the interface of the electronic device shown is an exemplary demonstration of the embodiments of this application. The schematic diagram of the interface of the electronic device may also be in other styles, and this application does not limit it.

[0085] exist Figure 2A In the user interface 03, the user can click on the camera 310. Correspondingly, after receiving the operation applied to the camera 310, the electronic device can display... Figure 3B The user interface shown is 04.

[0086] Figure 2B This is an example of a shooting interface shown in an embodiment of this application. For example... Figure 2B As shown, the user can take pictures using the electronic device, meaning the electronic device is currently in photo mode.

[0087] In user interface 04, the electronic device can display Figure 2B The specific controls or switches are explained below:

[0088] The shooting mode menu 322 may include options for various camera modes such as portrait 3221, video 3222, photo 3223, night scene 3224, and more 3225. Different camera modes can achieve different shooting functions. The camera mode pointed to by the "triangle" in the shooting mode menu 322 is used to indicate the initial or user-selected camera mode, such as... Figure 2B As shown, the triangle points to "Take a picture," indicating that the camera is currently in photo mode.

[0089] Preview area 320 is used to display the image captured by the camera.

[0090] Small window 323 is used for users to view the photos and videos that have been taken.

[0091] The shooting control 324 is used to respond to user operations, enabling the electronic device to capture images or record videos.

[0092] The camera switching control 325 is used to switch the camera that captures images between the front camera and the rear camera.

[0093] The shooting function switch 326 may include a smart vision 3261, an artificial intelligence (AI) mode switch 3262, a flash switch 3263, a filter switch 3264, and a settings control 3265. Specifically, the smart vision 3261, in response to a trigger operation, enters... Figure 2C The user interface 16 of the intelligent vision system is shown.

[0094] The shooting function switch 326 may also include an AI mode switch 3262, used to turn the automatic recognition mode function on or off. A flash switch 3263 is used to turn the flash on or off. A filter switch 3264 is used to turn the filter on or off. A settings control 3265 is used to set various parameters when acquiring images.

[0095] Figure 2C The user interface 16 shown includes controls and switches, as detailed below:

[0096] Preview area 161 is used to display the image captured by the camera, i.e., a photo preview.

[0097] Function control 162 is used to open settings;

[0098] The function menu 163 includes: text extraction 1631, object recognition 1632, barcode scanning 1633, image cutout 1634, and document scanning 1635.

[0099] The small window 164 is used for users to view the photos and videos that have been taken.

[0100] The shooting control 165 is used to respond to user operations, enabling the electronic device to capture images or record videos.

[0101] In this process, the user clicks the shooting control 165 in the user interface 16 to capture the original image and display it. Figure 2D The user interface 17 is shown. User interface 17 displays the original image and various functional controls required for operations on the original image, such as controls for text extraction, object recognition, barcode scanning, image cutout, and document scanning. It should be noted that user interface 17 may or may not display the aforementioned functional controls. If the corresponding functional controls are not displayed, the user can activate the corresponding functions through touch operations, voice, or other means; this is not limited. Figure 2D The illustration uses the display of relevant functional controls as an example. Figure 2D Based on the user interface 17 shown, the user selects the "cutout" control in the user interface 17, and the electronic device performs a cutout operation on the original image, displaying the first cutout image, such as... Figure 2E User interface 18. Figure 2D The user interface 17 shown is Figure 2E The user interface 18 shown differs primarily in the objects displayed in the display area. User interface 17 displays the original image, while user interface 18 displays the first cutout image obtained by performing a cutout operation on the original image. Furthermore, user interface 18 also displays category information corresponding to the first cutout image, illustrated using a small icon of the cutout object as an example.

[0102] In this application embodiment, the method of displaying the first cutout image is not limited. In an optional embodiment, a first outline line can be superimposed on the original image, and the image within the range of the first outline line in the original image is the first cutout image. See also Figure 2EIn the user interface 18, the first outline is the outer outline of the target cutout object "car" in the original image, so "car" is the first cutout image obtained by cutting out from the original image.

[0103] In this embodiment of the application, the original image includes a target object for matting, which can be a subject object or a text object. Figures 3A-3E Taking the subject object as an example, this paper explains the service method based on image matting. Figures 3A-3E In this example, we use a "car" as the main object to be cut out, but it is not limited to "cars". Figures 4A-4D Taking a text object as an example, this section explains the service method based on image matting. Figures 4A-4D The example shown is "Work location: Unit 2, Building C, Community B, Street A", but it is not limited to this.

[0104] Figures 3A-3E This embodiment of the present application provides a schematic diagram illustrating the interface changes of an electronic device when a subject object contained in an original image is cut out and an application is recommended. Figures 3A-3E This example uses the application interface displaying the original image in the "Gallery" to illustrate the service method based on image cutout. Figure 3A As shown, the user interface 07 may include a status bar 071 and an image 072, where image 072 is the original image and includes the main object 0721. The status bar 071 may include the operator's name (e.g., China Mobile), time, Wi-Fi icon, signal strength, and current remaining battery power. The user can perform a first user operation on the image 072 on the user interface 07. For example, the first user operation could be a click, pinch, or press operation on the original image. Figure 3A The illustration uses a two-finger press as an example of the first user operation, but it is not limited to this. The electronic device can respond to the first user operation on image 072, perform image matting on image 072, obtain a first matted image corresponding to the main object 0721, and display it as shown below. Figure 3BThe user interface 08 shown displays a first cutout image. Detailed implementation methods for the cutout processing can be found in subsequent embodiments. The first cutout image is implemented as a main cutout image 0821 in the user interface 08. At this time, the main image 0821 is in a state that can be dragged by the user. Optionally, after the user performs a first user operation, the main cutout image 0821 can be highlighted. For example, a first outline line can be superimposed on the original image, and the image within the first outline line is the first cutout image. That is, a dynamic outline is performed along the outline edge of the main cutout image 0821 in the original image, and finally, the outline edge of the main cutout image 0821 is thickened. Another example is that the main cutout image can be enlarged. Furthermore, the category information 0822 of the first cutout image is displayed in the associated area of ​​the first cutout image on the user interface 08. It should be noted that the category information 0822 of the main object 0721 can be text information, icon information, or other information forms that can represent the category of the main object 0721, and is not limited thereto. Figure 3B In the example shown, the category information 0822 of the main object 0721 is implemented as icon information.

[0105] The subject cutout image 0821 in user interface 08 supports dragging by the user in any direction. For example... Figure 3B As shown, when a user drags a subject cutout image 0821 on user interface 08, the electronic device responds to this dragging operation, allowing the subject cutout image 0821 to move within user interface 08 with the user's fingertip, and the subject cutout image 0821 can cover any element in user interface 08. Optionally, after the subject cutout image 0821 is dragged, the electronic device can still display image 072 in user interface 08. Optionally, the subject object 0721 in image 072 can be blurred or blurred, meaning the sharpness of the subject object 0721 in user interface 08 can be lower than the sharpness of the subject object 0721 in user interface 07, to indicate to the user that the subject object 0721 is selected and dragged. Of course, the subject object 0721 can also have the same sharpness as the subject image 0821. It should be noted that the size of the subject cutout image 0821 and the subject object 0721 can be the same, such as... Figure 3A and Figure 3B As shown. Optionally, the size of the subject cutout image 0821 is smaller than the size of the subject object 0721.

[0106] Furthermore, the user inputs a second user operation on the subject cutout image 0821. In response to this second user operation, the electronic device displays the icons of the first application and the second application in the target area, such as... Figure 3CThe user interface 09 is shown. The second user operation triggers an application recommendation or displays the icon of a recommended application for the main cutout object. In this embodiment, recommending the first and second applications is taken as an example. In this embodiment, the implementation method of the second user operation is not limited; it can be a drag operation, a click operation, or a long press operation, etc. In one specific way, the user inputs the second user operation on the first cutout image as a drag operation, more specifically, it is implemented as dragging the first cutout image into a third area (i.e., a hotspot, such as the right side hotspot).

[0107] like Figure 3C As shown, the user interface 09 includes a sub-interface 091, an icon area 092, and a main cutout image 093. The target area 092 is the target area mentioned above.

[0108] Specifically, the elements displayed in sub-interface 091 (e.g., images) are identical to those displayed in user interface 08 (except for the status bar). In effect, sub-interface 091 can be considered as the interface obtained by the electronic device after a three-dimensional transformation of all elements (except the status bar) of user interface 08. In user interface 09, sub-interface 091 can create a visual effect on the screen where the original interface (i.e., user interface 08) pushes inward. Understandably, this "opening" visual effect metaphorically represents the act of transmitting elements (information), which is more intuitive than copying and pasting. Optionally, in user interface 09, the background behind sub-interface 091 can be set to the electronic device's desktop background (i.e., desktop wallpaper) with a blurred effect.

[0109] Icon area 092 can be used to display application icons recommended by one or more electronic devices based on information elements selected by the user (i.e., the main subject cutout image 0821 in this embodiment). The application icons displayed in icon area 092 can include icons for a first application and icons for a second application. Application icons can be displayed horizontally or vertically; in vertical display, they can be displayed in a single column or multiple columns. Figure 3C The illustration uses the application icon displayed vertically in a single column as an example.

[0110] Optionally, each icon in icon area 092 has a corresponding text description below it. This text description can be used to explain the specific application name or application service name corresponding to the icon. Taking icon 0921 and text 0922 as an example, when icon 0921 represents an application... When the icon is 0922, the text 0922 can be "WeChat"; when the icon is 0921, it represents an application. When the icon for the "Moments" app service is displayed, the text 0922 can be either "Moments" or "WeChat Moments". When the icon 0921 is for the app... When using the "Scan" function to scan an application service icon, the text "0922" can be either "Scan" or "WeChat Scan". It should be understood that the text below each application service icon is only to help users understand the specific application or function corresponding to the icon; however, this application does not impose any restrictions on the specific icon style or text content.

[0111] The image content displayed in the subject cutout image 093 is the same as that displayed in the subject object 0721 and the subject cutout image 0821. Optionally, the subject cutout image 093 can be a thumbnail obtained by reducing the size of the subject cutout image 0821 on an electronic device. Similarly, the subject cutout image 093 and the subject cutout image 0821 are identical except for their size. Thus, when a user selects their desired application service in the icon area 092, the subject cutout image 093 will not obscure a large area of ​​the icons and text in the icon area 092, making it easier for the user to select icons.

[0112] It should be noted that after the subject cutout image 0821 is reduced to obtain the subject cutout image 093, the subject cutout image 0821 can still be displayed in the user interface 09. That is, the subject cutout image 093 can be regarded as an image obtained by the electronic device copying the subject cutout image 0821 and then reducing it, and the subject cutout image 0821 can be regarded as a copy of the subject object 0721.

[0113] It should be noted that when the second user operation involves dragging the first cutout image into the right-side hotspot area, the electronic device can display the icon area 092 on the right side of the user interface 09, such as... Figure 3C As shown. When the second user operation involves dragging the first cutout image into the left edge hotspot, the electronic device can display an icon area on the left side of the user interface, but no illustration is shown.

[0114] In user interface 09, the user drags the subject cutout image 093 into the target area, performs a third user operation to continue dragging the subject cutout image 093 into the icon area 092 where the icon of the first application is located, and then releases it. The electronic device can then respond to this third user operation and display as shown below. Figure 3E The user interface 21 shown.

[0115] The third user operation is the operation of selecting the first application from the recommended applications. The above-mentioned "third user operation of dragging the subject cutout image 093 into the area where the icon of the first application is located in the icon area 092 and releasing it" is only an example, but is not limited to this. In addition, the third user operation can also be the operation of clicking the icon of the first application in the target area, or the operation of long-pressing the icon of the first application in the target area, or the operation of pinching the icon of the first application, etc.

[0116] In user interface 09, the user drags the subject cutout image 093 into the target area, performs a fourth user operation, and then drags the subject cutout image 093 into the icon area 092 where the icon of the second application is located and releases it. The electronic device can then respond to this fourth user operation by displaying, as shown below. Figure 3D User interface 20.

[0117] The fourth user operation is the selection of a second application from the recommended applications. The above example of "dragging the subject cutout image 093 into the icon area 092 containing the icon of the second application and releasing it" is merely an example and is not limited to this. In addition, the fourth user operation can also be clicking the icon of the second application in the target area, long-pressing the icon of the second application in the target area, or pinching the icon of the second application, etc.

[0118] Furthermore, to guide users to accurately place the dragged subject cutout image 093 into the desired application service, when the subject cutout image 093 approaches an icon in the icon area 092, or when the distance between the subject cutout image 093 and an icon in the icon area 092 is less than a certain threshold, the icon approached by the subject cutout image 093 can display certain special effects. For example, the icon and its corresponding text can be snapped closer to the subject cutout image 093, and the icon and its corresponding text can be enlarged. Taking icon 0923 and its corresponding text 0924 in user interface 09 as an example, when the subject cutout image 093 approaches icon 0923, icon 0923 and text 0924 are positioned slightly to the left compared to other icons and text, that is, closer to the subject cutout image 093 than other icons and text; in addition, icon 0923 is also enlarged to prompt the user that if they release their grip here, the subject cutout image 093 (or subject cutout image 0821) will be transferred to the application corresponding to icon 0924.

[0119] Among them, such as Figure 3DAs shown, the user interface 20 includes: a second interface 201 of the second application, which includes a second image and second service information associated with the second image. The second image corresponds to a first cutout image; for example, the second image is the same as the first cutout image, or the second image is obtained by enlarging or reducing the first cutout image. Specifically, the second image is implemented as a main cutout image 2011. The second service information associated with the second image is implemented as a pop-up window 2012. The pop-up window 2012 includes text 20121, a card 20122, and a card 20123.

[0120] For example, the user drags the subject cutout image 093 to the second application corresponding to the icon. The second application is a shopping service. The second interface 201 of the second application is a search service page for the subject cutout image 2011. The pop-up window 2012 contains the second service information (such as search results information) obtained from the search for the subject cutout image 2011. For example, the text 20121 can be some explanatory text, such as "The following similar products were found for you". Cards 20122 and 20123 are card information of similar products obtained from the search for the subject cutout image 093.

[0121] Optionally, in user interface 20, the background behind the second interface 201 of the second application can be set to the image obtained by overlaying a blur effect on user interface 07 or user interface 08.

[0122] Among them, such as Figure 3E As shown, the user interface 21 includes: a first interface 211 of the first application, which includes a first image and first service information associated with the first image. The first image may be a third cutout image after image filtering. Specifically, the first image is implemented as a subject cutout image 2113, and the first service information associated with the first image is implemented as: user avatar 2111 and text chat content 2112.

[0123] It should be noted that this can be understood as Figure 3D The subject cutout image 2011 is the same as the subject cutout image 0821 (or subject cutout image 093). Figure 3E The subject cutout image 2113 is obtained by performing image filtering on the subject cutout image 093. The image precision of the subject cutout image 2113 is greater than that of the subject cutout image 2011. Compared with the image precision of the subject cutout image 2011, the cutout effect of the subject cutout image 2113 is more accurate, the image edges are clearer and smoother, and the detail retention is higher.

[0124] The circumstances under which the electronic device displays the second interface of the second application in response to a user inputting a fourth user operation in the target area are not limited.

[0125] In one optional embodiment, after the user displays the first interface of the first application, in response to the user's close operation on the first interface, the first interface is closed, and the user returns to the interface containing the icons of the first application and the second application (e.g., ...). Figure 3C The user interface shown is 09); in response to the user entering a fourth user operation in the target area, the second interface of the second application is displayed.

[0126] In another alternative embodiment, the electronic device displays the original image, displays a first cutout image in response to a first user operation input by the user onto the original image, and displays the icons of a first application and a second application in the target area in response to a second user operation input by the user onto the first cutout image, i.e., displays as shown. Figure 3C The user interface 09 shown; in response to the user entering a fourth user operation in the target area, the second interface of the second application is displayed.

[0127] In some alternative embodiments of this application, the applications participating in the service recommendation may not be categorized, and all applications are assumed to have detailed image cutout requirements. In these alternative embodiments, the following is still used: Figures 3A-3C Taking the attached diagram as an example, when an electronic device receives a third-user operation input for any application, it does not need to distinguish the category of the application. By default, it performs an image filtering operation on the subject cutout image 093 to improve the accuracy of the subject cutout image 093, and provides the subject cutout image after the image filtering operation to the corresponding application, as shown in the diagram. Figure 3E The user interface 21 shown.

[0128] Of course, in some alternative embodiments of this application, the applications participating in the service recommendation may not be categorized, and it is assumed that all applications do not have a need for detailed image cutout. In these alternative embodiments, it is still based on... Figures 3A-3C Taking the attached diagram as an example, when the electronic device receives a fourth user operation input for any application, it does not need to identify the category of any application, but directly provides the subject cutout image 093 to the corresponding application, as shown in the diagram. Figure 3D The user interface 20 shown.

[0129] Understandable Figures 3A-3E The image cutout service process shown is continuous, from the user dragging the subject cutout image to providing the cutout image to the corresponding application. "Continuous" here means that the user does not need to copy, paste, or switch applications; the user's fingers do not need to leave the electronic device screen throughout the entire process, making it simple and fast. Furthermore, it provides cutout images of different qualities for different types of applications to offer users higher-quality services. The entire process integrates application recommendation with the image cutout service, resulting in simple operation, high efficiency, and an improved user experience.

[0130] Figures 4A-4D This application provides a schematic diagram illustrating the interface changes of an electronic device when text objects contained in an original image are extracted and recommended for application purposes. Figures 4A-4D The example of displaying the original image in the "Gallery" application interface illustrates the service method based on image cutout.

[0131] like Figure 4A As shown, the user interface 11 may include an image 1100, which includes text objects 1101 and 1102. Text objects 1101 and 1102 may be text located within the image 1100. A user can perform a first user operation on the user interface 11, for example, in... Figure 4A The illustration shows a user interface including a "cutout" control. The first user operation is illustrated as a trigger action on this control, but it is not limited to this. For example, the first user operation could be a two-finger press, a long press, or a click, etc. The electronic device responds to the first user operation on the user interface 11 by performing cutout processing on the image 1100, obtaining the cutout images corresponding to text objects 1101 and 1102 respectively, and displaying them. Figure 4B The content shown.

[0132] like Figure 4B As shown, the user interface 12 includes an image 120, which includes text objects 1101 and 1102, corresponding to text cutout images 1201 and 1202, respectively. The content displayed in text cutout image 1201 is an outline added to the area containing text object 1101; similarly, the content displayed in text cutout image 1202 is an outline added to the area containing text object 1102. Text cutout image 1202 includes text object 12021 and category information 12022. The category information of the text object may include, but is not limited to, landmarks or telephone numbers. For example, text object 12021 may be implemented as address information, and category information 12022 may be implemented as a landmark. Optionally, after the user performs the first user operation, the text-cutout image 1201 or text-cutout image 1202 can be highlighted. For example, a dynamic outline can be applied along the contour edge of the text-cutout image 1201 or text-cutout image 1202, ultimately making the contour edge of the text-cutout image 1201 or text-cutout image 1202 bolder. It should be noted that the category information 12022 can be any information format that can represent the category of the text object 12021, such as text information or image information, and is not limited thereto. Figure 4B In the illustration, the category information 12022 of the text object 12021 is implemented as image information.

[0133] The user performs a user operation on the recognized text cutout image, such as a drag operation. In response to the drag operation, the electronic device can drag the selected text cutout image on the user interface 12. For example, the user performs a drag operation on text cutout image 1202. Text cutout image 1202 can move with the user's fingertip within the user interface 12, and text cutout image 1202 can be displayed on top of the user interface 12 (i.e., text cutout image 1202 can cover any element in the user interface 12). While text cutout image 1202 is being dragged, the electronic device can still display text object 1102 in the user interface 12, and text cutout image 1202 also includes the content of text object 1102. During the dragging of text cutout image 1202, the size of text object 12021 and text object 1102 in text cutout image 1202 can be the same. Of course, the sizes can also be different; this is not limited.

[0134] exist Figure 4B In the process, a second user operation is performed on the text-cutout image 1202. The second user operation is implemented by dragging the text-cutout image 1202 to a hot area of ​​the electronic screen. The electronic device can respond to the user's second user operation on the text-cutout image 1202 and display as shown. Figure 4C The user interface 13 is shown. For a detailed description of the target area, please refer to the preceding text; it will not be repeated here.

[0135] like Figure 4C As shown, the user interface 13 includes a sub-interface 131, an icon area 132, and a text cutout image 133.

[0136] The specific style and function of sub-interface 131 and icon area 132 can be referred to the aforementioned description. Figure 3C The relevant descriptions of the Chinese user interface 09 will not be repeated here.

[0137] It should be noted that the text content contained in the text cutout image 133 is the same as the text content contained in text objects 1101 and 1102. However, as can be seen from user interface 12 and user interface 13, the text cutout image 133 is obtained by the electronic device after shrinking the text cutout image 1202; after dragging the text cutout image 133 to the target area, the electronic device can shrink the text cutout image dragged by the user to a certain extent, so that the text cutout image 133 will not cause a large area of ​​obscuring the icons and text in the icon area 132, making it easier for the user to select.

[0138] Similarly, in user interface 13, when a user drags the text cutout image 133 into the target area and performs a third user operation—dragging the text cutout image 133 into the corresponding icon in icon area 132 and releasing it—the electronic device can respond to this user operation by transferring the text cutout image 133 to the second application corresponding to the icon and displaying user interface 14 as shown in 4D. Of course, the electronic device can also respond to this user operation by transferring the text cutout image 133 to the first application corresponding to the icon; this is not limited.

[0139] exist Figure 4D In this application, the user interface 14 includes a second interface 141, which includes a text-based cutout image 1412 and its icon 1411. The second interface 141 serves as a navigation page, the text-based cutout image 1412 represents address information, and the icon 1411 represents a navigation icon. The second interface 141 also includes other navigation information, such as distance information and navigation time information for different traffic situations. Furthermore, the second interface 141 also includes a navigation control 1413 and a route control 1414. When the navigation control 1413 is triggered, a navigation route to the address information corresponding to the text-based cutout image 1412 is planned in real time. When the route control 1414 is triggered, navigation route information from the current location to the address corresponding to the text-based cutout image 1412 is planned and displayed.

[0140] An exemplary embodiment of this application provides a method for image matting services, which is applied to electronic devices, such as... Figure 7A As shown, the method includes:

[0141] T10. Display the original image, which includes the target object to be cut out.

[0142] T11. In response to a first user operation inputting the original image, perform image matting processing on the original image to obtain a first matted image corresponding to the target matted object, and display the first matted image.

[0143] In this embodiment, the first user operation can be any user operation capable of triggering the image cutout process on the original image. For example, a click operation, a pinch operation, or a press operation; a press operation can include a long press or a two-finger press, etc. Figure 3A The illustration uses the first user operation, a two-finger press, as an example. Figure 4A In the electronic device, the user interface 11 includes a "cutout" control, and the first user operation is implemented by clicking on the "cutout" control.

[0144] T12, in response to a second user operation input to the first cutout image, displays the icons of the first application and the second application in the target area.

[0145] The second user operation can be to drag the first cutout image into the third region (i.e., the hot zone). For a detailed description of the hot zone and the target region, please refer to the foregoing embodiments, which will not be repeated here.

[0146] The application category information can be preset or adaptively determined based on the characteristics of the associated services. For example, in this embodiment, the first application and the second application are determined in an exemplary manner. For example, the first application is an application that requires image accuracy higher than a set image accuracy threshold and does not have low latency requirements; the second application is an application that requires image accuracy higher than a set image accuracy threshold and has low latency requirements; and an application that requires image accuracy lower than or equal to a set image accuracy threshold.

[0147] In this embodiment, the application's position in the target area is pre-configured. The user can input a user operation in the target area based on the first cutout image. This user operation can be input for a first application or a second application. For ease of distinction and description, the user operation input for the first application is referred to as the third user operation, and the user operation input for the second application is referred to as the fourth user operation. When the user performs the third user operation on the first application, the first interface of the first application is displayed; when the user performs the fourth user operation on the second application, the second interface of the second application is displayed. The third and fourth user operations can be of the same type, for example, both being drag operations. However, it should be noted that the termination positions of the two drag operations are different when performing drag operations on the first and second applications. Of course, the third and fourth user operations can also be of different types.

[0148] Optionally, the third user operation is to drag the first cutout image into the first region and release it; the fourth user operation is to drag the first cutout image into the second region and release it, where the first region and the second region are non-overlapping regions in the target region.

[0149] Further optionally, the first region is the region where the icon of the first application is located, and the second region is the region where the icon of the second application is located.

[0150] Specifically:

[0151] T13. The electronic device responds to the user inputting a user operation on the target area of ​​the first cutout image, and determines whether the application selected by the user requires the image precision to be higher than a set image precision threshold based on the position information of the user operation in the target area.

[0152] In this embodiment, the first application is for applications with high image matting accuracy requirements (image accuracy higher than a set image accuracy threshold), and conversely, the second application is for applications with low image matting accuracy requirements (image accuracy lower than or equal to a set image accuracy threshold).

[0153] T14. If it is determined that the application selected by the user requires image precision higher than the set image precision threshold, then determine whether the application selected by the user has low latency requirements.

[0154] T15. If the application selected by the user does not have low latency requirements, the application selected by the user is the first application. Perform image filtering processing on the first cutout image to obtain the third cutout image, and then execute step T16.

[0155] In this embodiment, the image precision of the third cutout image is higher than that of the first cutout image.

[0156] T16. Display the first interface of the first application, which includes the third cutout image and its service information.

[0157] The image filtering algorithms involved in the above steps are not limited. Image filtering algorithms can be guided filtering algorithms, algorithms combining image closing operations and mean filtering, and edge-preserving smoothing algorithms, etc. Edge-preserving smoothing algorithms can include, but are not limited to, guided filtering, bilateral filtering, and weighted least squares filtering.

[0158] The image closing operation is a process performed on the first cutout image after sequential dilation and erosion. Dilation followed by erosion helps to close small holes inside foreground objects or small black spots on objects, improving the accuracy of image filtering. Mean filtering is a typical linear filtering algorithm. It involves applying a filter template to the target pixel (any pixel) in the first cutout image. This template includes its surrounding neighboring pixels (eight pixels surrounding the target pixel, including the target pixel itself). The average value of all pixels in the filter template is then used to replace the original pixel value.

[0159] The image filtering algorithms perform the same or similar filtering processes on both the target subject and the target text object. The following example uses a guided filtering algorithm and the target subject as the image filtering algorithm to illustrate the process of filtering the original image to obtain a mask image that preserves the edges of the target subject. Input image I is the mask image of the target subject, which has the same size as the original image. Guided image G is the original image, and result image O is the mask image of the target subject after edge preservation.

[0160] The principle of the guided filtering algorithm:

[0161] 1) The result graph O and the guide graph G satisfy a local linear model, that is, O i =a k G i +b k ,i∈ω k k is the midpoint of the localized window. This local linear model ensures that the output graph O coincides with the edge of the guided graph G. This local linear model includes the coefficients to be solved (a k ,b k ).

[0162] 2) The input graph I can be considered as a combination of the output graph O and the noise n, i.e., O i =I i -n i .

[0163] 3) By minimizing the difference between the output graph O and the input graph I, a local linear transformation between the output graph O and the guide graph G can be obtained: ∈ is the L2 norm regularization coefficient, which is used to prevent a k Too large. The coefficients (a) in the local linear transformation between the output graph O and the guiding graph G are solved using the least squares method. k ,b k ).

[0164] Among them, based on the guide graph G and the coefficients (a) of the local linear model k ,b k ) and O i =a k G i +b k The output image O is obtained, which is the mask image of the target subject after edge preservation, that is, the third cutout image after performing image filtering operation.

[0165] T17. If the application selected by the user has low latency requirements, the application selected by the user is the second application, and the second interface of the second application is displayed. The second interface includes the first cutout image and its service information.

[0166] T18. If it is determined that the image precision required by the application selected by the user is lower than or equal to the set image precision threshold, and the application selected by the user is the second application, then the second interface of the second application is displayed. The second interface includes the first cutout image and its service information.

[0167] In this embodiment, the image precision of the cutout image is correlated with the application's image precision requirements and low latency requirements. For applications that require image precision higher than a set image precision threshold but do not have low latency requirements, a high-precision cutout image is needed. Therefore, the first cutout image is subjected to image filtering processing to ensure that the application can accurately provide the corresponding service to the user based on the high-precision cutout image. Conversely, for applications that do not have fine-grained cutout requirements or have low latency requirements, the requirements for cutout precision are lower, but the requirements for latency are higher. The application can directly provide the corresponding service to the user based on the first cutout image to ensure that the application can provide the corresponding service to the user in a timely and efficient manner.

[0168] It should be noted that the above embodiments, which differentiate the different requirements of various applications for image matting accuracy based on image accuracy requirements and low latency requirements, are merely examples. In addition, different applications can also differentiate their different requirements for image matting accuracy based solely on image accuracy requirements or solely on low latency requirements.

[0169] against Figure 7A The implementation of step T11, namely "in response to a first user operation input to the original image, performing matting processing on the original image to obtain a first matted image corresponding to the target matted object, and displaying the first matted image", includes:

[0170] P1. In response to a first user action on the input of the original image, determine the cutout region in the original image, the cutout region including the target cutout object;

[0171] P2. Crop the cutout area from the original image to obtain the first cutout image;

[0172] P3. Display the first cutout image.

[0173] In this embodiment, the target object for matting can be a target subject object, or it can be a target text object. The following sections will describe each case separately.

[0174] Case 1: The target object to be cut out is implemented as the target subject object. like Figure 7B As shown, the implementation of step P1 above, namely "determining the cutout region in the original image in response to a first user operation inputting the original image," includes:

[0175] P11a. In response to the first user operation on the original image, the original image is input into the image segmentation model for image segmentation to obtain the mask region of the target subject.

[0176] The original image includes salient regions, which are the most attractive or important areas within the original image. Typically, salient regions can be areas that occupy a large area, areas located in the center, areas at trisection points or lines, etc., without limitation. The salient region corresponds to the target matting object; for example, the target matting object may be the main object contained within the salient region, or it may be the text object contained within the salient region.

[0177] The image segmentation model can employ a subject segmentation algorithm. Subject segmentation algorithms can predict salient regions in the original image.

[0178] In terms of segmentation methods, subject segmentation algorithms include, but are not limited to, semantic segmentation, instance segmentation, and panoptic segmentation.

[0179] From the perspective of the algorithm technology employed, subject segmentation algorithms can be mainly divided into three categories: non-learning-based subject segmentation algorithms, traditional machine learning-based subject segmentation algorithms, and deep learning-based subject segmentation algorithms. Non-learning-based subject segmentation algorithms predict salient regions by calculating the contrast between each sub-region in the original image and its surrounding neighboring regions. Based on the size of the neighboring regions, this algorithm can be further divided into two types: subject segmentation algorithms based on local contrast and subject segmentation algorithms based on global contrast. Traditional machine learning-based subject segmentation algorithms predict salient regions by extracting salient features from the original image, such as multi-scale contrast, center-surround histogram, and color space distribution. Deep learning-based subject segmentation algorithms use an encoder-decoder structure, where the encoder is responsible for extracting image features from the input image, and the decoder is responsible for predicting the salient target mask.

[0180] In cases where the mask area of ​​the target subject is small, the image cutout effect is not ideal. To improve the cutout effect, before cropping the target local image corresponding to the target subject from the original image, the following steps are also included:

[0181] P12a. Determine whether the area ratio of the mask region of the target subject in the original image is greater than the set area ratio threshold a%. For example, a% can be 1%, 2%, or 5%, etc.

[0182] P13a. If the area of ​​the mask region of the target subject in the original image is less than or equal to a%, do not respond to image matting.

[0183] Accordingly, step P2, namely "cropping the cutout region from the original image as the first cutout image", is implemented as follows:

[0184] P21a. If the area ratio of the mask region of the target subject in the original image is greater than a%, perform image matting on the original image to obtain the first matted image corresponding to the target subject.

[0185] The matting process in step P21a can be performed using a matting algorithm. The principle of the matting algorithm is as follows: for the original image I, the subject is called the foreground F, and the rest is called the background B. The original image I can then be considered a weighted fusion of the foreground F and the background B, i.e., I = alpha * F + (1 - alpha) * B = alpha * F + (1 - alpha) * B, where alpha is the weight. The matting algorithm is used to find a suitable alpha, which is a continuous value between 0 and 1. For example, the matting algorithm can be based on a trimap-based algorithm or a trimap-free algorithm. The trimap-based algorithm predicts the weights between the subject and the background based on the original image and prior information, including but not limited to: a trimap, a coarse mask, a background image without people, and pose information. The trimap-free algorithm predicts the weights between the subject and the background based on the original image. Trimap-free algorithms mainly include two types: one is a portrait matting algorithm, which uses an end-to-end portrait matting model (Rethinking Portrait Matting with Privacy Preserving, P3Mnet) to matte portraits; the other is a subject matting algorithm, which uses a PositionFocused Attention (PFAN) mechanism to emphasize the relative positional relationship between subjects in the original image, better encoding the original image for subsequent matting. This algorithm supports matting subjects such as portraits, animals, plants, buildings, vehicles, food, goods, cartoon characters, and logos.

[0186] Accordingly, step P3, namely "displaying the first cutout image", is implemented as follows:

[0187] P31a. Identify the category information of the target subject object;

[0188] P32a. Based on the category information of the target object to be cut out, determine whether the target object to be cut out belongs to sensitive information;

[0189] P33a. If the target object to be cut out is not sensitive information, display the first cut-out image.

[0190] P34a. If the target object to be cut out is sensitive information, the first cut-out image will not be displayed.

[0191] Not displaying the first cutout image indicates that the first cutout image is in a state where a second user operation (such as dragging) cannot be performed.

[0192] One approach is to employ a taboo detection algorithm. The category information of the target matted object and the first matted image are input into the taboo detection algorithm for detection to determine whether the target matted object constitutes sensitive information. For example, the taboo detection algorithm may include multiple detection branches. From these branches, the target detection branch corresponding to the category information of the target matted object is determined. Based on the compliance detection strategy of the target detection branch, it is then determined whether the target matted object contained in the first matted image constitutes sensitive information.

[0193] Sensitive information may include, but is not limited to, the following:

[0194] Copyright infringement: If the original image is protected by copyright, any unauthorized removal of the original image and its use for commercial purposes or other illegal activities will infringe on the copyright of the original image.

[0195] Privacy issues: Extracting human images from original images may infringe on privacy rights.

[0196] False information category: After cutting out the original image, the target subject's partial image may be placed in an incorrect background or context to create false information, which may include political propaganda, online fraud, or malicious advertising.

[0197] Sensitive content category: The original image may contain disturbing content, such as violence, pornography, or disgusting material. Extracting images from these original images and using them in inappropriate settings may violate ethical guidelines or relevant laws and regulations.

[0198] Commercial Competition: If the original image may be a trademark or logo of another company or brand, using the original image without authorization in a competitor's advertisement or product may constitute unfair commercial competition.

[0199] Alternatively, one implementation of P31, namely "identifying the category information of the target subject object," is as follows:

[0200] P311a. Based on the mask area of ​​the target subject, extract the corresponding local image of the target subject from the original image.

[0201] Specifically, based on the mask region of the target object, the smallest bounding rectangle of the target object can be cropped from the original image as a local target image. Of course, it is also possible to expand outward from the smallest bounding rectangle; there is no limitation on this. In addition, based on the mask region of the target object, the bounding circle, bounding trapezoid, bounding triangle, or other bounding polygons of the target object can also be cropped from the original image as local target images.

[0202] P312a. Input the local image of the target into the image classification model, perform image classification on the local image of the target, and obtain the category information of the target subject.

[0203] The image classification model is not limited. For example, image classification models can include, but are not limited to: deep residual network (ResNet), deep convolutional neural networks (Deep Convolutional Neural Networks), visual geometry group network (VGGNet), etc.

[0204] Case 2: The target object to be cut out is implemented as the target text object. like Figure 7B As shown, step P1, namely "in response to a first user operation inputting the original image, determining a cutout region in the original image, the cutout region including the target cutout object," includes the following implementation:

[0205] P11b: In response to the first user operation on the original image, a text detection algorithm is used to perform text detection on the original image to obtain the target text region.

[0206] Text detection algorithms include, but are not limited to, color segmentation-based methods, edge detection-based methods, and feature-based methods.

[0207] Accordingly, step P2, namely "cropping the cutout region from the original image as the first cutout image", is implemented as follows:

[0208] P21b. Using a cutout algorithm, perform a cutout operation on the target text region to obtain the first cutout image containing the target text object.

[0209] For details on the image matting algorithm, please refer to the previous text, which will not be repeated here.

[0210] Accordingly, step P3, namely "displaying the first cutout image", is implemented as follows:

[0211] P31b. Use a character recognition algorithm to perform character recognition on the first cutout image to obtain the text content of the target text object. The character recognition algorithm includes, but is not limited to: Optical Character Recognition (OCR) algorithm, Convolutional Recurrent Neural Network (CRNN) based on deep learning, and Connectionist Temporal Classification (CTC) algorithm.

[0212] P32b Determine whether the text content corresponding to the target text object is sensitive information.

[0213] P33b. If the text content corresponding to the target text object is sensitive information, the first cutout image will not be displayed.

[0214] P34b. If the text content corresponding to the target text object is not sensitive information, display the first cutout image.

[0215] When the target text object is implemented as the target image, if a user action is detected on the application's icon, the text content of the target text object can be directly provided to the application. Alternatively, if the user performs a third user action on the first application, the first image is filtered to obtain a third image, which is then provided to the first application. If the user performs a fourth user action on the second application, the first image is provided to the second application.

[0216] In one alternative embodiment, the original image includes different regions, and the user performs user operations on different regions. The electronic device can recommend different types of applications to the user to suit the user's needs.

[0217] For example, when an electronic device displays the original image, the electronic device may display a first cutout image in response to a first user operation input to the original image; and display the icons of a first application and a second application in the target area in response to a second user operation input to the first cutout image.

[0218] For example, an electronic device may, in response to a fifth user operation inputting on the original image, display a second cutout image, the first and second cutout images being of different categories; and, in response to a second user operation inputting on the second cutout image, display an icon of a third application in the target area; the icon of the third application being different from the icons of the first and second applications.

[0219] The first user operation can be a user pressing, clicking, or sliding operation on the area corresponding to the first cutout image in the original image; the fifth user operation can be a user pressing, clicking, or sliding operation on the area corresponding to the second cutout image in the original image. The second user operation can be dragging either the first or second cutout image into the hotspot. It should be noted that after the icon of the third application is displayed in the target area, the electronic device can also respond to the user's selection of the third application in the target area by displaying a third interface of the third application. The third interface includes a third image, which corresponds to the second cutout image. For example, the third image can be the second cutout image, or it can be a third cutout image obtained by performing a filtering operation on the second cutout image. The implementation methods for performing user operations on the second cutout image and for the electronic device responding to user operations can be found in the aforementioned implementation methods for performing user operations on the first cutout image and for the electronic device responding to user operations, and will not be repeated here.

[0220] In one optional embodiment, when the original image includes multiple (≥2) cutout objects, for example, the original image includes a first cutout object (e.g., a vehicle) and a second cutout object (e.g., a cat), the first and second cutout objects belong to different categories. The user can trigger a cutout-based service process for the first cutout object or initiate a cutout-based service process for the second cutout object. An exemplary description follows.

[0221] For example, if a user wants to trigger a cutout-based service process for a first cutout object, the user can input a first user operation on the first cutout image in the original image. The electronic device then responds to the first user operation by displaying the first cutout image, which includes the first cutout object. In response to the user inputting a second user operation on the first cutout image, the device displays the icons of a first application and a second application in the target area. In response to the user inputting a third user operation in the target area, the device displays a first interface of the first application, which includes a first image corresponding to the first cutout image. In response to the user inputting a fourth user operation in the target area, the device displays a second interface of the second application, which includes a second image corresponding to the first cutout image. The first image and the second image have different image resolutions.

[0222] For example, if a user wants to trigger a background removal-based service process for a second background removal object, the user can input a fifth user operation on the second background removal image in the original image. The electronic device then responds to this fifth user operation by displaying the second background removal image, which includes the second background removal object. In response to the user inputting a second user operation on the second background removal image, an icon of a third application is displayed in the target area. The icon of the third application is different from the icons of the first and second applications. In response to the user's user operation of selecting the third application in the target area, a third interface of the third application is displayed. The third interface includes a third image, which corresponds to the second background removal image.

[0223] In this embodiment, users are allowed to initiate cutout operations on different categories of objects in the original image, and different applications are recommended to users, which can adapt to different user needs and improve the user experience.

[0224] against Figure 7A The implementation of step T12, namely "displaying the icons of the first application and the second application in the target area in response to a second user operation input to the first cutout image," includes:

[0225] R1, in response to a second user operation inputting a first matted image, determines a first application and a second application from candidate applications based on at least one of the following information: attribute information of the first matted image, usage frequency of candidate applications, running status, recent usage time, and application type; candidate applications include applications on electronic devices and / or applications on other devices that are communicatively connected to the electronic device.

[0226] R2. Display the icons of the first application and the second application in the target area.

[0227] Candidate applications are those already installed on the electronic device, or those that can be installed through the electronic device's "app store." Additionally, candidate applications can also be applications from other devices that are communicatively connected to the electronic device. For example, applications installed on tablets, laptops, or desktop computers that are communicatively connected to the electronic device.

[0228] The usage frequency of candidate applications refers to the number of times a user uses the application within a certain period (such as one month, six months, or one year). The running status of candidate applications can be in the foreground or background. The most recent usage time of candidate applications can be morning, forenoon, noon, afternoon, or evening, etc. The application type of candidate applications can be social applications, games, shopping applications, or office applications, etc.

[0229] The attribute information of the first matted image may include, but is not limited to, pixels, resolution, size, color, bit depth, hue, saturation, brightness, color channels, and image hierarchy. If the first matted image contains the target matted object, its attribute information includes the attribute information of the target matted object. The attribute information of the target matted object may include, but is not limited to, the target matted object's category information, color information, size information, or text content.

[0230] The following is an exemplary description of an implementation method for determining the first application and the second application from all candidate applications in an electronic device.

[0231] In one optional embodiment, a first application or a second application is determined based on the attribute information of the candidate applications. The attribute information of the candidate applications includes at least one of the following: usage frequency, running status, recent usage time, and application type. For example, the first application or the second application may be a background application determined based on the running status of the candidate applications, or an application with a high usage frequency (usage frequency greater than a set frequency threshold) determined based on the usage frequency of the candidate applications, or the application last used by the user selected based on the recent usage time of the candidate applications, etc.

[0232] In another alternative embodiment, a first application and a second application can be determined from candidate applications based on the attribute information of the target matted object, so as to recommend an application that is more suitable for the target matted object, thereby displaying an application that better meets the user's needs and improving service quality and efficiency.

[0233] For example, when the original image includes different categories of cutout objects, such as vehicles and cats. If the user inputs a first user action for "vehicle," the electronic device performs a cutout operation on the original image to obtain a first cutout image containing vehicles, and determines the category information of the current cutout image as "vehicle." If the user inputs a second user action for the first cutout image, the electronic device can identify at least one application related to vehicles, such as a "car buying app" or a "used car app," and display the icon of the "car buying app" or "used car app." If the user inputs a first user action for "cat," the electronic device performs a cutout operation on the original image to obtain a second cutout image containing cats, and determines the category information of the second cutout image as the category "cat." If the user inputs a second user action for the first cutout image, the electronic device can identify at least one application related to the "cat" category and display it in the target area, such as a "wallpaper making app" (a third application). It should be understood that the applications displayed in the target area may be partially or completely different depending on the category of the target cutout object.

[0234] In addition to the attribute information of the target object to be matted, the first and second applications can be determined by combining the attribute information of the original image or the user's attribute information. The attribute information of the original image may include, but is not limited to, the text content in the original image, the geographical location information where the original image was captured, and the time information when the original image was captured. The user's attribute information may include, but is not limited to, the number of clicks the user made on the original image, the user's search history, browsing history, and the frequency of the user's use of the application. For example, based on the attribute information of the target object to be matted, a subset of candidate applications can be selected from the candidate applications; based on at least one of the attribute information of the original image and the user's attribute information, the selected subset of candidate applications can be further filtered to obtain the final first and second applications that are compatible with the target object to be matted.

[0235] In another alternative embodiment, the first application and the second application can be determined from the candidate applications based on the attribute information of the target object to be matted, combined with the attribute information of the candidate applications (e.g., usage frequency, running status, recent usage time, application type, etc.).

[0236] For example, if a user frequently uses a "third-party instant messaging application," then the icon for that application can also be displayed. Furthermore, if the geographic location of the original image is near the user's company, then office software, such as a document editor, can also be identified, and its corresponding icon can be displayed.

[0237] In the above embodiments, during the process of selecting the application to be displayed in the target area based on the attribute information of the target matted object, the category information of the target matted object can be used, but it is not limited to this.

[0238] Alternatively, in addition to displaying at least one of the above applications, icons of applications running in the background of the electronic device, icons of applications last used on the electronic device, or icons corresponding to global favorites can also be displayed.

[0239] The global collection feature, provided in this application embodiment, is an application service that can be used to collect images, text, audio, web pages, and other elements that a user has browsed on an electronic device. After collecting elements into the global collection application service, users can also edit the collected elements, such as adding tags to collect videos to mark key points, making it easier for users to efficiently review the key information carried by the elements when reviewing them.

[0240] In this embodiment, image matting service is combined with service recommendation. By matting the original image to obtain a matted image, and based on the attribute information of the matted image, combined with the attribute information of the original image and / or user attribute information, a first application and a second application related to the matted image are recommended to the user in a personalized manner. Furthermore, based on the image accuracy requirements and / or latency requirements of the first and second applications, the target image can be filtered if necessary to obtain an image that meets the application requirements, so as to display accurate service information to the user. The entire process integrates application recommendation and image matting service, which is simple to operate, highly efficient, and improves the user experience.

[0241] This application also provides a method for image matting based on an electronic device, the method comprising:

[0242] S11. Display the original image;

[0243] S12. In response to a first user operation inputting the original image, perform image matting processing on the original image and display the first matted image obtained by performing image matting processing on the original image;

[0244] S13. In response to a second user operation inputting the first cutout image, display the target area;

[0245] S14. In response to a third user operation where the user selects a first application in the target area, a first interface of the first application is displayed; the first interface includes a third cutout image obtained by performing image filtering processing on the first cutout image.

[0246] The method provided in this embodiment is similar to Figure 7A The difference in the methods shown is that: 1) the icon of the first application is displayed in the target area, which can be considered as performing image filtering operations on all first cutout images. For a detailed introduction to steps S11 to S14, please refer to the foregoing, and will not be repeated here.

[0247] This application also provides a method for image matting based on an electronic device, the method comprising:

[0248] S21. Display the original image;

[0249] S22. In response to a first user operation inputting the original image, perform image matting processing on the original image and display the first matted image obtained by performing image matting processing on the original image;

[0250] S23. In response to a second user operation inputting the first cutout image, display the target area;

[0251] S24. In response to a fourth user operation where the user selects a second application in the target area, a second interface is displayed, the second interface including the first cutout image.

[0252] The method provided in this embodiment is similar to Figure 7A The difference in the methods shown is that: 1) the icon of the first application is displayed in the target area, which can be considered as performing image filtering operations on all first cutout images. For a detailed introduction to steps S11 to S14, please refer to the foregoing, and will not be repeated here.

[0253] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps T11 to T13 can be device A; or the execution subject of steps T11 and T12 can be device A, and the execution subject of step T13 can be device B; and so on.

[0254] Furthermore, in some processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as T11, T12, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0255] Figure 8 This is a structural block diagram of an electronic device provided by an exemplary embodiment of this application.

[0256] The system framework of an electronic device is divided into five layers, from highest to lowest: the application layer, the application framework layer, the system runtime library layer, and the kernel layer. Each layer is built using a software stack approach.

[0257] The application layer can include a series of application packages. For example... Figure 8 As shown, the application package can include applications such as camera, gallery, screenshot, SMS, phone, and third-party apps. In addition, it also includes a background removal service, which comprises an algorithm invocation module, a data fusion module, and an application recommendation module.

[0258] The algorithm invocation module invokes various algorithms within the algorithm engine layer, while the data fusion module fuses multi-dimensional information such as the category information of the target object being matted, the attribute information of the original image, and user attribute information to prepare data for recommendation applications. The fused data serves as input to the service recommendation module, which determines at least one application based on the input data.

[0259] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications within the application layer. The application framework layer includes predefined functions. For example... Figure 8 As shown, the application framework layer can include a basic control framework, a resource manager, an activity manager, a drag-and-drop framework, a notification manager, and a camera framework. The drag-and-drop framework module is primarily used to support drag-and-drop effects for cutout content and to enable user interaction.

[0260] The system runtime layer includes: the algorithm engine layer, system libraries, and runtime.

[0261] The algorithm engine layer includes: image segmentation algorithm, image matting algorithm, sensitive taboo detection algorithm, image classification algorithm, text detection algorithm, and character recognition algorithm, etc.

[0262] Among them, image segmentation algorithms are used to segment the original image. Sensitive taboo detection algorithms are mainly used to detect whether the segmented target matted image contains sensitive taboo information, while image classification models are mainly used to detect the category of the target matted object in the original image. Text detection algorithms are mainly used to detect whether the original image contains text. Text recognition algorithms are mainly used to extract text from the target matted image.

[0263] The system library includes computer vision and machine learning software libraries, and camera services, among others. The machine learning software libraries include at least neural network models for object detection.

[0264] The runtime is the engine that powers applications and is responsible for scheduling and managing the software system.

[0265] The Hardware Abstraction Layer (HAL) provides standard interfaces. HAL contains multiple library modules, each implementing a set of interfaces for a specific type of hardware component, such as a Wi-Fi / Bluetooth module. When the framework API requests access to device hardware, the electronic device's system loads the corresponding library module for that hardware. For example... Figure 8 As shown, the hardware abstraction layer includes: camera (Canmera), audio (Audio), or sensor (Sensor), etc.

[0266] The kernel layer can implement memory management, driver model, power management, network management, security model, and process management, etc. Figure 8 As shown, the kernel layer includes at least a driver layer and power management. The driver layer can include: display driver, camera driver, audio driver, camera driver, or sensor driver, etc.

[0267] Understandable Figure 8 The layers in the illustrated structure and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of this application, the structure may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0268] In some embodiments, the following example uses the display of the original image in a "Gallery Application" and the target cutout object to be implemented as the target subject object, combined with... Figure 8 The structure shown illustrates the service process based on image matting. For example... Figure 9 As shown:

[0269] P1. The gallery application responds to the first user's action by providing the original image to the image cutout-based service.

[0270] P2. The image matting service performs image matting processing on the original image to obtain the first matted image corresponding to the target image matting object.

[0271] P3. The image matting service displays the first matted image on the display screen based on the display driver.

[0272] P4. Based on the image matting service, the first and second applications are determined from the candidate applications according to the attribute information of the target image matting object.

[0273] P5. The drag-and-drop frame responds to a second user action of dragging the first cutout image into the target area and displays the drag-and-drop effect on the display screen based on the display driver.

[0274] P6. The image cutout service displays the icons of the first application and the second application on the display screen based on the display driver.

[0275] P7. The drag-and-drop frame responds to user actions that drag the first cutout image into the area where any application icon is located and release it, providing the release position of the user action in the target area to the cutout-based service.

[0276] P8. Based on the image matting service, the service determines whether the application selected by the user requires image precision higher than a set image precision threshold based on the release position of the user's operation in the target area.

[0277] P9. If it is determined that the application selected by the user requires image precision higher than the set image precision threshold, then determine whether the application selected by the user has low latency requirements.

[0278] P10. If the application selected by the user does not have low latency requirements, the application selected by the user is the first application. Image filtering processing is performed on the first cutout image to obtain the third cutout image.

[0279] P11. The image matting service provides the third matted image to the first application.

[0280] P12. The first application displays the first interface based on the display driver. The first interface includes the third cutout image and its service information.

[0281] P13. If the application selected by the user has low latency requirements, or if it is determined that the image accuracy required by the application selected by the user is lower than or equal to the set image accuracy threshold, the application selected by the user is the second application, and the image matting service provides the first matted image to the second application.

[0282] P14. The second application displays a second interface based on the display driver. The second interface includes the first cutout image and its service information.

[0283] The image matting service method provided in this application can be applied to various electronic devices with touchscreens. These electronic devices can be smartphones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc.; this application does not impose any limitations on the specific type of electronic device.

[0284] For example, Figure 10 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application, such as... Figure 10 As shown, the electronic device 100 may include a processor 101, a touch screen 102, an internal memory 103, a power management module 104, a battery module 105, antennas 106a and 106b, a mobile communication module 107, a wireless communication module 108, a camera 109, an audio module 110, a speaker 111, a microphone 112, various interface modules 113, various sensors 114, various physical buttons 115, etc.

[0285] Among them, the various interface modules 113 include, but are not limited to: external memory interface, headphone jack, universal serial bus (USB) interface, subscriber identification module (SIM) card interface, high definition multimedia interface (HDMI) interface, etc.

[0286] Among them, various sensors 114 include, but are not limited to: pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0287] The physical buttons include, but are not limited to: volume control (±) buttons, power on / off buttons, and other physical buttons.

[0288] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. Figure 10 The components shown can be implemented in hardware, software, or a combination of both.

[0289] The processor 101 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller can generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution.

[0290] The processor 101 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 101 is a cache memory. This memory can store instructions or data that the processor 101 has just used or that are used repeatedly. If the processor 101 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 101, and thus improves the efficiency of the system.

[0291] In some embodiments, the processor 101 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0292] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (DCL). In some embodiments, the processor 101 may include multiple I2C buses. The processor 101 can couple to the touch sensor, charger, flash, camera 109, etc., through different I2C bus interfaces. For example, the processor 101 can couple to the touch sensor through the I2C interface, enabling the processor 101 and the touch sensor to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0293] The I2S interface can be used for audio communication. In some embodiments, the processor 101 may include multiple I2S buses. The processor 101 can be coupled to the audio module 110 via the I2S bus to enable communication between the processor 101 and the audio module 110. In some embodiments, the audio module 110 can transmit audio signals to the wireless communication module 108 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0294] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 110 and the wireless communication module 108 can be coupled via the PCM bus interface. In some embodiments, the audio module 110 can also transmit audio signals to the wireless communication module 108 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0295] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 101 and the wireless communication module 108. For example, the processor 101 communicates with the Bluetooth module in the wireless communication module 108 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 110 can transmit audio signals to the wireless communication module 108 via the UART interface to enable music playback through Bluetooth headphones.

[0296] The MIPI interface can be used to connect the processor 101 to peripheral devices such as the touchscreen 102 and the camera 109. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 101 and the camera 109 communicate via the CSI interface to enable the electronic device 100 to perform its shooting function. The processor 101 and the touchscreen 102 communicate via the DSI interface to enable the electronic device 100 to perform its display function.

[0297] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 101 to a camera 109, a touchscreen 102, a wireless communication module 108, an audio module 110, a sensor 114, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0298] A USB interface is an interface that conforms to the USB standard specification, specifically including Mini USB, Micro USB, and USB Type-C interfaces. A USB interface can be used to connect a charger to charge electronic device 100, and also for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0299] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0300] The external storage interface can be used to connect external memory cards, such as Micro SD cards, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 101 through the external storage interface to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0301] Internal memory 103 can be used to store computer executable program code, which includes instructions. Internal memory 103 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phone book, messages, pictures, videos, etc.). In addition, internal memory 103 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 101 executes various functional applications and data processing of electronic device 100 by running instructions stored in internal memory 103 and / or instructions stored in memory disposed in the processor.

[0302] The power management module 104 connects the battery module 105 to the processor 101. The power management module 104 receives input from the battery module 105 and supplies power to the processor 101, internal memory 103, touchscreen 102, camera 109, and wireless communication module 108. The power management module 104 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 104 may also be located within the processor 101.

[0303] The wireless communication function of the electronic device 100 can be implemented through antenna 106a, antenna 106b, mobile communication module 107, wireless communication module 108, modem processor, and baseband processor.

[0304] Antennas 106a and 106b are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 106a can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0305] The mobile communication module 107 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 107 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 107 can receive electromagnetic waves via antenna 106a, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 107 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 106a. In some embodiments, at least some functional modules of the mobile communication module 107 may be housed in the processor 101. In some embodiments, at least some functional modules of the mobile communication module 107 and at least some modules of the processor 101 may be housed in the same device.

[0306] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 111, etc.) or displays images or videos through the touchscreen 102. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 101 and may be housed in the same device as the mobile communication module 107 or other functional modules.

[0307] The wireless communication module 108 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 108 can be one or more devices integrating at least one communication processing module. The wireless communication module 108 receives electromagnetic waves via antenna 106b, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 101. The wireless communication module 108 can also receive signals to be transmitted from processor 101, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 106b.

[0308] In some embodiments, antenna 106a of electronic device 100 is coupled to mobile communication module 107, and antenna 106b is coupled to wireless communication module 108, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0309] Electronic device 100 implements display functions through a GPU, touchscreen 102, and application processor. The GPU is a microprocessor for image processing, connected to the touchscreen 102 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. Processor 101 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0310] Electronic device 100 can perform shooting functions through ISP, camera 109, video codec, GPU, touch screen 102 and application processor.

[0311] The ISP (Image Signal Processor) is used to process data fed back from the camera 109. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 109.

[0312] Camera 109 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 109, where N is a positive integer greater than 1.

[0313] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0314] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0315] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0316] Electronic device 100 can implement audio functions such as music playback and recording through audio module 110, speaker 111, microphone 112, headphone jack, and application processor.

[0317] The audio module 110 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 110 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 110 may be located in the processor 101, or some functional modules of the audio module 110 may be located in the processor 101.

[0318] The speaker 111, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 111.

[0319] Microphone 112, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to the microphone 112, inputting the sound signal into the microphone 112. Electronic device 100 may have at least one microphone 112. In some embodiments, electronic device 100 may have two microphones 112, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 112, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0320] The headphone jack is used to connect wired headphones. The headphone jack can be a USB interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, or a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0321] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the touchscreen 102. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to the touchscreen 102, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor. The electronic device 100 may also calculate the touch position based on the detection signal from the pressure sensor. In some embodiments, touch operations applied to the same touch position but with different intensities may correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0322] A gyroscope sensor can be used to determine the motion attitude of an electronic device 100. In some embodiments, the gyroscope sensor can determine the angular velocity of the electronic device 100 around three axes (i.e., the x, y, and z axes). The gyroscope sensor can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor detects the angle of the electronic device 100's movement, calculates the distance the lens module needs to compensate based on the angle, and allows the lens to counteract the movement of the electronic device 100 through reverse motion, thus achieving image stabilization. The gyroscope sensor can also be used in navigation and motion-sensing gaming scenarios.

[0323] A barometric pressure sensor is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor to assist in positioning and navigation.

[0324] The magnetic sensor includes a Hall effect sensor. The electronic device 100 can use the magnetic sensor to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, it can detect the opening and closing of the flip cover using the magnetic sensor. Then, based on the detected opening and closing state of the cover or the flip cover, it can set features such as automatic flip unlocking.

[0325] An accelerometer can detect the magnitude of acceleration of an electronic device 100 in various directions (typically three axes). When the electronic device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of the electronic device and is applied to applications such as screen orientation switching and pedometers.

[0326] A distance sensor is used to measure distance. Electronic device 100 can measure distance using infrared or laser. In some embodiments, during a shooting scene, electronic device 100 can utilize the distance sensor to measure distance for rapid focusing.

[0327] The proximity sensor may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that an object is near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that no object is near the electronic device 100. The electronic device 100 may use the proximity sensor to detect when a user holds the electronic device 100 close to their ear for a phone call, so as to automatically turn off the screen to save power. The proximity sensor can also be used in holster mode and pocket mode for automatic unlocking and screen locking.

[0328] An ambient light sensor is used to detect ambient light levels. Electronic device 100 can adaptively adjust the brightness of touchscreen 102 based on the detected ambient light level. The ambient light sensor can also be used to automatically adjust white balance when taking photos. Furthermore, the ambient light sensor can work in conjunction with a proximity sensor to detect whether electronic device 100 is in a pocket, preventing accidental touches.

[0329] A fingerprint sensor is used to collect fingerprints. Electronic device 100 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0330] A temperature sensor is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor to execute a temperature handling strategy. For example, when the temperature reported by the temperature sensor exceeds a threshold, the electronic device 100 performs thermal protection by reducing the performance of a processor located near the temperature sensor to reduce power consumption. In other embodiments, when the temperature is below another threshold, the electronic device 100 heats the battery module 105 to prevent abnormal shutdown of the electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, the electronic device 100 boosts the output voltage of the battery module 105 to prevent abnormal shutdown caused by low temperature.

[0331] A touch sensor, also known as a "touch device," can be located on the touchscreen 102. The touchscreen, also called a "touch display," consists of the touch sensor and the touchscreen 102. The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the touchscreen 102. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the touchscreen 102.

[0332] Bone conduction sensors can acquire vibration signals. In some embodiments, the bone conduction sensor can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor can also be integrated into headphones to form bone conduction headphones. The audio module 110 can parse the voice signal based on the vibration signal from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor to realize voice functionality. The application processor can parse heart rate information based on the blood pressure signals acquired by the bone conduction sensor to realize heart rate detection functionality.

[0333] Buttons 115 include a power button, volume buttons, etc. Buttons 115 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0334] The electronic device 100 may also include a motor. The motor can generate vibration alerts. The motor can be used for incoming call vibration alerts or for touch vibration feedback. For example, touch operations applied to different applications (such as taking photos, playing audio, etc.) can correspond to different vibration feedback effects. Touch operations applied to different areas of the touchscreen 102 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0335] The electronic device 100 may also include an indicator. The indicator may be an indicator light, which can be used to indicate charging status, power level changes, or to indicate messages, missed calls, notifications, etc.

[0336] The SIM card interface is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface simultaneously. The multiple cards can be of the same or different types. The SIM card interface is also compatible with different types of SIM cards. The SIM card interface is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0337] This application also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, enable the processor to implement the steps in the foregoing method embodiments of this application.

[0338] This application also provides a computer program product, which includes a computer program / instructions that, when executed by a processor, enable the processor to implement the steps in the foregoing method embodiments of this application.

[0339] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0340] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0341] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0342] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0343] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0344] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0345] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0346] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0347] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A service method based on image matting, characterized in that, Applied to electronic devices, the method includes: Display the original image; In response to a user inputting a first user operation on the original image, a first cutout image is displayed, wherein the first cutout image is obtained by performing cutout processing on the original image; In response to a user inputting a second user operation on the first cutout image, icons of one or more first applications and one or more second applications are displayed in the target area. The one or more first applications and the one or more second applications are determined from candidate applications based on at least one of the following information: attribute information of the first cutout image, usage frequency of candidate applications, running status, recent usage time, and application type. At least one of the one or more first applications and the one or more second applications is associated with the type of the first cutout image. The candidate applications include applications on the electronic device and / or applications on other devices that are communicatively connected to the electronic device. In response to a user inputting a third user operation in the target area, it is determined whether a target first application among the one or more first applications has a low latency requirement. The third user operation is an operation of dragging the first cutout image into a first area of ​​the target area and releasing it. The first area is the area where the icon of the target first application is located. The target first application is an application that requires image precision higher than a set image precision threshold. If it is determined that the target first application does not have a low latency requirement, image filtering processing is performed on the first cutout image to obtain a third cutout image, and a first interface of the first application is displayed. The first interface includes a first image, and the first image corresponds to the third cutout image. If it is determined that the target first application has a low latency requirement, a second interface of a second application is displayed. The second interface includes the first cutout image. During the triggering of the first user operation, the second user operation, and the third user operation, the user's finger does not leave the screen of the electronic device. In response to a user inputting a fourth user operation in the target area, a second interface of the target second application in one or more second applications is displayed. The second interface includes a second image, which corresponds to the first cutout image. The fourth user operation is an operation of dragging the first cutout image into a second area in the target area and releasing it. The second area is the area where the icon of the target second application is located. The target second application is an application that requires image precision to be lower than or equal to a set image precision threshold. During the triggering of the first user operation, the second user operation, and the fourth user operation, the user's finger does not leave the screen of the electronic device. The image precision of the first image is greater than that of the second image, and the first image, the second image, and the first cutout image contain the same cutout object.

2. The method according to claim 1, characterized in that, The first region and the second region are non-overlapping regions within the target region.

3. The method according to claim 1, characterized in that, The user inputs a second user operation on the first cutout image, which is to drag the first cutout image into the third region; wherein the target region and the third region partially or completely overlap.

4. The method according to claim 1, characterized in that, The display of the first cutout image includes: A first outline is superimposed on the original image, and the image within the range of the first outline in the original image is the first cutout image.

5. The method according to claim 1, characterized in that, The method further includes: In response to a user inputting a fifth user operation on the original image, a second cutout image is displayed, the second cutout image being of a different category than the first cutout image; in response to a user inputting a second user operation on the second cutout image, an icon of a third application is displayed in the target area; The icon of the third application is different from the icons of the first application and the second application.

6. The method according to claim 5, characterized in that, The third application is associated with the category of the second cutout image.

7. The method according to claim 6, characterized in that, Also includes: Displays the category information of the first cutout image or the second cutout image.

8. The method according to any one of claims 1-7, characterized in that, Also includes: The first image and the first service information associated with the first image are displayed on the first interface; The second image and the second service information associated with the second image are displayed on the second interface.

9. The method according to any one of claims 1-7, characterized in that, Before displaying the second interface of the second application in response to the user inputting a fourth user action in the target area, the method further includes: In response to the user's close operation on the first interface, the first interface is closed, and the interface containing the icons of the first application and the second application is returned. or Display the original image; in response to a user inputting a first user operation on the original image, display the first cutout image; and in response to a user inputting a second user operation on the first cutout image, display the icon of the first application and the icon of the second application in the target area.

10. A service method based on image matting, characterized in that, Applied to electronic devices, the method includes: Display the original image; In response to a first user operation inputting the original image, the original image is subjected to image matting processing, and a first matted image obtained by performing image matting processing on the original image is displayed; In response to a second user operation inputting the first cutout image, one or more icons of a first application and one or more icons of a second application are displayed in the target area. The one or more first applications and the one or more second applications are determined from candidate applications based on at least one of the following information: attribute information of the first cutout image, usage frequency of candidate applications, running status, recent usage time, and application type. At least one of the one or more first applications and the one or more second applications is associated with the type of the first cutout image. The candidate applications include applications on the electronic device and / or applications on other devices that are communicatively connected to the electronic device. In response to a third user operation input to a first application, it is determined whether a target first application in one or more first applications has a low latency requirement. The third user operation is an operation of dragging the first cutout image into a first region of the target region and releasing it. The first region is the region where the icon of the target first application is located. The target first application is an application that requires image precision to be higher than a set image precision threshold. If it is determined that the target first application does not have low latency requirements, image filtering processing is performed on the first cutout image to obtain a third cutout image, and a first interface of the first application is displayed, the first interface including the third cutout image; if it is determined that the target first application has low latency requirements, a second interface of the second application is displayed, the second interface including the first cutout image; during the triggering of the first user operation, the second user operation, and the third user operation, the user's finger does not leave the screen of the electronic device, and the first cutout image and the third cutout image contain the same cutout object.

11. The method according to claim 10, characterized in that, After displaying the icons of the first and second applications in the target area, it also includes: In response to a fourth user action input by the user for the second application, a second interface of the second application is displayed, the second interface including the first cutout image.

12. The method according to claim 10, characterized in that, In response to a first user operation inputting the original image, image matting is performed on the original image, including: In response to a first user operation inputting the original image, the matting region where the target matting object is located in the original image is determined; The cutout region is cropped from the original image to form the first cutout image.

13. The method according to any one of claims 10-12, characterized in that, Displaying the first masked image obtained by performing masking processing on the original image, including: Identify the category information of the target object to be cut out in the original image; If it is determined, based on the category information of the target object, that the target object does not contain sensitive information, the first cutout image is displayed.

14. The method according to claim 13, characterized in that, Identifying the category information of the target object to be cut out includes: When the target object to be cut out is the target subject object in the original image, the original image is input into the image segmentation model for image segmentation to obtain the mask region of the target subject object; Based on the mask region of the target subject object, a target local image corresponding to the target subject object is cropped from the original image; The target local image is input into an image classification model to perform image classification on the target local image, thereby obtaining the category information of the target subject object.

15. A service method based on image matting, characterized in that, Applied to electronic devices, the method includes: Display the original image; In response to a first user operation inputting the original image, the original image is subjected to image matting processing, and a first matted image obtained by performing image matting processing on the original image is displayed; In response to a second user operation inputting the first cutout image, a target area is displayed, and icons of one or more first applications are displayed in the target area. The one or more first applications are determined from candidate applications based on at least one of the following information: attribute information of the first cutout image, usage frequency of candidate applications, running status, recent usage time, and application type. The first applications are associated with the type of the first cutout image, and the candidate applications include applications on the electronic device and / or applications on other devices that are communicatively connected to the electronic device. In response to a third user operation where the user selects a target first application in the target area, it is determined whether the target first application has low latency requirements. The third user operation is an operation of dragging the first cutout image into the first area and releasing it. The first area is the area where the icon of the target first application is located. The target first application is an application among the one or more first applications that requires image precision to be higher than a set image precision threshold. If it is determined that the target first application does not have low latency requirements, an operation is performed to perform image filtering on the first cutout image to obtain a third cutout image, and a first interface of the target first application is displayed. The first interface includes the third cutout image, and the first image corresponds to the third cutout image. If it is determined that the target first application has low latency requirements, a second interface of the second application is displayed. The second interface includes the first cutout image, and the first image and the first cutout image contain the same cutout object. During the triggering of the first user operation, the second user operation, and the third user operation, the user's finger does not leave the screen of the electronic device.

16. A service method based on image matting, characterized in that, Applied to electronic devices, the method includes: Display the original image; In response to a first user operation inputting the original image, the original image is subjected to image matting processing, and a first matted image obtained by performing image matting processing on the original image is displayed; In response to a second user operation inputting the first cutout image, a target area is displayed, and icons of one or more second applications are displayed in the target area. The one or more second applications are determined from candidate applications based on at least one of the following information: attribute information of the first cutout image, usage frequency of candidate applications, running status, recent usage time, and application type. The second applications are associated with the type of the first cutout image, and the candidate applications include applications on the electronic device and / or applications on other devices that are communicatively connected to the electronic device. In response to a fourth user operation where the user selects a target second application in the target area, a second interface is displayed. The second interface includes the first cutout image. The fourth user operation is an operation of dragging the first cutout image into the second area and releasing it. The second area is the area where the icon of the target second application is located in the one or more second applications. The target second application is an application that requires the image precision to be lower than or equal to a set image precision threshold. During the triggering of the first user operation, the second user operation, and the fourth user operation, the user's finger does not leave the screen of the electronic device.

17. An electronic device, characterized in that, include: Memory, processor, and display; The memory is used to store computer programs; the display screen is used to display images or application interfaces. The processor, coupled to the memory, is configured to execute the computer program to implement the steps of the method according to any one of claims 1-9, 10-14, 15, and 16.

18. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-9, 10-14, 15, and 16.

Images