Local screen capture method, electronic equipment and storage medium

Abstract

The invention provides a local screen capture method, electronic equipment and a storage medium. In the method, in response to a local screen capture operation executed by a user on a display interface of the electronic equipment, the electronic equipment can directly display a cross-application transmission interface. Wherein the cross-application transmission interface comprises a local screenshot and at least one icon. In response to an operation of clicking one icon by a user, the electronic equipment can transmit information included in the local screenshot to an application service corresponding to the icon. Thus, the user only needs to execute the local screenshot operation, the electronic equipment can display the cross-application transmission interface, so that the user can transmit information included in the local screenshot in a cross-application mode, the operation mode is simple and convenient, and the use experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of smart terminal technology, and in particular to a method for partial screenshotting, an electronic device, and a storage medium. Background Technology

[0002] With the development of technology, the screenshot function of electronic devices (such as mobile phones, tablets, etc.) has incorporated more user-friendly features, aiming to provide users with a more flexible and secure way of operation.

[0003] For example, electronic devices allow users to select (or box) a portion of the content displayed on the screen, enabling them to capture only a part of the content instead of the entire screen. This partial screenshot function not only improves the accuracy of the screenshot but also enhances the user's control over the captured content. Summary of the Invention

[0004] This application provides a method for partial screenshotting, an electronic device, and a storage medium. In this method, the user only needs to perform a partial screenshot operation to enable the electronic device to display a cross-application transfer interface, facilitating the transfer of information included in the partial screenshot across applications. The operation is simple and convenient.

[0005] In a first aspect, embodiments of this application provide a method for partial screenshotting. The method includes: an electronic device displaying a first interface; the electronic device, in response to a user's first operation on the first interface, displaying a second interface, the second interface including a first image and a first list, the first image being a partial screenshot of the first interface, the first image including first information, and the first list including at least one icon; the electronic device, in response to a user's first operation on a first icon in the first list, transmitting the first information to the application service corresponding to the first icon.

[0006] The first interface can be the interface of any application, the second interface can be the cross-application transfer interface mentioned below, and the first operation can be a partial screenshot operation.

[0007] The icons included in the first list refer to the icons of various applications recommended to the user. Specifically, they may be recommended to the user based on the first information included in the first image. For example, the first information may be image information or text information, and this embodiment does not limit it.

[0008] For example, the first action could be a click action.

[0009] In this way, users only need to perform a partial screenshot operation to enable the electronic device to display a cross-application transfer interface, so that users can transfer the information included in the partial screenshot across applications. The operation is simple and convenient, improving the user experience.

[0010] According to the first aspect, the first operation is the operation of drawing a trajectory on the first interface.

[0011] According to the first aspect, or any implementation of the first aspect above, the first operation is to draw a trajectory on the first interface using a knuckle.

[0012] According to the first aspect, or any implementation of the first aspect above, the shape of the first image is independent of the shape of the trajectory.

[0013] In this embodiment, the shape of the first image is not determined based on the shape of the user's selection trajectory; that is, the first image is not obtained by directly cropping the first interface based on the user's selection trajectory.

[0014] According to the first aspect, or any implementation of the first aspect above, the shape of the trajectory selected by the user is an arbitrary shape, and the shape of the first image is a rectangle.

[0015] According to the first aspect, or any implementation of the first aspect above, the first text information is displayed on the first interface, and when the trajectory surrounds a part of the first text information, the first image is a screenshot of the first text information.

[0016] For example, the first text information can be any line of text displayed on the first interface, or any paragraph of text displayed on the first interface, etc.

[0017] In this embodiment, the user selects text information on the first interface. When the user's selection on the first interface encloses a portion of the first text information, the partial screenshot displayed on the cross-application transmission interface of the electronic device corresponds to the partial screenshot of the first text information. Thus, even if the user's selected text information is incomplete—for example, if the user selects only part of a line of text—the electronic device can still crop the image corresponding to the entire line of text as a partial screenshot on the first interface, achieving automatic adjustment of the partial screenshot area and improving the user experience.

[0018] According to the first aspect, or any implementation of the first aspect above, the second image is displayed on the first interface, and when the trajectory surrounds a part of the second image, the first image is a screenshot of the second image.

[0019] The second image can be any image displayed on the first interface. The image may or may not contain text information, and this embodiment does not limit this.

[0020] In this embodiment, the user selects image information on the first interface. When the trajectory selected by the user on the first interface encloses a local area of ​​the second image, the local screenshot is taken as the second image in the cross-application transmission interface displayed on the electronic device. Thus, even if the image information selected by the user is incomplete, the electronic device can crop the entire second image as a local screenshot on the first interface, achieving the effect of automatically adjusting the local screenshot range and improving the user experience.

[0021] According to the first aspect, or any implementation of the first aspect above, the second text information and the third image are displayed on the first interface. When the trajectory surrounds a part of the second text information and a part of the third image, the first image is a screenshot of the second text information, or the first image is a screenshot of the third image, or the first image is a screenshot of the second text information and the third image.

[0022] For example, the second text information can be any line of text displayed on the first interface, or any paragraph of text displayed on the first interface. The third image can be any image displayed on the first interface, which may or may not contain text information; this embodiment does not limit this.

[0023] In this embodiment, the user simultaneously selects text information and image information on the first interface. When the trajectory selected by the user on the first interface encloses a portion of the second text information and a portion of the third image, the partial screenshot displayed on the cross-application transmission interface of the electronic device can be a partial screenshot corresponding to the second text information, the third image, or a partial screenshot containing both the second text information and the third image. In this way, the electronic device can automatically adjust the partial screenshot range according to the user's selection trajectory, improving the user experience.

[0024] According to the first aspect, or any implementation of the first aspect above, when the electronic device displays the second interface, the method further includes: the electronic device responding to a second operation by the user on the second interface to display a third interface, the third interface including a fourth image and a second list, the fourth image being a partial screenshot of the first interface, the fourth image including second information, and the second list including at least one icon; the electronic device responding to a first operation by the user on a second icon in the second list to transmit the second information to the application service corresponding to the second icon.

[0025] The second operation involves adjusting the currently generated partial screenshot (i.e., the first image mentioned above) on the second interface. The third interface may be the cross-application transfer interface mentioned below, and the fourth image is the partial screenshot redefined by the electronic device.

[0026] The icons included in the second list refer to the icons of various applications recommended to the user, specifically those recommended based on the second information included in the fourth image. For example, the second information can be image information or text information; this embodiment does not limit this.

[0027] In this way, if the partial screenshot displayed in the second interface does not meet the user's expectations, the user can make adjustments so that the electronic device can re-crop the partial screenshot in the first interface, so that the user can transmit the information contained in the partial screenshot across applications.

[0028] According to the first aspect, or any implementation of the first aspect above, the first cursor and the last cursor are displayed on the first image; the second operation includes: adjusting the position of the first cursor and / or the last cursor; the fourth image is a partial screenshot redefined in the first interface based on the adjusted positions of the first cursor and the last cursor; the second information includes text information selected by the first cursor and the last cursor on the fourth image.

[0029] In this embodiment, the user can adjust the position of the first cursor and / or the last cursor displayed on the first image so that the electronic device can re-crop a partial screenshot in the first interface according to the adjusted positions of the first cursor and the last cursor. In this way, the user can transmit the text information selected by the first cursor and the last cursor in the partial screenshot across applications.

[0030] According to the first aspect, or any implementation of the first aspect above, the positions of the first cursor and the last cursor on the first image are determined based on the selection trajectory drawn by the user on the first interface. For example, the positions of both the first cursor and the last cursor are located on the selection trajectory drawn by the user on the first interface.

[0031] According to the first aspect, or any implementation of the first aspect above, the second interface includes a first control, the first control being in a first state; the electronic device transmitting first information to the application service corresponding to the first icon includes: the electronic device transmitting text information included in the first image to the application service corresponding to the first icon. Correspondingly, the second operation includes: adjusting the state of the first control to a second state; the electronic device transmitting second information to the application service corresponding to the second icon includes: the electronic device transmitting a fourth image to the application service corresponding to the first icon.

[0032] For example, the first control is the text recognition option mentioned below.

[0033] For example, the first state is the selected state, and the second state is the unselected state.

[0034] Optionally, the first and fourth images can be the same.

[0035] Thus, when the first control is selected, the electronic device transmits the text information included in the first image to the application service corresponding to the first icon. When the user changes the state of the first control to unselected, the electronic device transmits the fourth image to the application service corresponding to the first icon.

[0036] According to the first aspect, or any implementation of the first aspect above, the second interface includes a second control. Correspondingly, the method further includes: the electronic device, in response to a third operation by the user on the second control, displays a fourth interface; the fourth interface includes a first image and at least one third control, the third control being used for editing the first image.

[0037] For example, the second control is the screenshot editing control mentioned below, and the third operation can be a click operation.

[0038] For example, the fourth interface is the screenshot editing interface mentioned below, which can be referred to. Figure 1a (4) or Figure 2a The interface shown in (4) is as follows. The third control can be any control used for editing partial screenshots in the screenshot editing interface, as can be found in [reference]. Figure 1a (4) or Figure 2a The relevant controls in the interface shown in (4).

[0039] According to the first aspect, or any implementation of the first aspect above, the second interface also includes a first sub-interface, the content of which is the same as that of the first interface; the first sub-interface is obtained by performing a three-dimensional transformation on the first interface.

[0040] In this way, the first interface is like a door, and the change from the first interface to the first sub-interface can create a "door opening" visual effect on the screen, thereby enhancing the user's visual experience.

[0041] According to the first aspect, or any implementation of the first aspect above, before the electronic device displays the second interface, the method further includes: playing an animation; the animation includes at least one of the following motion effects: a motion effect of generating a screenshot frame based on a trajectory, a motion effect of obtaining a partial screenshot by cropping based on the screenshot frame, a motion effect of scanning the partial screenshot, and a motion effect of transforming the first interface into a first sub-interface.

[0042] In this way, the animation serves as a transition effect when an electronic device switches from the first interface to the second interface, making the interface switching of the electronic device smoother and more continuous, thus improving the user experience.

[0043] According to the first aspect, or any implementation of the first aspect above, before the electronic device displays the third interface, the method further includes: playing an animation; the animation includes at least one of the following motion effects: an animation effect of adjusting a partial screenshot, an animation effect of scanning the partial screenshot, and an animation effect of generating the third interface.

[0044] According to the first aspect, or any implementation of the first aspect above, before the electronic device displays the second interface, the method further includes: the electronic device performing layout analysis on the first interface to obtain layout analysis results; wherein, the layout analysis results include information about the image area, information about the text area, and all text information contained in the first interface; the electronic device matching the area enclosed by the trajectory with the image area and the text area to determine the first screenshot area; and the electronic device cropping the first image from the first interface based on the first screenshot area.

[0045] In this way, the electronic device performs layout analysis on the first interface based on the layout analysis algorithm, and determines the target text area and / or image area by combining the user's selection trajectory with the layout analysis results. Even if the user's selection range is inaccurate, the electronic device will automatically adjust the local screenshot range based on the selection trajectory and layout analysis results, simplifying the user's operation steps and improving the user experience.

[0046] According to the first aspect, or any implementation of the first aspect above, the electronic device matches the area enclosed by the trajectory with the image area and the text area to determine the first screenshot area, including: if the area enclosed by the trajectory is contained in the target image area, and the area ratio of the area enclosed by the trajectory in the target image area reaches a preset threshold, then the electronic device determines the target image area as the first screenshot area.

[0047] For example, the preset threshold can be 70% or 60%, etc.

[0048] According to the first aspect, or any implementation of the first aspect above, the electronic device matches the area enclosed by the trajectory with the image area and the text area to determine the first screenshot area, including: if the area enclosed by the trajectory is contained in the target text area, and the area ratio of the area enclosed by the trajectory in the target text area reaches a preset threshold, then the electronic device determines the target text area as the first screenshot area.

[0049] According to the first aspect, or any implementation of the first aspect above, the electronic device matches the area enclosed by the trajectory with the image area and the text area to determine the first screenshot area, including: if the area enclosed by the trajectory intersects with at least one image area and at least one text area, the electronic device determines each valid area in at least one image area and / or text area; wherein, the area ratio of the intersection of the valid area and the area enclosed by the trajectory in the valid area reaches a preset threshold; the electronic device takes one area in the first interface that contains each valid area as the first screenshot area.

[0050] According to the first aspect, or any implementation of the first aspect above, the electronic device transmits the first information to the application service corresponding to the first icon, including: if the first screenshot area only includes a text area, the electronic device transmits the text information included in the first image to the application service corresponding to the first icon; if the first screenshot area includes an image area, the electronic device transmits the first image to the application service corresponding to the first icon.

[0051] In this way, electronic devices can flexibly determine the type of information to be transmitted in the cross-application transmission interface based on the content involved in the screenshot area.

[0052] According to the first aspect, or any implementation of the first aspect above, before the electronic device displays the second interface, the method further includes: if the first screenshot area only includes a text area, the electronic device performs intent recognition based on the text information included in the first image, and determines a first list based on the intent recognition result; if the first screenshot area includes an image area, the electronic device performs intent recognition based on the first image, and determines a first list based on the intent recognition result.

[0053] In this way, electronic devices can identify the user's intent based on the content of the screenshot area to determine the list of applications to recommend, thereby improving the accuracy of application service recommendations.

[0054] According to the first aspect, or any implementation of the first aspect above, before the electronic device displays the third interface, the method further includes: the electronic device responding to the user's second operation on the second interface to determine a second screenshot area; the electronic device cropping a fourth image from the first interface based on the second screenshot area; the electronic device performing intent recognition based on the second information included in the fourth image, and determining a second list based on the intent recognition result.

[0055] In this way, when the partial screenshot displayed on the second interface does not meet the user's expectations, and the user adjusts the partial screenshot, the electronic device re-crops the partial screenshot on the first interface and performs intent recognition on the partial screenshot again to re-recommend application services to the user, further improving the accuracy of application service recommendations.

[0056] According to the first aspect, or any implementation of the first aspect above, when the area of ​​the region enclosed by the trajectory is less than the area threshold, the first interface is cropped according to the circumscribed rectangular region of the trajectory to obtain the first image.

[0057] According to the first aspect, or any implementation of the first aspect above, the first image is obtained by cropping the third screenshot area in the first interface, where the third screenshot area is the bounding rectangle area of ​​the trajectory.

[0058] According to the first aspect, or any implementation of the first aspect above, before the electronic device displays the second interface, the method further includes: the electronic device performing layout analysis on the first interface to obtain layout analysis results; wherein, the layout analysis results include information on image areas, information on text areas, and all text information contained in the first interface; if the area enclosed by the trajectory intersects with at least one image area and / or text area, and at least one image area and / or text area is not a valid area, then the electronic device determines the bounding rectangle area of ​​the trajectory as the third screenshot area; wherein, the area ratio of the intersection of the valid area and the area enclosed by the trajectory in the valid area reaches a preset threshold.

[0059] In this way, when the electronic device cannot determine the effective area based on the user's selected trajectory and the layout analysis results, a partial screenshot is taken based on the bounding rectangle of the trajectory, so that the partial screenshot matches the user's selected area as closely as possible.

[0060] According to the first aspect, or any implementation of the first aspect above, the second interface includes a first control (such as a text recognition option), and the state of the first control is a second state (such as an unselected state). In this way, when the electronic device performs a partial screenshot based on the bounding rectangle of the trajectory, it predicts that the information to be transmitted by the user is an image rather than text, so the text recognition option displayed on the second interface is in an unselected state.

[0061] According to the first aspect, or any implementation of the first aspect above, when the electronic device displays the second interface, the method further includes: the electronic device displays a fifth interface in response to the user's operation of adjusting the state of the first control to a first state (such as a selected state); the fifth interface includes a fifth image and a third list, the fifth image is a partial screenshot of the first interface, the text information in the fifth image is more than the text information in the first image, and the third list includes at least one icon; the electronic device transmits the text information in the fifth image to the application service corresponding to the third icon in the third list in response to the user's first operation on the third icon in the third list.

[0062] In this way, when an electronic device takes a partial screenshot based on the bounding rectangle of the trajectory, if the user changes the information to be transmitted from an image to text, the electronic device will automatically enlarge the partial screenshot to make the text information on the partial screenshot in the cross-application transmission interface more complete and easier for the user to select.

[0063] Secondly, embodiments of this application provide an electronic device. The electronic device includes: one or more processors; a memory; a display for displaying an interface; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when executed by the one or more processors, the electronic device performs the partial screenshot method of the first aspect and any one thereof.

[0064] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0065] Thirdly, embodiments of this application provide a computer-readable storage medium. This computer-readable storage medium includes a computer program that, when run on an electronic device, causes the electronic device to perform the partial screenshot method of the first aspect and any one thereof.

[0066] The third aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the third aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here. Attached Figure Description

[0067] Figures 1a-1b An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0068] Figures 2a-2b An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0069] Figure 3 An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0070] Figure 4 An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0071] Figure 5 A schematic diagram of the hardware structure of an electronic device as an example;

[0072] Figure 6A schematic diagram of the software structure of an electronic device as an example;

[0073] Figure 7a This is a schematic diagram illustrating the module interactions involved in the partial screenshot method;

[0074] Figure 7b An exemplary diagram illustrating the layout analysis results of an image is shown;

[0075] Figure 8a An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0076] Figures 8b-8c An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0077] Figures 8d-8e An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0078] Figures 8f-8g An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0079] Figures 8h-8j An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0080] Figure 9a An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0081] Figures 9b-9c An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0082] Figures 9d-9e An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0083] Figure 9f An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0084] Figures 10a-10b An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0085] Figure 10c An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0086] Figures 10d-10e An exemplary diagram illustrating a scenario of selecting a screenshot is provided.

[0087] Figure 10f An exemplary diagram illustrating a scenario of selecting a screenshot is provided. Detailed Implementation

[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0090] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0091] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0092] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0093] Partial screenshot is a screen capture feature that allows users to select a specific area of ​​the screen to capture, rather than taking a screenshot of the entire screen. This feature is especially common on electronic devices such as mobile phones and tablets, providing users with great convenience and enabling them to accurately capture any area of ​​the screen. The screen area corresponding to a partial screenshot is not limited to a rectangular area; it can also be a circular, elliptical, heart-shaped, or other any other closed shape, providing users with greater flexibility and creativity.

[0094] In one implementation, users can take partial screenshots by selecting a shape on the screen. This is achieved by the user selecting a closed shape (such as a rectangle or ellipse) on the electronic device screen. For example, the user can select a shape, and the movement trajectory of the user's knuckle will be displayed on the screen. After the selection is complete, the screen will display the area selected based on the knuckle movement trajectory. The user can then edit, save, or share the selected area using the screenshot tools menu.

[0095] Figures 1a-1b An example is shown of an application scenario where text content is captured based on partial screenshot functionality and then transmitted across applications.

[0096] like Figure 1a As shown in (1), the electronic device displays a user interface 10. The user interface 10 can be the program interface of a browser application in the electronic device, or it can be the program interface of other applications in the electronic device; this embodiment does not limit this. In response to the user's selection of a screenshot area, a partial screenshot frame is displayed on the user interface 10. For example, refer to... Figure 1a In steps (1) and (2), in response to the user tapping the screen with their knuckle (keeping the knuckle on the screen) and dragging the knuckle to draw a closed shape (such as an ellipse), the movement trajectory 101 of the knuckle (i.e., the user's screenshot trajectory or selection trajectory) is displayed on the user interface 10. In response to the user completing the drawing operation, the electronic device displays the user interface 11, which can be referenced... Figure 1a As shown in (3). For example, the user interface 11 may include, but is not limited to, a thumbnail 111, a partial screenshot frame 112 (ellipse), a partial screenshot frame adjustment control 113, a screenshot editing control 114, a screenshot save control 115, a return control 116, a screenshot sharing control 117, and a screenshot frame shape option area 118.

[0097] It should be noted that the shapes of the motion trajectory 101 and the partial screenshot frame 112 are the same or nearly the same. For example, when the shape of the motion trajectory 101 is circular, the shape of the partial screenshot frame 112 is also circular, and the two are exactly the same or almost the same in size; when the shape of the motion trajectory 101 is heart-shaped, the shape of the partial screenshot frame 112 is also heart-shaped, and the two are exactly the same or almost the same in size.

[0098] Wherein: thumbnail 111 can be a thumbnail of the aforementioned user interface 10.

[0099] The area image enclosed by the partial screenshot frame 112 is the screenshot image determined by the electronic device in response to the user's knuckle selection screenshot operation on the user interface 10. In this embodiment, the user can adjust the size of the partial screenshot frame 112 by dragging the partial screenshot frame adjustment control 113 to adjust the range of the area enclosed by the partial screenshot frame 112. In this embodiment, the user can also drag the partial screenshot frame 112 on the thumbnail 111 to adjust the position of the partial screenshot frame 112 on the thumbnail 111, thereby adjusting the content of the area image enclosed by the partial screenshot frame 112.

[0100] The screenshot editing control 114 is used to trigger the electronic device to display the editing interface corresponding to the partial screenshot.

[0101] The screenshot save control 115 is used to trigger the electronic device to save the image of the area enclosed by the partial screenshot frame 112.

[0102] The return control 116 is used to trigger the electronic device to exit the display user interface 11.

[0103] The screenshot sharing control 117 is used to trigger the display of a sharing interface corresponding to a partial screenshot on an electronic device.

[0104] The screenshot frame shape option area 118 includes multiple shape options, such as freeform, rectangle, ellipse, and heart shapes. The electronic device can respond to a user's click on any of the graphic options in the screenshot frame shape option area 118, adjusting the shape of the partial screenshot frame 112 to the shape indicated by the corresponding graphic option. Figure 1a In the example shown in (3), the ellipse option in the screenshot shape option area 118 is displayed in bold, indicating that the shape of the partial screenshot frame 112 is elliptical. It should be noted that when the user adjusts the shape of the partial screenshot frame 112, the image of the area enclosed by the partial screenshot frame 112 will also be adjusted adaptively.

[0105] Continue to refer to Figure 1a In steps (3) and (4), in response to a user clicking the screenshot editing control 114, the electronic device displays the user interface 12. For example, the user interface 12 may include, but is not limited to, a partial screenshot 121, a partial screenshot frame 122, a partial screenshot frame adjustment control 123, a screenshot frame shape option area 124, a text recognition control 125, an image editing option area 126, a scrolling screenshot control 127, and a screenshot sharing control 128.

[0106] The partial screenshot 121 is the image of the area enclosed by the partial screenshot frame 122. In this embodiment, the user can adjust the size of the partial screenshot frame 122 by dragging the partial screenshot frame adjustment control 123 to adjust the content of the partial screenshot 121.

[0107] The screenshot frame shape option area 124 includes multiple shape options, such as freeform, rectangle, ellipse, and heart shapes. The electronic device can respond to a user's click on any of the graphic options in the screenshot frame shape option area 124, adjusting the shape of the partial screenshot frame 122 to the shape indicated by the corresponding graphic option. Figure 1a In the example shown in (4), the ellipse option in the screenshot shape option area 124 is displayed in bold, indicating that the shape of the partial screenshot frame 112 is elliptical. It should be noted that when the user adjusts the shape of the partial screenshot frame 122, the image of the area enclosed by the partial screenshot frame 122 will also be adjusted adaptively.

[0108] The text recognition control 125 is used to trigger the electronic device to recognize the text information in the partial screenshot 121.

[0109] The image editing options area 126 includes multiple editing options, such as pen shape selection options, color selection options, pen thickness selection options, doodling options, mosaic options, eraser options, etc. These editing options are used to edit the partial screenshot 121.

[0110] The scrolling screenshot control 127 is used to trigger electronic devices to perform interface scrolling and long screenshot operations.

[0111] The screenshot sharing control 128 is used to trigger the display of a sharing interface corresponding to a partial screenshot on an electronic device.

[0112] Assume that, in this application scenario, the text information in partial screenshot 121 is the information needed by the user. Continue referring to... Figure 1a Zhong (4) and Figure 1b In step (1), in response to the user clicking the text recognition control 125, the electronic device recognizes the text information in the partial screenshot 121 and displays the user interface 13 after recognition is completed. Optionally, the electronic device displays... Figure 1a When the user interface 12 shown in (4) is in use (at this time, the user interface 12 may not include the text recognition control 125), the electronic device responds to the user's operation of long-pressing the partial screenshot 121, recognizes the text in the partial screenshot 121, and displays the user interface 13 after the recognition is completed (at this time, the user interface 13 may not include the text recognition control).

[0113] Continue to refer to Figure 1b In the user interface 13, the text information 131 in the partial screenshot 121 is selected by the first cursor 132 and the last cursor 133. At this time, the user can perform a long press operation on the text information 131. In response to the user's long press operation on the text information 131, the text 141 floats up, as shown in the image. Figure 1b As shown in (2).

[0114] Understandably, text information 131 and text 141 contain the same text content. The user drags text 141 towards the side of the electronic device screen, for example, dragging text 141 towards the right side of the screen. In response to the user dragging text 141 to a preset position, the electronic device displays user interface 15, which can be referred to... Figure 1b As shown in (3). The user interface 15 can be understood as an interface for transmitting information across applications (or a cross-application transmission interface), and may include, but is not limited to, sub-interfaces 151, text 152, and icon areas 153. Among them:

[0115] The elements displayed in sub-interface 151 (such as text, images, etc.) are the same as those displayed in user interface 14. Sub-interface 151 can be considered as the interface obtained by the electronic device after performing a three-dimensional transformation of all elements in user interface 14. In user interface 15, sub-interface 151 can create a visual effect on the screen that makes the original interface (i.e., user interface 14) appear to push inwards. User interface 14 is like a door, and sub-interface 151 can create the visual effect of "opening the door" on the screen.

[0116] The text content displayed in text 152 is the same as the text content displayed in text 141 and text information 131. Optionally, text 152 can be a scaled-down version of text 141 obtained by an electronic device; text 152 and text 141 are identical except for the size of the area they occupy. This way, when the user drags text 152, it will not obscure a large area of ​​the application icon (or application service icon) and its corresponding text description in the icon area 153, making it easier for the user to select the desired application icon or application service icon.

[0117] The icon area 153 can be used to display one or more application icons or application service icons recommended by an electronic device based on the information element selected by the user (i.e., text information 131 in this embodiment). Optionally, each icon in the icon area 153 has a corresponding text description below it, which can be used to indicate the application name or service name corresponding to the icon. For example, when the icon is an application... When the icon is a WeChat icon, the corresponding text description can be "WeChat"; when the icon is an application icon... When the icon represents the "Moments" app service, the corresponding text description can be either "Moments" or "WeChat Moments"; when the icon represents an app... When using the "Scan" icon in the application service, the corresponding text description can be "Scan" or "WeChat Scan". It should be understood that the text below each icon in icon area 153 is only for the user's convenience in understanding the specific application or service function corresponding to the icon; this embodiment does not limit the specific icon style or text content.

[0118] Continue to refer to Figure 1b In step (3), after the electronic device displays the user interface 15, the user continues to drag the text 152 onto the icon 1531 of the "Notes" application and releases their hand. It should be understood that the process of the user dragging the text is continuous; the user's finger does not need to leave the screen of the electronic device throughout the entire process, until the text is dragged onto the icon of the "Notes" application and released. In response to this release operation, the electronic device transmits the text content corresponding to the text 152 to the "Notes" application and displays the program interface of the "Notes" application. For example, as shown... Figure 1b As shown in (4), in response to the user's operation, the text content corresponding to text 152 is transmitted to the "Notes" application, and the user interface 16 is displayed. The user interface 16 includes a sub-interface 161 and a window 162 of the "Notes" application. The element content (e.g., text, images, etc.) displayed in the sub-interface 161 is the same as the element content displayed in the user interface 10 (except for the status bar), that is, the element content displayed in the sub-interface 161 is the element content displayed on the electronic device before the user selects the screenshot with their knuckle (except for the status bar). The window 162 of the "Notes" application can be displayed on the sub-interface 161 in the form of a floating window. The "Notes" application interface displayed in the window 162 of the "Notes" application includes the text content corresponding to text 152.

[0119] Understandably, after the electronic device displays user interface 15, the user can continue to drag and drop text 152 onto icons of other applications and then release it. In response to this release, the electronic device transfers the text content corresponding to text 152 to the corresponding application and displays the corresponding application interface (which includes the text content corresponding to text 152) or displays feedback indicating that the information transfer is complete. Optionally, the feedback may also include an option to jump to the corresponding application interface.

[0120] It should be pointed out that, in cases such as Figure 1bThe text content selected by the first cursor 132 and the last cursor 133 in the user interface 13 shown in (1) may not meet the user's expectations. In this case, the user can adjust the selected text content within the selected area of ​​the screenshot. In response to the user's operation of adjusting the position of the first cursor 132 and / or the last cursor 133, the electronic device adjusts the text content selected by the first cursor 132 and the last cursor 133. At this time, in response to the user's operation of dragging the text selected by the first cursor 132 and the last cursor 133 to the icon of the "Notes" application and releasing the hand, the electronic device transfers this text content to the "Notes" application. For a detailed explanation of the text content transfer, please refer to the above content, which will not be repeated here. In this way, the user can transfer the text content they need to other applications or application services through the cross-application information transfer interface (as shown in user interface 15).

[0121] Figures 2a-2b An example is shown of an application scenario where image content is captured based on a partial screenshot function and the image is transmitted across applications.

[0122] like Figure 2a As shown in (1), the electronic device displays a user interface 20. The user interface 20 can be the program interface of a browser application within the electronic device, or it can be the program interface of other applications within the electronic device; this embodiment does not limit this. In response to the user's selection of a screenshot area, a partial screenshot frame is displayed on the user interface 20. For example, refer to... Figure 2a In steps (1) and (2), in response to the user tapping the screen with their knuckle (keeping the knuckle on the screen) and then dragging the knuckle to draw a closed shape (such as a rectangle), the movement trajectory 201 of the knuckle (i.e., the user's screenshot trajectory) is displayed on the user interface 20. In response to the user completing the drawing operation, the electronic device displays the user interface 21, which can be referenced... Figure 2a As shown in (3). For example, the user interface 21 may include, but is not limited to, a thumbnail 211, a partial screenshot frame 212 (rectangle), a partial screenshot frame adjustment control 213, a screenshot editing control 214, a screenshot save control 215, a return control 216, a screenshot sharing control 217, and a screenshot frame shape option area 218.

[0123] It should be noted that the shapes of the motion trajectory 201 and the partial screenshot frame 212 are the same or basically the same.

[0124] The thumbnail 211 can be a thumbnail of the aforementioned user interface 20. The image area enclosed by the partial screenshot frame 212 is the screenshot image determined by the electronic device in response to the user's knuckle selection screenshot operation on the user interface 20. In this embodiment, the user can adjust the size of the partial screenshot frame 212 by dragging the partial screenshot frame adjustment control 213 to adjust the range of the area enclosed by the partial screenshot frame 212. In this embodiment, the user can also drag the partial screenshot frame 212 on the thumbnail 211 to adjust the position of the partial screenshot frame 212 on the thumbnail 211, thereby adjusting the content of the image area enclosed by the partial screenshot frame 212.

[0125] For explanations of the other controls displayed in user interface 21, please refer to... Figure 1a The examples shown will not be repeated here.

[0126] Continue to refer to Figure 2a In steps (3) and (4), in response to a user clicking the screenshot editing control 214, the electronic device displays the user interface 22. For example, the user interface 22 may include, but is not limited to, a partial screenshot 221, a partial screenshot frame 222, a partial screenshot frame adjustment control 223, a screenshot frame shape option area 224, a text recognition control 225, an image editing option area 226, a scrolling screenshot control 227, and a screenshot sharing control 228.

[0127] Among them, partial screenshot 221 is the image of the area enclosed by partial screenshot frame 222.

[0128] However, the area selected by the user in the screenshot may not be accurate. For example, in areas like... Figure 2a In the interface shown in (4), the partial screenshot 221 may be incomplete and not the screenshot the user needs. In this embodiment, the user can adjust the size of the partial screenshot frame 222 by dragging the partial screenshot frame adjustment control 223 to adjust the content of the partial screenshot 221. For explanations of other controls displayed in the user interface 22, please refer to Figure 1a The examples shown will not be repeated here.

[0129] Continue to refer to Figure 2a In step (4), the user presses and holds the upper control 2231 in the partial screenshot frame adjustment control 223 and drags it upwards. In response to this user action, the electronic device expands the partial screenshot frame 222 upwards, and the partial screenshot frame adjustment control 223 also adjusts adaptively as the size of the partial screenshot frame 222 increases. For example, in response to the user's dragging operation, the electronic device displays the user interface 23, such as... Figure 2b As shown in (1). Comparison Figure 2a Zhong (4) and Figure 2bAs can be seen from (1), the user's adjustment operation will adjust the partial screenshot frame 222 to the partial screenshot frame 222. Correspondingly, the partial screenshot 221 will be adjusted to the partial screenshot 231, and the partial screenshot frame adjustment control 223 will be adapted to the partial screenshot frame adjustment control 233.

[0130] Assume that the partial screenshot 231 in user interface 23 is the screenshot required by the user. Continue referring to... Figure 2b In (1) and (2), the user can perform a long press operation on the partial screenshot 231. In response to the user's long press operation on the partial screenshot 231, the screenshot 241 floats up, as shown. Figure 2b As shown in (2). It is understood that the image content included in partial screenshot 231 and screenshot 241 is the same. The user drags screenshot 241 towards the side of the electronic device screen, for example, the user drags screenshot 241 towards the right side of the electronic device screen. In response to the user dragging screenshot 241 to the right to a preset position, the electronic device displays user interface 25, which can be referred to... Figure 2b As shown in (3). The user interface 25 can be understood as an interface for transmitting information across applications, and may include, but is not limited to, sub-interfaces 251, screenshots 252, and icon areas 253. Among them:

[0131] The elements displayed in sub-interface 251 (such as text, images, etc.) are the same as those displayed in user interface 24. Sub-interface 251 can be considered as the interface obtained by the electronic device after performing a three-dimensional transformation of all elements in user interface 24. In user interface 25, sub-interface 251 can create a visual effect on the screen that makes the original interface (i.e., user interface 24) appear to be pushing inwards. User interface 24 is like a door, and sub-interface 251 can create the visual effect of "opening the door" on the screen.

[0132] The content displayed in screenshot 252 is the same as that displayed in screenshot 241 and partial screenshot 231. Optionally, screenshot 252 can be obtained by shrinking screenshot 241 on an electronic device; screenshot 252 and screenshot 241 are identical except for the size of the area they occupy. This way, when the user drags screenshot 252, it will not obscure a large area of ​​the application icon (or application service icon) and its corresponding text description in icon area 253, making it easier for the user to select the desired application icon or application service icon. For an explanation of icon area 253, please refer to... Figure 1b The examples shown will not be repeated here.

[0133] Continue to refer to Figure 2bIn step (3), after the electronic device displays the user interface 25, the user continues to drag the screenshot 252 onto the "Gallery" application icon 2531 and releases their hand. It should be understood that the process of dragging the screenshot is continuous; the user's finger does not need to leave the screen of the electronic device throughout the entire process, until the screenshot is dragged onto the "Gallery" application icon and released. In response to this release operation, the electronic device transmits the image content corresponding to the screenshot 252 to the "Gallery" application and displays feedback information. For example, as shown... Figure 2b As shown in (4), in response to the user's operation, the image content corresponding to screenshot 252 is transferred to the "Gallery" application, and user interface 26 is displayed. User interface 26 includes sub-interface 261 and feedback information 262. The element content (e.g., text, images, etc.) displayed in sub-interface 261 is the same as the element content displayed in user interface 20 (except for the status bar), that is, the element content displayed in sub-interface 261 is the element content displayed on the electronic device before the user selects the screenshot with their knuckle (except for the status bar). The content displayed in feedback information 262 can be used to indicate that the screenshot has been saved to the Gallery application. For example, feedback information 262 may also include a "Go to View" option 2621. The user can open the "Gallery" application by clicking the "Go to View" option 2621, so that the electronic device displays the program interface of the "Gallery" application.

[0134] Understandably, after the electronic device displays user interface 25, the user can continue to drag and drop screenshot 252 onto icons of other applications and then release it. In response to this release, the electronic device transfers the image content corresponding to screenshot 252 to the corresponding application and displays the corresponding application interface (which includes the image content corresponding to screenshot 252) or displays feedback indicating that the information transfer is complete. Optionally, the feedback may also include an option to jump to the corresponding application interface.

[0135] As an optional implementation, when the electronic device displays such as Figure 2a When the user interface 21 shown in (3) is active, the electronic device can also adjust the partial image enclosed by the partial screenshot frame in response to the user's operation on the partial screenshot frame adjustment control 213. At this time, if the user long-presses the partial image enclosed by the partial screenshot frame, the electronic device can trigger the corresponding screenshot image to float up. Then, the user can drag the floated screenshot image to the side of the electronic device to make the electronic device display the cross-application information transmission interface. In this way, the user can transmit the image content they need to other applications or application services through the cross-application information transmission interface. For details not explained in this embodiment, please refer to Figures 2a-2b The examples shown will not be repeated here.

[0136] As mentioned above, after selecting and taking a screenshot, the user can trigger the electronic device to perform text recognition on the resulting partial screenshot to obtain text information. The user can then drag and drop the text information to display an interface for cross-application information transfer, where they can complete the cross-application text transfer operation. Alternatively, after selecting and taking a screenshot, the user can drag and drop the screenshot image to display an interface for cross-application information transfer, where they can complete the cross-application image transfer operation.

[0137] This user operation method is relatively cumbersome. Moreover, if the user's selection area is inaccurate when performing the selection screenshot operation, the user needs to manually adjust the selected area, which further affects the user experience.

[0138] This application provides a method for partial screenshotting. In this method, after a user manually selects a portion of the target interface for screenshotting, the electronic device performs layout analysis on the target interface based on a layout analysis algorithm, and determines the target text area and / or image area by combining the user's selection trajectory and the layout analysis results. The electronic device performs a partial screenshot based on the target text area and / or image area, and performs intent recognition based on the content of the target text area and / or image area to obtain a recommended application list. Subsequently, the electronic device displays the adjusted selected screenshot and the recommended application list.

[0139] In this way, users only need to select a portion of the screen to take a screenshot, and the electronic device will display an interface for cross-application information transfer. Users can then complete the cross-application information transfer within this interface. This user operation method is relatively simple. Furthermore, even if the user's selection is inaccurate, the electronic device will automatically adjust the screenshot area, further simplifying the user's steps and improving the user experience.

[0140] Figure 3 An exemplary application scenario diagram is shown. In this scenario, a user selects a text area in the interface to take a screenshot, triggering the electronic device to display a cross-application transfer interface.

[0141] like Figure 3As shown in (1), the electronic device displays a user interface 30. The user interface 30 can be the program interface of the "browser" application in the electronic device, or it can be the program interface of other applications in the electronic device. This embodiment does not limit this. In response to the user's operation of selecting a screenshot, the user's screenshot trajectory 301 is displayed on the user interface 30. For example, taking the user's selection of a screenshot using a knuckle as an example, in response to the user's operation of tapping the screen with a knuckle (keeping the knuckle on the screen) and dragging the knuckle to draw a closed shape (such as an irregular shape), the movement trajectory 301 of the knuckle (that is, the user's screenshot trajectory) is displayed on the user interface 30. Next, the electronic device can play animation 1, and display the user interface 33 after the animation 1 has finished playing. You can refer to Figure 3 As shown in (4). The user interface 33 is the cross-application transmission interface. For example, animation 1 may include three parts. The first part may refer to the animation of the joint's motion trajectory 301 changing into a partial screenshot frame 302, which can be referred to... Figure 3 As shown in (1) and (2). Optionally, the shape of the partial screenshot frame 302 can be rectangular. The second part can be an animation of determining the partial screenshot 322 based on the partial screenshot frame 302 and scanning the partial screenshot 322, which can be referred to Figure 3 As shown in (2) and (3). Figure 3 As shown in (3), partial screenshot 322 can display the target location of sub-interface 321, and the content displayed at the target location of sub-interface 321 is the same as or consistent with the content contained in partial screenshot 322. It is understandable that the interface elements (except for the status bar) contained in sub-interface 321 are consistent with user interface 30 and user interface 31. (Continue to refer to...) Figure 3 In part (3), the scanning ray 3221 can move from top to bottom on the partial screenshot 322, simulating the animation effect of scanning the partial screenshot 322. The third part can be the animation of the sub-interface 321 pushing inward and changing into the sub-interface 331, simulating the visual effect of "opening a door" on the screen. It is understood that animation 1 can also include any combination of the various motion effects involved in these three parts, such as the motion effect of the motion trajectory changing into the partial screenshot frame and the motion effect of "opening a door". This embodiment does not limit the specific content of animation 1.

[0142] Continue to refer to Figure 3 The user interface 33 shown in (4) may include, but is not limited to, a sub-interface 331, a partial screenshot 332, text 333 selected by the first cursor 334 and the last cursor 335, a text recognition option 336, a screenshot editing control 337, and an icon area 338. Among them:

[0143] The elements displayed in sub-interface 331 (such as text, images, etc.) are the same as those displayed in sub-interface 321 in user interface 32. Sub-interface 331 can be considered as the interface obtained by the electronic device after performing a three-dimensional transformation of all elements in sub-interface 321.

[0144] A partial screenshot 332 is displayed on sub-interface 331. It should be noted that partial screenshot 332 is not subject to 3D transformation. Optionally, partial screenshot 332 is displayed at the target location on sub-interface 331, and the content displayed at the target location on sub-interface 331 is the same as or matches the content of partial screenshot 332. This allows the user to clearly understand which area of ​​the user interface the partial screenshot was cropped from. It should be noted here that partial screenshot 332 is the final partial screenshot determined by the electronic device in this scenario. Figure 3 The partial screenshot 322 shown in (3) is only schematically shown in Animation 1.

[0145] Text 333 refers to the text content contained in partial screenshot 332. It can be all the text content contained in partial screenshot 332, or only a portion of the text content contained in partial screenshot 332. In this embodiment, the text content displayed on partial screenshot 332 can be determined by the electronic device performing layout analysis on the user interface 30 based on a layout analysis algorithm, combined with the user's selection trajectory and the layout analysis results. In this embodiment, the user can also reselect the text content to be transmitted across applications by adjusting the positions of the first cursor 334 and / or the last cursor 335.

[0146] The state of text recognition option 336 indicates the type of information to be transmitted in the cross-application transmission interface. This responds to the user changing the state of the text recognition option from selected to active (see reference...). Figure 3 The text recognition option 336 shown in (4) is switched to an unselected state (refer to...). Figure 8g (1) As shown in the state of the text recognition option 3163, the electronic device uses a partial screenshot as the information to be transmitted, indicating that the information to be transmitted in the cross-application transmission interface is an image. In response to the user switching the state of the text recognition option from unselected to selected, the electronic device uses the text information in the partial screenshot as the information to be transmitted, indicating that the information to be transmitted in the cross-application transmission interface is text.

[0147] It is understandable that the same control may be identified using different reference numerals in different examples, but the functional description of the control remains consistent. Taking text recognition options as an example, in the examples below, file recognition options are identified using different reference numerals, such as text recognition option 336, text recognition option 3127, etc. The functions of each file recognition option identified by different reference numerals are the same. Detailed application examples of text recognition option 336 can be found below and will not be repeated here.

[0148] The screenshot editing control 337 is used to open a screenshot editing interface, and the screenshot to be edited in the opened screenshot editing interface is a partial screenshot 332. For instructions on using the screenshot editing control 337, please refer to... Figure 1a Examples related to the screenshot editing control 114 shown in (3) will not be repeated here. For example, in response to a user clicking the screenshot editing control 337, the electronic device displays, for example... Figure 1a The screenshot editing interface shown in (4) is similar to... Figure 1a The difference between the interfaces shown in (4) is that the screenshots to be edited are different.

[0149] Icon area 338 can be used to display application icons or application service icons recommended by one or more electronic devices based on the information to be transmitted. Examples of icons and text in icon area 338 can be found above and will not be repeated here.

[0150] In this embodiment, the icon area 338 can display application icons or application service icons recommended based on text 333. This differs from... Figure 1b , Figure 2b For example, when the electronic device displays user interface 33, if a user needs to transfer text 333 across applications to any application or application service in icon area 338, they can click the icon of that application or application service. In response to this user click, the electronic device transfers text 333 to the corresponding application or application service and displays the corresponding application interface or feedback information indicating successful information transfer. For example, in response to a user clicking the icon of the "Notes" application in icon area 338, the electronic device transfers text 333 to the "Notes" application and displays the "Notes" application's interface, such as displaying the "Notes" application's interface as a floating window, and the "Notes" application's interface includes the text information contained in text 333. Understandably, the effect of cross-application text transfer can be referenced... Figure 1b The examples shown will not be repeated here.

[0151] For example, the user interface 33 also includes a prompt message 339 "Click the icon to access the service," which prompts the user on how to perform the operation. The prompt message 339 may be a toast message, and it may disappear from the interface 33 after its display duration reaches a threshold (e.g., 2 seconds). Optionally, the electronic device may only display the prompt message 339 when the user first uses the partial screenshot function provided in this embodiment.

[0152] Figure 4 An exemplary application scenario diagram is shown. In this scenario, a user selects a region of an image on the interface to take a screenshot, triggering the electronic device to display a cross-application transfer interface.

[0153] like Figure 4 As shown in (1), the electronic device displays a user interface 40. The user interface 40 can be the program interface of the "browser" application in the electronic device, or it can be the program interface of other applications in the electronic device. This embodiment does not limit this. In response to the user's operation of selecting a screenshot, the user's screenshot trajectory 401 is displayed on the user interface 40. For example, taking the user's selection of a screenshot using a knuckle as an example, in response to the user's operation of tapping the screen with a knuckle (keeping the knuckle on the screen) and dragging the knuckle to draw a closed shape (such as an irregular shape), the movement trajectory 401 of the knuckle (that is, the user's screenshot trajectory) is displayed on the user interface 40. Next, the electronic device can play animation 2, and after the animation 2 is completed, it displays the user interface 43, which can be referred to Figure 4 As shown in (4). The user interface 43 is the cross-application transmission interface. For animation 2, please refer to... Figure 4 As shown in (1), (2), and (3), the specific explanations can be found in the preceding text and will not be repeated here.

[0154] Continue to refer to Figure 4 The user interface 43 shown in (4) may include, but is not limited to, sub-interfaces 431, partial screenshots 432, screenshot editing controls 433, and icon areas 434. Wherein:

[0155] The elements displayed in sub-interface 431 (such as text, images, etc.) are the same as those displayed in sub-interface 421 in user interface 42. Sub-interface 431 can be considered as the interface obtained by the electronic device after performing a three-dimensional transformation of all elements in sub-interface 421.

[0156] A partial screenshot 432 is displayed on sub-interface 431. It should be noted that partial screenshot 432 is not subject to 3D transformation. Optionally, partial screenshot 432 is displayed at the target location on sub-interface 431, and the content displayed at the target location on sub-interface 431 is the same as or matches the content contained in partial screenshot 432. It should be noted here that partial screenshot 432 is the final partial screenshot determined by the electronic device in this scene. Figure 4 The partial screenshot 422 shown in (3) is only schematically illustrated in Animation 2. The partial screenshot 432 can be used to determine the service recommendation list. The partial screenshot 432 can be transferred to any application in the service recommendation list. For details, please refer to the following text, which will not be repeated here.

[0157] Optionally, in this scenario, since the partial screenshot 432 does not contain any text information, that is, the electronic device does not recognize any text information in the partial screenshot 432, then in the case of... Figure 4 The user interface 43 (i.e., the cross-application transfer interface) shown in (4) does not include a text recognition option. When the cross-application transfer interface does not include a text recognition option, it indicates that the information to be transferred in that cross-application transfer interface is an image.

[0158] The screenshot editing control 433 is used to open a screenshot editing interface, and the screenshot to be edited in the opened screenshot editing interface is a partial screenshot 432. For instructions on using the screenshot editing control 433, please refer to... Figure 2a Examples of the screenshot editing control 214 shown in (3) will not be repeated here.

[0159] Icon area 434 can be used to display application icons or application service icons recommended by one or more electronic devices based on the information to be transmitted. Examples of icons and text in icon area 434 can be found above and will not be repeated here.

[0160] In this embodiment, the icon area 434 can display application icons or application service icons recommended based on the partial screenshot 432. This differs from... Figure 1b , Figure 2b For example, when the electronic device displays user interface 33, if a user needs to transfer a partial screenshot 432 across applications to any application or application service in icon area 434, they can click the icon of that application or application service. In response to this user click, the electronic device transfers the partial screenshot 432 to the corresponding application or application service and displays the corresponding application interface or feedback information indicating successful transfer. For example, in response to the user clicking the icon of the "Gallery" application in icon area 434, the electronic device transfers the partial screenshot 432 to the "Gallery" application and displays feedback information. Understandably, the effect of cross-application image transfer can be referenced... Figure 2b The examples shown will not be repeated here.

[0161] Thus, in the partial screenshot method provided in this application embodiment, the user only needs to perform a selection screenshot operation to enable the electronic device to display an interface for cross-application information transmission. The user can then trigger the cross-application transmission of text or images by clicking on the application icon or application service icon in this interface. The user operation is relatively simple. Even if the user's selection area is not standardized, the electronic device will automatically adjust the selection area and recommend applications or application services to the user based on the text or images contained in the adjusted screenshot area, greatly simplifying the user's operation steps and improving the user experience.

[0162] Figure 5 A schematic diagram of the electronic device is shown. It should be understood that... Figure 5 The electronic device shown is merely an example of an electronic device, and electronic devices may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. Figure 5 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0163] The electronic device may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0164] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0165] The controller can serve as the nerve center and command center of an electronic device. Based on the instruction opcode and timing signals, the controller generates operation control signals to control the fetching and execution of instructions.

[0166] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0167] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0168] The wireless communication function of electronic devices can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Mobile communication module 150 can provide wireless communication solutions for electronic devices, including 2G / 3G / 4G / 5G. Wireless communication module 160 can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0169] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology.

[0170] Electronic devices implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0171] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0172] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0173] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc., which will not be listed here, and this application does not limit them.

[0174] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 194. When a touch operation is applied to the display screen 194, the electronic device detects the intensity of the touch operation based on the pressure sensor. The electronic device may also calculate the location of the touch based on the detection signal from the pressure sensor.

[0175] A touch sensor, also known as a "touch panel," can be located on the display screen 194. The touch sensor and display screen 194 together form a touchscreen, also called a "touch screen." The touch sensor detects touch operations applied to or near it. It then transmits the detected touch operation to the application processor to determine the type of touch event.

[0176] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application uses the layered architecture Android system as an example to illustrate the software structure of an electronic device.

[0177] Figure 6 This is a software structure block diagram of an electronic device according to an embodiment of this application.

[0178] The layered architecture of electronic devices divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android Runtime and system libraries, and the kernel layer.

[0179] The application layer can include a series of application packages.

[0180] like Figure 6 As shown, the application package may include business modules, intent recognition modules, layout analysis modules, browser applications, note-taking applications, etc. The application package may also include applications such as gallery, camera, call, calendar, map, navigation, music, video, SMS, Bluetooth, etc.

[0181] The business module can be used to implement the partial screenshot method provided in the embodiments of this application.

[0182] The intent recognition module can be used to identify user intent based on text and / or images, and determine the applications or application services to recommend to the user.

[0183] The layout analysis module can be used to analyze the layout of images and determine the text information, image information, etc. contained in the images.

[0184] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0185] like Figure 6 As shown, the application framework layer may include a window manager, content provider, view system, resource manager, etc.

[0186] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0187] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0188] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0189] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0190] The Android Runtime consists of core libraries and a virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0191] The core library consists of two parts: one part contains the functionalities that the Java language needs to call, and the other part is the Android core library. The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0192] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0193] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. The Media Library supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0194] The kernel layer is the layer between hardware and software. It includes at least display drivers, audio drivers, Wi-Fi drivers, and sensor drivers. The hardware includes at least a processor, display screen, Wi-Fi module, and sensors.

[0195] Understandable Figure 6 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer layers than illustrated, and each layer may include more or fewer components, or combine some components, or split some components, or arrange the components differently; this application does not impose any limitations.

[0196] It is understood that, in order to implement the partial screenshot method in the embodiments of this application, the electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0197] like Figure 7a The diagram shows the interaction between each module. (Refer to...) Figure 7a The process of the partial screenshot method provided in this application embodiment specifically includes:

[0198] S501, in response to the user's selection of a screenshot, the business module captures a full-screen image, determines the user's screenshot trajectory on the full-screen image, and sends the full-screen image to the layout analysis module.

[0199] For example, a user's selection and screenshot operation can be performed by using their knuckles to select a portion of the screen for a screenshot, as shown in the example. Figure 3 Zhong (1) and Figure 4 In (1), the user taps the screen with their knuckles (keeping their knuckles on the screen) and then drags their knuckles to draw a closed shape.

[0200] In response to a user's selection and screenshot operation on the user interface, the business module captures a full-screen image corresponding to that user interface. It's important to understand that the elements included in the full-screen image are the same elements displayed on that user interface. Figure 3 In the example shown in (1), the full-screen image is the image obtained by the business module capturing the user interface 30 (excluding the screenshot trajectory 301). Similarly, in Figure 4 In the example shown in (1), the full-screen image is the image obtained by the business module from the user interface 40 (excluding the screenshot trajectory 401).

[0201] The screenshot trajectory of a user on a full-screen image is the movement trajectory of their knuckles on the display screen, which can be referenced. Figure 3 The screenshot trajectory 301 shown in (1) or Figure 4 The screenshot trajectory 401 is shown in (1).

[0202] S502, the layout analysis module performs layout analysis on the full-screen image, determines the image area, text area, and text information in the full-screen image, and sends the image area, text area, and text information in the full-screen image to the business module.

[0203] In this embodiment, the layout analysis module can perform layout analysis on the full-screen image using a layout analysis algorithm. Wherein:

[0204] The input to the layout analysis algorithm is a full-screen image. For example, the format of the full-screen image can be a bitmap.

[0205] The output of the layout analysis algorithm is a list of image regions, a list of text regions, and text information (i.e., string values) in the full-screen image.

[0206] Each image region in the image region list and each text region in the text region list can be identified by the coordinates of points in the full-screen image. For example, each region (image region or text region) can be identified by the coordinates of the top left vertex, bottom left vertex, bottom right vertex, and top right vertex.

[0207] Optionally, an image region may include one image from a full-screen image. For example, if a full-screen image includes Image 1 and Image 2, layout analysis of the full-screen image can yield two image regions: Image 1 region and Image 2 region.

[0208] Optionally, an image region can be a local area of ​​an image within a full-screen image. If the image contains multiple objects, the local area corresponding to each object can be considered as an image region. For example, if the full-screen image includes Image 1, and Image 1 contains multiple objects such as "hat," "flower," and "bicycle," then by performing layout analysis on the full-screen image, three image regions can be obtained: the image region corresponding to the "hat," the image region corresponding to the "flower," and the image region corresponding to the "bicycle."

[0209] Optionally, a text region may include a text paragraph in a full-screen image, or a text region may include a text line in a full-screen image, or a text region may include a character or word in a full-screen image. This embodiment does not limit the division of text regions in a full-screen image.

[0210] Figure 7b This example demonstrates the layout analysis results for a full-screen image. In this example, each image region in the full-screen image is represented as an image, and each text region as a text line (or a line of text). Figure 7b The full-screen image shown includes two images: one displaying the text "The Most Beautiful Tulips," "Scenery Image Appreciation," and "Tulips are perennial herbaceous plants of the genus Tulipa in the family Liliaceae, with bulbs." The other image below displays the text "Colorful Tulips." This text is displayed across six lines of text. (The last sentence appears to be incomplete and possibly refers to a different image or text.) Figure 7b The full-screen image shown can be analyzed to identify two image regions and six text regions, namely image region 513, image region 517, text region 511, text region 512, text region 514, text region 515, text region 516, and text region 518.

[0211] S503, the business module compares the screenshot trajectory with the image area and text area in the full-screen image to determine the target screenshot area 1 and the text information contained in the target screenshot area 1.

[0212] The business module compares the screenshot trajectory with the image and text regions in the full-screen image, analyzes the relationship between the area contained in the screenshot trajectory (referred to as the screenshot trajectory area) and each image and text region, and determines the target screenshot area 1. Understandably, relative to the screenshot trajectory area, the target screenshot area 1 is the screenshot area adjusted by the business module.

[0213] In this embodiment, the target screenshot area 1 can also be identified by the coordinates of points in the full-screen image, such as by the coordinates of the top left vertex, bottom left vertex, bottom right vertex, and top right vertex of the area.

[0214] For example, if the screenshot trajectory area is completely contained within a certain area (image area or text area) of the full-screen image, the business module can use that area as the target screenshot area 1.

[0215] For example, if the screenshot trajectory area is completely contained within a certain area (image area or text area) of the full-screen image, and the area ratio of the screenshot trajectory area in that area reaches a preset threshold (e.g., 70%), then the business module can use that area as the target screenshot area 1.

[0216] For example, if the screenshot trajectory area is completely contained within a certain area (image area or text area) of the full-screen image, but the area ratio of the screenshot trajectory area in that area does not reach a preset threshold (e.g., 70%), then the business module can use the bounding rectangle of the screenshot trajectory area as the target screenshot area 1.

[0217] For example, if the screenshot trajectory area intersects with a certain area (image area or text area) of the full-screen image, and the area ratio of the intersection in that area reaches a preset threshold (e.g., 70%), then the business module can use that area as the target screenshot area 1.

[0218] For example, if the screenshot trajectory area intersects with a certain area (image area or text area) of the full-screen image, but the area ratio of the intersection in that area does not reach a preset threshold (e.g., 70%), the business module can use the outer rectangular area of ​​the screenshot trajectory area as the target screenshot area 1.

[0219] For example, if the screenshot trajectory area intersects with multiple areas (image areas and / or text areas) of the full-screen image, the business module determines whether each area is a valid area. Specifically, if the area of ​​the intersection between the screenshot trajectory area and a certain area (image area or text area) of the full-screen image reaches a preset threshold (e.g., 70%), the business module determines that area as a valid area. Then, the business module can determine the target screenshot area 1 based on all valid areas; for example, it can use an area containing all valid areas as the target screenshot area 1. If none of the areas intersecting with the screenshot trajectory area are valid areas, the business module can use the bounding rectangle of the screenshot trajectory area as the target screenshot area 1.

[0220] For example, if the screenshot trajectory area intersects with multiple areas (image area and / or text area) of the full-screen image, the business module can use the bounding rectangle area of ​​the screenshot trajectory area as the target screenshot area 1.

[0221] Since the business module obtains the text information in the full-screen image fed back by the layout analysis module, after it determines the target screenshot area 1, it can analyze and obtain the text information corresponding to the target screenshot area 1, that is, the text information contained in the target screenshot area 1.

[0222] S504, the business module crops the full-screen image based on the target screenshot area 1 to obtain partial screenshot 1, and sends partial screenshot 1 and / or the text information contained in partial screenshot 1 to the intent recognition module.

[0223] In this embodiment, the business module redetermines the target screenshot area 1 based on the screenshot trajectory, and then crops the full-screen image based on the target screenshot area 1 to obtain a partial screenshot, i.e., partial screenshot 1. The shape of partial screenshot 1 is independent of the shape of the screenshot trajectory, but is the same as the shape and size of the target screenshot area 1. Optionally, the target screenshot area 1 is rectangular, and the partial screenshot 1 obtained by cropping based on the target screenshot area 1 is also rectangular. In this case, if the shape of the screenshot trajectory is not rectangular (such as any shape, a heart, etc.), the shape of partial screenshot 1 will be completely different from the shape of the screenshot trajectory.

[0224] After cropping to obtain partial screenshot 1, the business module can save the URI (UniformResource Identifier) ​​of partial screenshot 1 so that it can be displayed in the subsequent cross-application transmission interface.

[0225] In one scenario, partial screenshot 1 contains only text information and no image information. In this case, the business module can send the text information contained in partial screenshot 1 to the intent recognition module for intent recognition processing.

[0226] In one scenario, partial screenshot 1 contains only image information and no text information. In this case, the business module can send partial screenshot 1 to the intent recognition module for intent recognition processing.

[0227] In one scenario, partial screenshot 1 contains both image and text information. In this case, the business module can send partial screenshot 1 to the intent recognition module, or it can send partial screenshot 1 along with the text information contained in partial screenshot 1 to the intent recognition module for intent recognition processing.

[0228] S505, the intent recognition module performs intent recognition based on partial screenshot 1 and / or the text information contained in partial screenshot 1, determines service recommendation list 1, and sends service recommendation list 1 to the business module.

[0229] Intent refers to the purpose that the user wants to achieve. Intents can be divided into various categories. For example, the categories of intents may include, but are not limited to, unlocking passwords, navigation, map search, hailing a ride, shopping, ordering takeout, etc. This embodiment does not limit the category of intent.

[0230] For example, each category of intent may include one or more of the following information: intent identifier, intent level, intent tag, and intent slot information (slotInfoList). The intent identifier indicates the intent category and may include one or more of the following: intent identification number (ID), intent code. The intent level represents the confidence level of the intent. The intent slot information may include one or more of the following: slot name, data type filled in the slot, Chinese name of the data filled in the slot, and indication of whether the slot is necessary. A slot can be considered a parameter, and its data type may include string, number, etc.

[0231] In this embodiment, the intent recognition module can perform intent recognition on partial screenshot 1 and / or the text information contained in partial screenshot 1 based on its loaded intent recognition model. Optionally, the intent recognition module can preload the intent recognition model when the user performs a selection screenshot operation. For example, in response to the user's selection screenshot operation, the business module sends an instruction message to the intent recognition module. After receiving the instruction message, the intent recognition module performs the operation of preloading the intent recognition model.

[0232] The intent recognition result obtained by the intent recognition module may include one intent category or multiple intent categories; this embodiment does not limit this.

[0233] In this embodiment, the intent recognition module can perform single-modal intent recognition, such as intent recognition based on images or intent recognition based on text.

[0234] For example, the intent recognition module can determine the corresponding intent category based on the modality of the recognized content.

[0235] For example, when the intent recognition module performs intent recognition based on text information, the modality of the content it recognizes is text, and thus the intent category can be determined to include, but is not limited to, copy intent, edit intent, search intent, translation intent, etc.

[0236] For example, when the intent recognition module performs intent recognition based on a partial screenshot, if the modality of the recognized content is an image, then the intent category can be determined to include, but is not limited to, storage intent, sharing intent to social applications, etc.

[0237] For example, the intent recognition module can also determine the corresponding intent category based on the semantic understanding of the recognized content.

[0238] For example, when the intent recognition module performs intent recognition based on text information, it can perform NLP (Natural Language Processing) on ​​the text information and perform intent recognition based on the NLP results. If the NLP results indicate that the text information is address-type text, the intent recognition module can determine the intent category, including but not limited to navigation intent, ride-hailing intent, etc.

[0239] For example, when the intent recognition module performs intent recognition based on partial screenshot 1, it can perform label recognition on partial screenshot 1 and perform intent recognition based on the label recognition result of partial screenshot 1 and partial screenshot 1. If the label recognition result indicates that the image belongs to the landscape category, the intent recognition module can determine the intent category, including but not limited to storage intent, editing intent, sharing to social applications intent, etc.

[0240] In this embodiment, the intent recognition module can also perform multimodal intent recognition, such as intent recognition based on images and text. For example, it can directly perform intent recognition on a partial screenshot using a multimodal model.

[0241] In another example, the intent recognition module receives a partial screenshot 1 and the text information contained in the partial screenshot 1 sent by the business module. It performs label recognition on the partial screenshot 1, performs NLP on the text information, and performs intent recognition based on the partial screenshot 1, the label recognition results of the partial screenshot 1, and the NLP results of the text information. For example, if the NLP results indicate that the text information is address-related text, and the label recognition results indicate that the image belongs to the map category, the intent recognition module can determine the intent category, including but not limited to navigation intent, ride-hailing intent, storage intent, and sharing to social media applications intent.

[0242] It should be noted that in the intent recognition results obtained by the intent recognition module, the order of intents by category is related to the level of the intent. For example, the higher the level of the intent, the higher it ranks in the intent recognition results.

[0243] After obtaining the intent recognition result, the intent recognition module can determine at least one recommended service to be displayed to the user, and generate a service recommendation list 1 based on these recommended services. The recommended service can be an application (such as...) Applications can be either services within an application (such as...). (The "scan" service in the application) In this embodiment, the type of recommended service is not limited.

[0244] For example, after obtaining the intent recognition result, the intent recognition module can query the corresponding recommendation service based on the mapping relationship between each intent included in the intent recognition result and the recommendation service, and generate a service recommendation list 1 based on these recommendation services.

[0245] Understandably, different intent recognition results obtained by the intent recognition module may lead to differences in the recommended services determined based on those results. In the service recommendation list, recommended services can be sorted according to default rules or user habits; this embodiment does not limit the sorting of recommended services in the service recommendation list.

[0246] S506, the business module generates a cross-application transmission interface 1, which includes a thumbnail of a full-screen image, a partial screenshot 1, and a service recommendation list 1.

[0247] After receiving the service recommendation list 1 from the intent recognition module, the business module can generate a cross-application transmission interface based on the thumbnail of the full-screen image, the partial screenshot 1, and the service recommendation list 1. (See reference...) Figure 3User interface 33 shown in (4), or as shown in Figure 4 User interface 43 shown in (3)

[0248] by Figure 3 Taking the user interface 33 shown in (4) as an example, the sub-interface 331 is a thumbnail of the full-screen image, the partial screenshot 332 is a partial screenshot 1, and the icons of the recommended services included in the service recommendation list 1 are the icons included in the icon area 338.

[0249] by Figure 4 Taking the user interface 43 shown in (4) as an example, sub-interface 431 is a thumbnail of the full-screen image, partial screenshot 432 is partial screenshot 1, and the icons of the recommended services included in the service recommendation list 1 are the icons included in the icon area 434. It can be understood that if there are too many recommended services included in the service recommendation list 1, the icons of these recommended services can be displayed in the icon area (such as icon area 338 or icon area 434) in a split-screen manner.

[0250] In this embodiment, after the user completes the knuckle selection screenshot operation, before the electronic device displays the cross-application transfer interface 1, the electronic device can play an animation to enhance the user's visual experience. For example, the animation may include a motion effect showing the selection trajectory changing towards a partial screenshot frame, and a motion effect showing the partial screenshot being cropped based on the partial screenshot frame and scanned, which can refer to Animation 1 or Animation 2 described above.

[0251] In the cross-application data transfer interface, the information to be transferred can be text (i.e., the text information included in partial screenshot 1). See [reference] Figure 3 In the example shown in (4), the information to be transmitted is text information 333 contained in partial screenshot 332. In response to the user clicking an icon in icon area 338, the electronic device sends text information 333 to the corresponding application or application service.

[0252] In cross-application data transfer interfaces, the information to be transferred can also be an image (i.e., partial screenshot 1). See reference... Figure 4 In the example shown in (4), the information to be transmitted is a partial screenshot 432. In response to the user clicking an icon in the icon area 434, the electronic device sends the partial screenshot 432 to the corresponding application or application service.

[0253] In one possible scenario, the information to be transmitted in the cross-application transfer interface can be an image (i.e., partial screenshot 1) and text (i.e., the text information included in partial screenshot 1). In response to the user's action of selecting an image (containing text information) in the interface, the electronic device displays the cross-application transfer interface, which includes partial screenshot 1 and an icon area. In response to the user's action of clicking an icon in the icon area, the electronic device sends partial screenshot 1 and the text information included in partial screenshot 1 to the corresponding application or application service.

[0254] In this way, users only need to perform a selection screenshot operation to trigger the cross-application transfer interface on the electronic device. Users can then click on the recommended service icon in this interface to trigger the cross-application transfer of text and / or images. Furthermore, the partial screenshot displayed in the cross-application transfer interface is determined by the electronic device based on the user's selection trajectory within the user interface and the content of the user interface layout, not solely based on the user's selection trajectory. Therefore, even if the user's selection trajectory is not standardized, the electronic device will automatically adjust the selection range to ensure that the content (text or image) in the resulting partial screenshot is relatively complete.

[0255] When the electronic device displays the cross-application transmission interface 1, if the information to be transmitted as determined by the service module does not meet the user's expectations, the user can manually adjust the information to be transmitted within the cross-application transmission interface 1. This user's operation of adjusting the information to be transmitted will be referred to as the adjustment operation below.

[0256] With cross-application transmission interface as Figure 3 Taking the user interface 33 shown in (4) as an example, the adjustment operation can be the user moving the position of the first cursor 334 and / or moving the position of the last cursor 335. In response to the user's operation of adjusting the position of the first cursor 334 and / or the last cursor 335, the service module adjusts the information to be transmitted to more or less text information than the text information 333.

[0257] With cross-application transmission interface as Figure 3 Taking the user interface 33 shown in (4) as an example, the adjustment operation can also be the operation of clicking the text recognition option 336. In response to the user clicking the text recognition option 336, the business module adjusts the information to be transmitted into a partial screenshot 332.

[0258] If the user manually adjusts the information to be transmitted, continue to refer to... Figure 7a The process of the partial screenshot method provided in this application embodiment further includes:

[0259] S507, in response to the user's adjustment operation, the business module determines the target screenshot area 2.

[0260] When a user adjusts the information to be transmitted, the business module needs to redetermine the target screenshot area.

[0261] In this embodiment, the target screenshot area 2 is the screenshot area redefined by the business module based on the user's adjustment operation. The target screenshot area 2 may or may not be the same as the target screenshot area 1 previously determined by the business module.

[0262] In some scenarios, such as when the user clicks the text recognition option 336, the target screenshot area 2 is consistent with the previously determined target screenshot area (such as target screenshot area 1). In other scenarios, such as when the user adjusts the position of the first cursor 334 and / or the position of the last cursor 335 to add text information to be transmitted, the target screenshot area 2 is inconsistent with the previously determined target screenshot area (such as target screenshot area 1). Detailed explanations of these scenarios can be found below and will not be repeated here.

[0263] Similarly, the target screenshot area 2 can also be identified by the coordinates of points in the full-screen image, such as by the coordinates of the top left vertex, bottom left vertex, bottom right vertex, and top right vertex of the area.

[0264] S508, the business module crops the full-screen image based on the target screenshot area 2 to obtain partial screenshot 2, and sends partial screenshot 2 and / or the text information contained in partial screenshot 2 to the intent recognition module.

[0265] If the business module redefines the target screenshot area 2, the business module will crop the full-screen image based on the redefining target screenshot area 2 to obtain a partial screenshot, i.e., partial screenshot 2.

[0266] After cropping to obtain partial screenshot 2, the business module can save the URI of partial screenshot 2 so that it can be displayed in the subsequent cross-application transmission interface.

[0267] Similar to partial screenshot 1, partial screenshot 2 may contain only text information and no image information. In this case, the business module can send the text information contained in partial screenshot 2 to the intent recognition module for intent recognition processing.

[0268] The partial screenshot 2 can also contain only image information and no text information. In this case, the business module can send the partial screenshot 1 to the intent recognition module for intent recognition processing.

[0269] The partial screenshot 2 can contain both image information and text information. In this case, the business module can send partial screenshot 2 to the intent recognition module, or it can send partial screenshot 2 and the text information contained in partial screenshot 2 to the intent recognition module for intent recognition processing.

[0270] S509, the intent recognition module performs intent recognition based on partial screenshot 2 and / or the text information contained in partial screenshot 2, determines service recommendation list 2, and sends service recommendation list 2 to the business module.

[0271] The intent recognition module re-recognizes the intent based on the received partial screenshot 2 and / or the text information contained in partial screenshot 2, and redetermines at least one recommended service based on the intent recognition result, and generates a service recommendation list 2 based on these recommended services.

[0272] Understandably, because the input to the intent recognition module has changed—from "partial screenshot 1 and / or the text information contained in partial screenshot 1" to "partial screenshot 2 and / or the text information contained in partial screenshot 2"—the intent recognition results of the intent recognition module will differ, and consequently, service recommendation list 2 and service recommendation list 1 will also be different. Even if service recommendation list 2 and service recommendation list 1 contain the same recommended services, the order of the recommended services in the list will be different.

[0273] S510, the business module generates a cross-application transmission interface 2, which includes a thumbnail of a full-screen image, a partial screenshot 2, and a service recommendation list 2.

[0274] After receiving the service recommendation list 2 from the intent recognition module, the business module can regenerate the cross-application transmission interface 2 based on the thumbnail of the full-screen image, the partial screenshot 2, and the service recommendation list 2. The cross-application transmission interface 2 can then be displayed on the electronic device screen for user use. For example, in response to a user's action on an icon in the icon area of ​​the cross-application transmission interface 2, the electronic device sends the partial screenshot 2 and the text information included in the partial screenshot 2 to the corresponding application or application service.

[0275] In this context, the thumbnails of the full-screen images in cross-application transfer interface 1 and cross-application transfer interface 2 are the same. However, partial screenshot 1 in cross-application transfer interface 1 and partial screenshot 2 in cross-application transfer interface 2 may be the same or different. The information to be transferred in cross-application transfer interface 1 and cross-application transfer interface 2 are different. Relevant examples can be found below and will not be repeated here.

[0276] In this embodiment, after the user completes the adjustment operation and before the electronic device displays the cross-application transmission interface 2, the electronic device can play an animation to enhance the user's visual experience. For example, the animation may include an animation effect of scanning a redefined screenshot area, and an animation effect of a thumbnail of a full-screen image creating a "door opening" effect on the screen.

[0277] Understandably, when the electronic device displays cross-application transmission interface 2, if the information to be transmitted as determined by the business module still does not meet the user's expectations, the user can continue to manually adjust the information to be transmitted in cross-application transmission interface 2, so that the electronic device displays the cross-application transmission interface 3 regenerated by the business module. Similarly, when the electronic device displays cross-application transmission interface 3, if the information to be transmitted as determined by the business module still does not meet the user's expectations, the user can continue to manually adjust the information to be transmitted in cross-application transmission interface 3, so that the electronic device displays the cross-application transmission interface 3 regenerated by the business module, and so on. The processing flow from the electronic device displaying cross-application transmission interface n (n≥1) to the electronic device displaying cross-application transmission interface n+1 can be referred to S507-S510, and will not be elaborated here.

[0278] Understandably, since the user only performed the selection and screenshot operation once (refer to S501), the full-screen image captured by the business module did not change. Therefore, the thumbnails of the full-screen images in these application transmission interfaces that are re-displayed on the electronic device are all the same.

[0279] In this way, users only need to select and take a screenshot to make the electronic device display the cross-application transfer interface, facilitating the transfer of information across applications. If the information to be transferred in the current cross-application transfer interface does not meet the user's expectations, the user can make adjustments within the current interface to allow the electronic device to regenerate and display the cross-application transfer interface for use.

[0280] The following section provides a detailed explanation of the partial screenshot method provided in this embodiment, using several specific application scenarios as examples.

[0281] Scene 1

[0282] In this scenario, the user selects a portion of the screen for a screenshot on the user interface, and the screenshot area contains only text information. For example, when the layout analysis module performs layout analysis on a full-screen image, the text area is divided by text lines, as can be seen in the example. Figure 7b The example shown.

[0283] (i) The business module determines at least one text region that intersects with the screenshot trajectory area as a valid region based on the layout analysis results of the screenshot trajectory and the full-screen image.

[0284] Figure 3 , Figure 8a This example illustrates an application scenario where a user selects and takes a screenshot of a text area displayed on an interface. Specifically, regarding... Figure 3 For further explanation, please refer to the previous text; it will not be repeated here. Figure 8a The user interface shown in (1) is Figure 3 The interface shown in (4) is shown in the middle.

[0285] Reference Figure 3 As shown in Figure (1), the user performs a selection screenshot operation on the user interface 30. In response to the user operation, the screenshot trajectory 301 is displayed on the user interface 30. In this embodiment, the business module determines the screenshot area (i.e., the target screenshot area 1) based on the layout analysis results of the full-screen image and the screenshot trajectory 301.

[0286] In this scenario, the layout analysis module performs layout analysis on the full-screen image, determining that the image contains only one image region and five text line regions: text line region 1 corresponding to the text "The most beautiful tulips," text line region 2 corresponding to the text "Enjoy scenery pictures," text line region 3 corresponding to the text "Tulips are lilies," text line region 4 corresponding to the text "Perennial herbaceous plants of the genus *Curcuma*," and text line region 5 corresponding to the text "Has bulbs." The business module determines that the area contained in the screenshot trajectory 301 intersects with text line regions 3, 4, and 5. The area of ​​the text line region 3, which intersects with the area contained in the screenshot trajectory 301, reaches a preset threshold (e.g., 70%). Therefore, the business module determines text line region 3 as a valid region. Similarly, the business module determines text line regions 4 and 5 as valid regions as well. Subsequently, the business module can use the region containing these three valid regions as the target screenshot region 1. The target screenshot area 1 contains only text information. Subsequently, the business module can crop the full-screen image based on the target screenshot area 1 to obtain a partial screenshot 332 (see reference). Figure 3 As shown in (4), the text information contained in the partial screenshot (i.e., text information 333) is sent to the intent recognition module. The intent recognition module performs intent recognition based on text information 333, obtains intent recognition results, and determines each recommended service corresponding to the intent recognition results, generating a recommended service list 1. The icons of each recommended service included in the recommended service list 1 are the icons included in the icon area 338.

[0287] Continue to refer to Figure 8aAs shown in (1), the electronic device displays an application transfer interface 33. In this interface, the information to be transferred is the text information 333 selected by the first cursor 334 and the last cursor 335. In response to the user clicking the "Notes" application icon 3381 in the icon area 338, the electronic device transfers the text information 333 to the "Notes" application and displays the program interface of the "Notes" application. For example, in response to the user clicking the "Notes" application icon 3381 in the icon area 338, the electronic device displays a user interface 34, as shown in (1). Figure 8a As shown in (2). The user interface 34 includes a sub-interface 341 and a floating window 342 displayed on the sub-interface 341. The program interface of the "Notes" application is displayed in the floating window 342, and text information 3421 is displayed on the program interface of the "Notes" application. Among them, the text information 3421 is the text information 333 to be transmitted in the cross-application transmission interface 33. It can be understood that the display content of the sub-interface 341 (except for the status bar) is consistent with the sub-interface 331 in the cross-application transmission interface 33. The change from sub-interface 331 to sub-interface 341 can create a "closing door" visual effect on the screen. Thus, the cross-application transmission of text information is completed.

[0288] It should be pointed out that, Figure 8a (1) (or) Figure 3 Example (4) illustrates an example of a cross-application transmission interface. In the partial screenshot shown in this example (as shown in partial screenshot 332), the text information to be transmitted is text information 333 selected by the first cursor 334 and the last cursor 335. Text information 333 includes three complete text lines. That is, in this embodiment, in the partial screenshot displayed in the cross-application transmission interface, the first cursor and the last cursor select all the text information in the partial screenshot.

[0289] In another alternative implementation, in the partial screenshot displayed in the cross-application transfer interface, the positions of the first and last cursors can also be determined based on the user's screenshot trajectory. For example, the positions of the first and last cursors are located on the user's screenshot trajectory and intersect with it. For instance, Figure 3 (1), (2), (3) and Figure 8a Figure (3) illustrates an application scenario. (Refer to...) Figure 8a In the cross-application transmission interface shown in (3), the positions of the first and last cursors in partial screenshot 332 are determined based on the user's screenshot trajectory 301. For example, the positions indicated by the first and last cursors are both located on the user's screenshot trajectory 301. At this time, the text selected by the first cursor 334 and the last cursor 335 is text information 3310. As mentioned above, in response to the user clicking the "Notes" application icon in the icon area, the electronic device can display as follows: Figure 8a The user interface shown in (4) displays text information 3422 on the program interface of the "Notes" application. Text information 3422 is the cross-application transfer interface (such as...). Figure 8a The text information 3310 to be transmitted is shown in (3). For details regarding this application scenario, please refer to the previous text, which will not be repeated here.

[0290] Figure 3 , Figure 8b , Figure 8c This example illustrates a scenario where a user reduces the amount of text to be transmitted in a cross-application transfer interface. About Figure 3 The explanation can be found in the previous text, and will not be repeated here. Among them, Figure 8b The user interface shown in (1) is Figure 3 The interface shown in (4) is shown in the middle.

[0291] like Figure 8b As shown in Figure (1), the information to be transmitted displayed in the cross-application transmission interface 33 is text information 333 selected by the first cursor 334 and the last cursor 335. In this scenario, the information to be transmitted does not meet the user's expectations, and the user needs to make adjustments. For example, the user drags the last cursor 335 forward to reduce the text information selected by the first cursor 334 and the last cursor 335. In response to this user operation, the electronic device can play an animation (e.g., create a "closing door" visual effect on the screen) and display the following after the animation is completed: Figure 8b The user interface 34 shown in (2) includes a sub-interface 341 and a partial screenshot 342. The content displayed in the sub-interface 341 (except for the status bar) is the same as that in the cross-application transfer interface 33. The change from sub-interface 331 to sub-interface 341 can create a "closing door" visual effect on the screen.

[0292] Continue to refer to Figure 8b In section (2), the partial screenshot 342 displayed on sub-interface 341 is the same as the partial screenshot 332 displayed in cross-application transfer interface 33. The partial screenshot 342 is displayed at the target location of sub-interface 341, and the content displayed at the target location of sub-interface 341 is the same as or matches the content contained in partial screenshot 342. In response to the user dragging the tail cursor 335 forward, the text information selected by the head cursor 334 and tail cursor 335 changes accordingly. For example, when the number of text lines corresponding to the selected text information changes, the partial screenshot also changes accordingly. For example, the partial screenshot changes from partial screenshot 342 (including three text line areas) to partial screenshot 352 (including two text line areas), which can be referred to... Figure 8bUser interface 35 is shown in (3). User interface 35 includes sub-interface 351 and partial screenshot 352. The content displayed in sub-interface 351 (except for the status bar) is the same as that in sub-interface 341. When the user adjusts the first cursor and / or the last cursor to their expected position, the user can release their hand. For example, as shown in partial screenshot 352 in user interface 35, the display positions of the first cursor 354 and the last cursor 355 are the user's expected positions, and the text information 353 selected by the first cursor 354 and the last cursor 355 is the text information to be transmitted after adjustment by the user.

[0293] In response to the user removing their hand, the electronic device can play animation 3 and display user interface 37 after animation 3 has finished playing, such as... Figure 8c As shown in (1). The user interface 37 is the cross-application transmission interface. For example, the effect of animation 3 can be referred to... Figure 8b (3), (4) and Figure 8c For the relevant explanation in (1), please refer to the second and third parts of Animation 1, which will not be repeated here.

[0294] Continue to refer to Figure 8c In (1), the user interface 37 may include, but is not limited to, a sub-interface 371, a partial screenshot 372, text 373 selected by the first cursor 374 and the last cursor 375, and an icon area 376. In this scenario, the user adjusts the position of the last cursor used to select the text information. Therefore, the business module redetermines the screenshot area (i.e., the target screenshot area 2). The target screenshot area 2 is determined based on the text area selected by the first cursor and the last cursor, and is different from the target screenshot area 1. The business module crops the full-screen image based on the target screenshot area 2, obtains a partial screenshot 372 again, and sends the text information (i.e., text information 373) contained in the partial screenshot to the intent recognition module. The intent recognition module performs intent recognition based on the text information 373, obtains the intent recognition result, and determines each recommended service corresponding to the intent recognition result, generating a recommended service list 2. The icons of each recommended service included in the recommended service list 2 are the icons included in the icon area 376.

[0295] It should be noted that in this scenario, the recommended service list 2 can be the same as or different from the recommended service list 1, depending on the analysis results of the intent recognition module. This embodiment does not limit this.

[0296] Continue to refer to Figure 8cAs shown in (1), the electronic device displays an application transfer interface 37. In this interface, the information to be transferred is the text information 373 selected by the first cursor 374 and the last cursor 375. In response to the user clicking the "Notes" application icon 3761 in the icon area 376, the electronic device transfers the text information 373 to the "Notes" application and displays the program interface of the "Notes" application, which can be referred to as... Figure 8c As shown in (2). At this point, the cross-application transmission of text is complete.

[0297] For details not covered in this scenario, please refer to the relevant examples above; they will not be repeated here. It should be noted that the above scenario uses the example of a user dragging the tail cursor to reduce the selected text information. Scenarios where a user drags the head cursor or both the head and tail cursors to reduce the selected text information are similar, and will not be elaborated upon in this embodiment.

[0298] Figure 3 , Figure 8d , Figure 8e This example illustrates a scenario where a user adds text to be transferred within a cross-application transfer interface. About Figure 3 The explanation can be found in the previous text, and will not be repeated here. Among them, Figure 8d The user interface shown in (1) is Figure 3 The interface shown in (4) is shown in the middle.

[0299] like Figure 8d As shown in Figure (1), the information to be transmitted displayed in the cross-application transmission interface 33 is text information 333 selected by the first cursor 334 and the last cursor (covered by the user's hand in the figure). For example, the user drags the first cursor 334 forward to add more text information selected by the first cursor 334 and the last cursor. In response to this user action, the electronic device can play an animation (e.g., creating a "closing door" visual effect on the screen) and display the following after the animation finishes playing: Figure 8d The user interface 39 shown in (2) is shown in the middle.

[0300] Continue to refer to Figure 8d In step (2), in response to the user dragging the first cursor forward, the text information selected by the first and last cursors changes accordingly. For example, in response to the user continuing to drag the first cursor forward, the electronic device displays user interface 310, such as... Figure 8d As shown in (3), the text information selected by the first and last cursors is the text information to be transmitted after being adjusted by the user.

[0301] In response to the user removing their hand, the electronic device can play animation 4 and display user interface 312 after animation 4 has finished playing, such as... Figure 8eAs shown in (1). The user interface 312 is the cross-application transmission interface. For example, the effect of animation 4 can be referred to... Figure 8d (3), (4) and Figure 8e (1) For related explanations, please refer to Parts 2 and 3 of Animation 1, which will not be repeated here. Continue to refer to Figure 8e In (1), the user interface 312 may include, but is not limited to, a sub-interface 3121, a partial screenshot 3122, text 3123 selected by the first cursor 3124 and the last cursor 3125, and an icon area 3126. The partial screenshot 3122 is the final partial screenshot determined by the electronic device in this scenario. In this scenario, the user adjusted the position of the first cursor used to select the text information. Therefore, the business module redetermines the screenshot area (i.e., the target screenshot area 3). The target screenshot area 3 is determined based on the text area selected by the first cursor and the last cursor, and is different from the target screenshot area 1. The business module crops the full-screen image based on the target screenshot area 3 to obtain the partial screenshot 3122 again, and sends the text information (i.e., text information 3123) contained in the partial screenshot to the intent recognition module. The intent recognition module generates a recommended service list 3 based on the text information 3123. The icons of each recommended service included in the recommended service list 3 are the icons included in the icon area 3126.

[0302] It should be noted that in this scenario, the recommended service list 3 can be the same as or different from the recommended service list 1, depending on the analysis results of the intent recognition module. This embodiment does not limit this.

[0303] It should be noted that in this scenario, text recognition option 3127 is selected (see reference). Figure 8e As shown in (1), the information to be transmitted in the application transmission interface 312 is the text information 3123 in the partial screenshot 3122, excluding the image information in the partial screenshot 3122.

[0304] Continue to refer to Figure 8e As shown in (1), in response to the user clicking the “Notes” application icon 31261 in the icon area 3126, the electronic device transmits text information 3123 to the “Notes” application and displays the program interface of the “Notes” application. Figure 8e As shown in (2).

[0305] For details not covered in this scenario, please refer to the relevant examples above; they will not be repeated here. It should be noted that the above scenario uses the example of a user dragging the first cursor to add selected text information. The scenario of a user dragging the last cursor or both the first and last cursors to add selected text information is similar, and will not be elaborated upon in this embodiment.

[0306] Figure 3 , Figure 8f , Figure 8g This example illustrates an application scenario where a user adjusts the type of information to be transmitted within a cross-application transfer interface. (About...) Figure 3 The explanation can be found in the previous text, and will not be repeated here. Among them, Figure 8f The user interface shown in (1) is Figure 3 The interface shown in (4) is shown in the middle.

[0307] like Figure 8f As shown in Figure (1), the text recognition option 336 is selected, and the information to be transmitted displayed in the cross-application transmission interface 33 is the text information 333 selected by the first cursor 334 and the last cursor 335. In this scenario, the type of information to be transmitted does not meet the user's expectations, and the user needs to make adjustments. For example, the user needs to change the type of information to be transmitted from text to image.

[0308] Continue to refer to Figure 8f In (1), for example, the user clicks the text recognition option 336. In response to this user action, the electronic device can play animation 5 and display user interface 316 after animation 5 has finished playing, as shown in the image. Figure 8g As shown in (1). The user interface 316 is the cross-application transfer interface. For example, animation 5 may include regenerating a partial screenshot for scanning the partial screenshot and simulating a "door opening" animation, etc., which can be referred to... Figure 8f (2), (3) and Figure 8g As shown in (1), the relevant explanation can be found in Animation 1, and will not be repeated here.

[0309] Continue to refer to Figure 8g In (1), the user interface 316 may include, but is not limited to, a sub-interface 3161, a partial screenshot 3162, a text recognition option 3163, and an icon area 3164. In this scenario, the text recognition option 3163 is unselected, indicating that the type of information to be transmitted in the cross-application transmission interface is an image. The partial screenshot 3162 is the partial screenshot finally determined by the electronic device in this scenario.

[0310] In this scenario, the user changed the type of information to be transmitted in the cross-application transmission interface, changing it from text to image. The business module redetermines the screenshot area (i.e., target screenshot area 4). Since the user did not adjust the screenshot area, target screenshot area 4 is the same as target screenshot area 1. The business module crops the full-screen image based on target screenshot area 4, obtaining a partial screenshot 3162, and sends partial screenshot 3162 to the intent recognition module. The intent recognition module performs intent recognition based on partial screenshot 3162, obtains the intent recognition result, and determines the recommended services corresponding to the intent recognition result, generating a recommended service list 4. The icons of the recommended services included in the recommended service list 4 are the icons included in icon area 3164.

[0311] It should be noted that in this scenario, the recommended service list 4 can be the same as or different from the recommended service list 1, depending on the analysis results of the intent recognition module. This embodiment does not limit this.

[0312] Continue to refer to Figure 8g As shown in (1), in response to the user clicking the “Notes” application icon 31641 in the icon area 3164, the electronic device transmits a partial screenshot 3162 to the “Notes” application and displays the program interface of the “Notes” application. (See Figure 1 for reference.) Figure 8g As shown in (2). At this point, the cross-application transmission of the image is complete.

[0313] Understandably, when electronic devices display... Figure 8g When using the user interface 316 shown in (1), the user can also change the type of information to be transmitted from an image to text. For example, in response to the user clicking the text recognition option 3163, the electronic device can display, after playing animation 6, the following... Figure 8f The user interface 33 shown in (1) is as follows. Among them, the animation 6 may include the animation of regenerating the partial screenshot, the animation of selecting text information in the partial screenshot by using the first and last cursors, the animation of scanning the regenerated partial screenshot, and the animation of the sub-interface changing to present a "door opening" visual effect, etc. For details, please refer to the other animations mentioned above, which will not be repeated here.

[0314] (ii) The business module determines, based on the layout analysis results of the screenshot trajectory and the full-screen image, that at least one text region intersecting with the screenshot trajectory area is not a valid region.

[0315] Figure 8h , Figure 8iThis example illustrates an application scenario where a user selects a text area displayed on an interface for screenshotting. In this scenario, the text area selected by the user is relatively small (e.g., the area enclosed by the user's selection trajectory is less than an area threshold). In this case, the business module uses the bounding rectangle of the screenshot trajectory area as the target screenshot area. For example, if none of the text areas intersecting with the screenshot trajectory area are valid areas, the business module uses the bounding rectangle of the screenshot trajectory area as the target screenshot area.

[0316] like Figure 8h As shown in Figure (1), the electronic device displays a user interface 318. In response to the user's selection of a screenshot area, the user's screenshot trajectory 3181 is displayed on the user interface 318. The area corresponding to the screenshot trajectory 601 intersects with two text line areas in the user interface 318.

[0317] Next, the electronic device can play an animation, and after the animation finishes playing, display user interface 323, which can be referred to as... Figure 8h As shown in (4). The user interface 323 is the cross-application transmission interface. For example, this animation can be referred to... Figure 8h The changes between the various interfaces shown can be explained in Animation 1, and will not be repeated here.

[0318] Continue to refer to Figure 8h The user interface 323 shown in (4) may include, but is not limited to, sub-interfaces 3231, partial screenshots 3232, icon areas 3233, text recognition options 3234, screenshot editing controls 3235, etc. For an explanation of this cross-application transfer interface, please refer to the previous text, which will not be repeated here.

[0319] In this scenario, the layout analysis module performs layout analysis on the full-screen image, determining that it contains only one image region and multiple text line regions. The business module identifies that the region corresponding to screenshot trajectory 3181 intersects with both text line regions, and the area ratio of the intersection in each text line region does not reach a preset threshold (e.g., 70%). Therefore, the business module determines that neither of these two text line regions is a valid region. Subsequently, the business module can use the bounding rectangle of the screenshot trajectory region as the target screenshot region. This target screenshot region contains only text information. Then, the business module can crop the full-screen image based on this target screenshot region to obtain a partial screenshot 3232 (refer to...). Figure 8hAs shown in (4), the partial screenshot is sent to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 3232, obtains the intent recognition result, determines the recommended services corresponding to the intent recognition result, and generates a list of recommended services. The icons of each recommended service included in the list of recommended services are the icons included in the icon area 3233.

[0320] Continue to refer to Figure 8i As shown in (1) and (2), in response to the user clicking the “Notes” application icon 32331 in the icon area 3233, the electronic device transmits a partial screenshot 3232 to the “Notes” application and displays the program interface of the “Notes” application. (See reference...) Figure 8i As shown in (2). At this point, the cross-application transmission of the image is complete.

[0321] Figure 8h , Figure 8j This example illustrates an application scenario where a user selects and takes a screenshot of a text area displayed on an interface. About Figure 8h The explanation can be found in the previous text, and will not be repeated here. Among them, Figure 8j The user interface shown in (1) is Figure 8h The interface shown in (4) is shown in the middle.

[0322] like Figure 8j As shown in (1), the text recognition option 3234 is not selected, and the information to be transmitted displayed in the cross-application transmission interface 323 is a partial screenshot 3232. In this scenario, the type of information to be transmitted does not meet the user's expectations. For example, the user needs to change the type of information to be transmitted from an image to text. (Continue referring to...) Figure 8j In example (1), the user clicks the text recognition option 3234. In response to this user action, the electronic device can play an animation and display the user interface 325 after the animation finishes playing, such as... Figure 8j As shown in (2).

[0323] Continue to refer to Figure 8j In section (2), the user interface 325 may include, but is not limited to, a sub-interface 3251, a partial screenshot 3252, a text recognition option 3256, and an icon area 3257. In this scenario, the text recognition option 3256 is selected, indicating that the type of information to be transmitted in the cross-application transmission interface is text.

[0324] In this scenario, because the user changes the type of information to be transmitted to text, the electronic device responds to the user's click on the text recognition option 3234 by adjusting the partial screenshot from partial screenshot 3232 to partial screenshot 3252. Partial screenshot 3252 contains more text than partial screenshot 3232. For example, partial screenshot 3252 and partial screenshot 3232 have the same height, but the width of partial screenshot 3252 is greater than the width of partial screenshot 3232. For example, the width of partial screenshot 3252 is equal to the width of the full-screen image. For example, partial screenshot 3252 includes each text line region that intersects with the screenshot trajectory area. Thus, when the user's selected area is relatively small, if the user changes the type of information to be transmitted to text, the complete text information is provided for the user to choose from, which better meets the user's expectations.

[0325] Continue to refer to Figure 8j In section (2), the text information 3255 is selected in the partial screenshot 3252 using the first cursor 3253 and the last cursor 3254. The display positions of the first cursor 3253 and the last cursor 3254 are determined based on the user's screenshot trajectory. At this time, the user can adjust the position of the first cursor 3253 and / or the last cursor 3254 to reselect the text information to be transmitted. The user can also select the icon of an application in the icon area 3257 to transmit the text information 3255 to that application. Examples can be found in the previous text and will not be repeated here.

[0326] Scene 2

[0327] In this scenario, the user selects a area on the user interface to take a screenshot, and the screenshot trajectory area only includes image information.

[0328] (i) The business module determines at least one image region that intersects with the screenshot trajectory area as a valid region based on the layout analysis results of the screenshot trajectory and the full-screen image.

[0329] Figure 4 , Figure 9a This example illustrates an application scenario where a user selects a region of an image displayed on an interface to take a screenshot. In this scenario, the user performs a selection screenshot operation on an image displayed on the user interface, and the image does not contain text information. Specifically, regarding... Figure 4 For further explanation, please refer to the previous text; it will not be repeated here.

[0330] Reference Figure 4As shown in Figure (1), the user performs a selection screenshot operation on the user interface 40. In response to the user operation, the screenshot trajectory 401 is displayed on the user interface 40. In this embodiment, the business module does not crop the full-screen image (i.e., the image corresponding to the user interface 40) based on the screenshot trajectory 401, but instead redetermines the screenshot area (i.e., the target screenshot area 5) based on the layout analysis results of the full-screen image and the screenshot trajectory. In this scenario, the layout analysis module performs layout analysis on the full-screen image and determines that the full-screen image includes only one image area. The business module determines that the area corresponding to the screenshot trajectory 401 is completely contained within the image area, and the area ratio of the area corresponding to the screenshot trajectory 401 in the image area reaches a preset threshold (e.g., 70%), then it determines the target screenshot area 5 as the image area.

[0331] Subsequently, the business module can crop the full-screen image based on the target screenshot area 5 to obtain a partial screenshot 432 (see reference). Figure 4 As shown in (4), the partial screenshot 432 is sent to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 432, obtains the intent recognition result, and determines the recommended services corresponding to the intent recognition result, generating a recommended service list 5. The icons of each recommended service included in the recommended service list 5 are the icons included in the icon area 434.

[0332] Continue to refer to Figure 9a As shown in (1), the electronic device displays a cross-application transfer interface 43. In this interface, the information to be transferred is a partial screenshot 432. In response to the user clicking the "Notes" application icon 4341 in the icon area 434, the electronic device transfers text information 333 to the "Notes" application and displays the program interface of the "Notes" application, which can be seen in the image. Figure 9a As shown in (2), image 4421 is displayed on the program interface of the "Notes" application. At this point, the cross-application transfer of the image is completed.

[0333] It should be noted that since the partial screenshot 432 does not include text information, the cross-application transmission interface 43 does not include a text recognition option, but only includes a screenshot editing control 433.

[0334] Figure 9b , Figure 9c An exemplary application scenario is illustrated where a user selects a region of an image displayed on an interface to take a screenshot. In this scenario, the user performs a selection screenshot operation on an image displayed on the user interface, and the image includes text information.

[0335] like Figure 9bAs shown in (1), in response to the user's selection of the screenshot, the user's screenshot trajectory 451 is displayed on the user interface 45. Next, the electronic device can play animation 7, and after animation 7 finishes playing, the user interface 48 is displayed, which can be referred to Figure 9b As shown in (4). The user interface 48 is the cross-application transfer interface. For example, animation 7 includes motion effects that can be referred to... Figure 9b The interface changes shown can be referred to the relevant explanation in Animation 1, and will not be repeated here.

[0336] Continue to refer to Figure 9b The user interface 48 shown in (4) may include, but is not limited to, sub-interfaces 481, partial screenshots 482, text recognition options 483, screenshot editing controls 484, and icon areas 485. Related explanations can be found above and will not be repeated here.

[0337] In this embodiment, the business module does not crop the full-screen image (i.e., the image corresponding to the user interface 45) based on the screenshot trajectory 451. Instead, it redetermines the screenshot area (i.e., the target screenshot area 6) based on the layout analysis results of the full-screen image and the screenshot trajectory. The layout analysis module performs layout analysis on the full-screen image to determine that it contains only one image region. If the business module determines that the area contained in the screenshot trajectory 451 is completely contained within this image region, and the area ratio of the region corresponding to the screenshot trajectory 451 within this image region reaches a preset threshold (e.g., 70%), then it determines the target screenshot area 6 as this image region. Subsequently, the business module can crop the full-screen image based on the target screenshot area 6 to obtain a partial screenshot 482 (refer to...). Figure 9b As shown in (4), the partial screenshot 482 is sent to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 482, obtains the intent recognition result, and determines the recommended services corresponding to the intent recognition result, generating a recommended service list 6. The icons of each recommended service included in the recommended service list 6 are the icons included in the icon area 485.

[0338] Continue to refer to Figure 9c As shown in Figure (1), the electronic device displays a cross-application transmission interface 48. It should be noted that... Figure 9c The user interface shown in (1) is Figure 9b The interface shown in (4) is shown in the cross-application transfer interface 48. In the cross-application transfer interface 48, the information to be transferred is the partial screenshot 482. In response to the user's click on the "Notes" application icon 4851 in the icon area 485, the electronic device transfers the partial screenshot 482 to the "Notes" application and displays the program interface of the "Notes" application, which can be referred to as Figure 9c As shown in (2). At this point, the cross-application transmission of the image is complete.

[0339] It should be noted that since the partial screenshot 482 includes text information, the cross-application transfer interface 48 includes both the screenshot editing control 484 and the text recognition option 483. Because the text recognition option 483 is not selected in the cross-application transfer interface 48, the type of information to be transferred in the cross-application transfer interface 48 is an image, and the image to be transferred is the partial screenshot 482.

[0340] Figure 9b , Figure 9d , Figure 9e This example illustrates an application scenario where a user selects a region of an image displayed on an interface to take a screenshot. In this scenario, the user changes the type of information to be transmitted from an image to text using a text recognition option in the cross-application transfer interface. Figure 9d The user interface shown in (1) is Figure 9b The interface shown in (4) is about Figure 9b The explanation can be found in the previous text, and will not be repeated here.

[0341] like Figure 9d As shown in (1), the text recognition option 483 is not selected, and the information to be transmitted displayed in the cross-application transmission interface 48 is a partial screenshot 482. In this scenario, the type of information to be transmitted does not meet the user's expectations, and the user needs to change the type of information to be transmitted from an image to text.

[0342] Continue to refer to Figure 9d In example (1), the user clicks the text recognition option 483. In response to this user action, the electronic device can play animation 8 and display the user interface 412 after animation 8 has finished playing, as shown in the image. Figure 9e As shown in (1). The user interface 412 is the cross-application transmission interface. For example, the animation effect of animation 8 can be referred to... Figure 9d as well as Figure 9e The interface changes shown in (1) can be referenced from the other animations mentioned above, and will not be repeated here.

[0343] Continue to refer to Figure 9e In (1), the user interface 412 may include, but is not limited to, a sub-interface 4121, a partial screenshot 4122, a text recognition option 4123, a screenshot editing control 4124, and an icon area 4125. In this scenario, the text recognition option 4123 is selected, indicating that the type of information to be transmitted in the cross-application transmission interface is text. The partial screenshot 4122 is the final partial screenshot determined by the electronic device in this scenario, and the text information 4128 on the partial screenshot 4122 is the information to be transmitted.

[0344] In this scenario, the user changed the type of information to be transmitted in the cross-application transmission interface, changing it from image to text. The business module redetermines the screenshot area (i.e., target screenshot area 7). Since the user did not adjust the screenshot area, target screenshot area 7 is the same as target screenshot area 6. The business module crops the full-screen image based on target screenshot area 7, obtaining a partial screenshot 4122. The text recognized by the business module in partial screenshot 4122 is the text information 4128 selected by the first cursor 4126 and the last cursor 4127. The business module sends the text information 4128 to the intent recognition module. The intent recognition module performs intent recognition based on the text information 4128, obtains the intent recognition result, and determines the recommended services corresponding to the intent recognition result, generating a recommended service list 7. The icons of the recommended services included in the recommended service list 7 are the icons included in the icon area 4125.

[0345] It should be noted that in this scenario, the recommended service list 7 may be the same as or different from the recommended service list 6, depending on the analysis results of the intent recognition module. This embodiment does not limit this.

[0346] Continue to refer to Figure 9e As shown in (1), in response to the user clicking the “Notes” application icon 41251 in the icon area 4125, the electronic device transmits text information 4128 to the “Notes” application and displays the program interface of the “Notes” application. Figure 9e As shown in (2).

[0347] Understandably, when electronic devices display... Figure 9e When using the user interface 412 shown in (1), the user can also change the type of information to be transmitted from text to an image. For example, in response to the user clicking the text recognition option 4123, the electronic device can display, after playing animation 9, an image such as... Figure 9d The user interface 48 shown in (1) is as follows. Among them, the animation 9 may include the animation of regenerating the partial screenshot, the animation of canceling the selection of text information in the partial screenshot, the animation of scanning the regenerated partial screenshot, and the animation of the sub-interface changing to present a "door opening" visual effect, etc. For details, please refer to the other animations mentioned above, which will not be repeated here.

[0348] Understandably, when electronic devices display... Figure 9e When using the user interface 412 shown in (1), the user can also adjust the position of the first cursor 4126 and / or the last cursor 4127 to reduce or increase the amount of text information to be transmitted. For details, please refer to Scenario 1. Figure 3 , Figure 8b , Figure 8c The example shown, or you can refer to Figure 3 , Figure 8d , Figure 8e The examples shown will not be repeated here.

[0349] (ii) The business module determines, based on the layout analysis results of the screenshot trajectory and the full-screen image, that at least one image region intersecting with the screenshot trajectory area is not a valid region.

[0350] Figure 9f This example illustrates an application scenario where a user selects a region of an image on the screen for a screenshot. In this scenario, the image region selected by the user is relatively small (e.g., the area enclosed by the user's selection is less than an area threshold). In this case, the business module uses the bounding rectangle of the selection trajectory as the target screenshot area. For example, if none of the text regions intersect with the screenshot trajectory are valid regions, the business module uses the bounding rectangle of the selection trajectory as the target screenshot area.

[0351] like Figure 9f As shown in Figure (1), in response to the user's selection of the screenshot area, the user's screenshot trajectory 4141 is displayed on the user interface 414. The area corresponding to the screenshot trajectory 4141 intersects with the image area in the user interface 414.

[0352] Next, the electronic device can play an animation, and after the animation finishes playing, display user interface 417, which can be referred to... Figure 9f As shown in (4). The user interface 417 is the cross-application transmission interface. For example, the animation effect can be referred to... Figure 9f The interface changes shown are explained in detail in Animation 1 above, and will not be repeated here. Continue referring to... Figure 9f In section (4), the user interface 417 may include, but is not limited to, sub-interfaces 4171, partial screenshots 4172, icon areas 4173, etc. For an explanation of this cross-application transfer interface, please refer to the previous text, which will not be repeated here.

[0353] In this scenario, the layout analysis module performs layout analysis on the full-screen image, determining that it contains only one image region and multiple text line regions. The business module identifies the region corresponding to screenshot trajectory 4141 as contained within the image region, and the area proportion of the region corresponding to screenshot trajectory 4141 within this image region does not reach a preset threshold (e.g., 70%). Therefore, the business module determines that this image region is not a valid region. Subsequently, the business module can use the bounding rectangle of the screenshot trajectory region as the target screenshot region. This target screenshot region contains only image information. Then, the business module can crop the full-screen image based on this target screenshot region to obtain a partial screenshot 4172 (refer to...). Figure 9fAs shown in (4), the partial screenshot is sent to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 4172, obtains the intent recognition result, determines the recommended services corresponding to the intent recognition result, and generates a list of recommended services. The icons of each recommended service included in the list of recommended services are the icons included in the icon area 4173.

[0354] At this point, the user can select an application icon in the icon area 4173 to transfer the partial screenshot 4172 to that application. If the partial screenshot 4172 contains text information, the user can also adjust the type of information to be transferred to text by clicking the text recognition option. At that time, the user can also increase or decrease the text to be transferred by adjusting the position of the first and / or last cursors. Examples can be found in the previous text and will not be repeated here.

[0355] Scene 3

[0356] In this scenario, the user selects a area on the user interface to take a screenshot, and the screenshot trajectory area includes both image and text information.

[0357] (i) Based on the layout analysis results of the screenshot trajectory and the full-screen image, the business module determines that at least one image region and / or at least one text region that intersects with the screenshot trajectory area is a valid region.

[0358] Figure 10a , Figure 10b An example is shown in an application scenario where a user selects a section of the interface to take a screenshot.

[0359] like Figure 10a As shown in Figure (1), in response to the user's selection of the screenshot area, the user's screenshot trajectory 601 is displayed on the user interface 60. The area corresponding to the screenshot trajectory 601 intersects with both the text area and the image area in the user interface 60.

[0360] Next, the electronic device can play animation 10, and after animation 10 finishes playing, display user interface 63, which can be referred to as... Figure 10b As shown in (1). The user interface 63 is the cross-application transmission interface. For example, the animation effects of animation 10 can be referenced... Figure 10a as well as Figure 10b As shown in (1), please refer to the explanation in Animation 1 above for details, which will not be repeated here.

[0361] Continue to refer to Figure 10bThe user interface 63 shown in (1) may include, but is not limited to, a sub-interface 631, a partial screenshot 632, a text recognition option 633, a screenshot editing control 634, and an icon area 635. For an explanation of this cross-application transfer interface, please refer to the preceding text, which will not be repeated here.

[0362] In this embodiment, the business module does not crop the full-screen image (i.e., the image corresponding to the user interface 60) based on the screenshot trajectory 601. Instead, it redetermines the screenshot area (i.e., the target screenshot area 8) based on the layout analysis results of the full-screen image and the screenshot trajectory. Specifically, the layout analysis module performs layout analysis on the full-screen image, determining that it includes only one image area and multiple text line areas. The business module determines that the area corresponding to the screenshot trajectory 601 intersects with the image area, and the area of ​​the intersection within the image area reaches a preset threshold (e.g., 70%). Therefore, the business module determines this image area as a valid area. Similarly, the business module can also determine two text line areas in the full-screen image as valid areas. Then, the business module can determine the target screenshot area 8 based on these multiple valid areas.

[0363] Subsequently, the business module can crop the full-screen image based on the target screenshot area 8 to obtain a partial screenshot 632 (see reference). Figure 10b As shown in (1), a partial screenshot 632 is sent to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 632, obtains the intent recognition result, determines the recommended services corresponding to the intent recognition result, and generates a recommended service list 8. The icons of each recommended service included in the recommended service list 8 are the icons included in the icon area 635.

[0364] Continue to refer to Figure 10b As shown in (1), the electronic device displays a cross-application transfer interface 63. In the cross-application transfer interface 63, the information to be transferred is a partial screenshot 632. In response to the user clicking the "Notes" application icon 6351 in the icon area 635, the electronic device transfers the partial screenshot 632 to the "Notes" application and displays the program interface of the "Notes" application, which can be seen in the image. Figure 10b As shown in (2), image 6421 is displayed on the program interface of the "Notes" application. At this point, the cross-application transfer of the image is completed.

[0365] It should be noted that although the partial screenshot 632 includes text information, the text recognition option 633 is not selected in the cross-application transmission interface 63. Therefore, the type of information to be transmitted in the cross-application transmission interface 63 is an image, and the image to be transmitted is the partial screenshot 632.

[0366] Understandably, when the electronic device displays the cross-application transfer interface 63, the user can also adjust the state of the text recognition option 633 to change the type of information to be transferred from image to text. For details, please refer to... Figure 9b , Figure 9d , Figure 9e The examples shown will not be repeated here.

[0367] Figure 10a , Figure 10c This example illustrates another application scenario where a user selects a section of the interface to take a screenshot. About Figure 10a The relevant explanations can be found in the previous text, and will not be repeated here. Distinguished from... Figure 10a , Figure 10b The scenario shown here involves images and text that need to be transmitted.

[0368] In response to the user's selection of a screenshot area, the user's screenshot trajectory 601 is displayed on the user interface 60. Next, the electronic device can play animation 10, and after animation 10 finishes playing, the user interface 65 is displayed, as can be seen from... Figure 10c As shown in (1). The user interface 65 is the cross-application transmission interface. The user interface 65 may include, but is not limited to, a sub-interface 651, a partial screenshot 652, a text recognition option 653, a screenshot editing control 654, and an icon area 655. The partial screenshot 652 is the final partial screenshot determined by the electronic device in this scenario, and it includes both text and image information. Details not explained here can be found in the preceding text and will not be repeated here.

[0369] same Figure 10a , Figure 10b In the scenario shown, the business module redefines the screenshot area as the target screenshot area 8. The business module crops the full-screen image based on the target screenshot area 8, obtaining a partial screenshot 652, and sends the partial screenshot 652 and the text information contained within it to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 652 and the text information contained within it, obtaining the intent recognition result, and determines the recommended services corresponding to this result, generating a recommended service list 9. The icons for each recommended service included in the recommended service list 9 are the icons included in the icon area 655.

[0370] Continue to refer to Figure 10cAs shown in (1), the electronic device displays a cross-application transfer interface 65. In the cross-application transfer interface 65, the information to be transferred is the image 659 in the partial screenshot 652, and the text information 658 in the partial screenshot 652 that is selected by the first cursor 656 and the last cursor 657. In response to the user clicking the "Notes" application icon 6551 in the icon area 655, the electronic device transfers the partial screenshot 652 to the "Notes" application and displays the program interface of the "Notes" application, which can be seen in the image. Figure 10c As shown in (2), the program interface of the "Notes" application displays image 6621 and text information 6622.

[0371] It should be noted that in this scenario, the partial screenshot 652 includes both text information and image information. Since the text recognition option 653 is selected, the text information in the partial screenshot 652 is selected by the first and last cursors. Therefore, the type of information to be transmitted in the cross-application transmission interface 65 is both image and text.

[0372] Figure 10d , Figure 10e An exemplary application scenario is illustrated where a user selects a portion of an interface to take a screenshot. In this scenario, although the area of ​​the user's screenshot trajectory intersects with the image area in the full-screen image, the business module ultimately determines that the partial screenshot does not include image information because the area of ​​the intersection in the image area does not reach a preset threshold (e.g., 70%).

[0373] like Figure 10d As shown in Figure (1), in response to the user's selection of the screenshot area, the user's screenshot trajectory 671 is displayed on the user interface 67. The area corresponding to the screenshot trajectory 671 intersects with both the text area and the image area in the user interface 67.

[0374] Next, the electronic device can play animation 11, and after animation 11 finishes playing, display user interface 610, which can be referred to as... Figure 10e As shown in (1). The user interface 610 is the cross-application transmission interface. For example, the animation effect of animation 11 can be referred to... Figure 10d as well as Figure 10e The interface changes shown in (1) can be explained in detail in Animation 1 above, and will not be repeated here.

[0375] Continue to refer to Figure 10e The user interface 610 shown in (1) may include, but is not limited to, a sub-interface 6101, a partial screenshot 6102, a text recognition option 6103, a screenshot editing control 6104, and an icon area 6105. The partial screenshot 6102 is the final partial screenshot determined by the electronic device in this scenario. Figure 10dThe partial screenshot 692 shown in (3) is only schematically illustrated in animation 11. In this scenario, although the user-selected trajectory area includes image information, the partial screenshot 6102 finally determined by the business module only includes text information.

[0376] In the user interface 610, the text recognition option 6103 is selected, so the text information 6108 selected by the first cursor 6106 and the last cursor 6107 in the partial screenshot 6102 is the information to be transmitted.

[0377] In this embodiment, the business module does not crop the full-screen image (i.e., the image corresponding to the user interface 67) based on the screenshot trajectory 671. Instead, it redetermines the screenshot area (i.e., the target screenshot area 9) based on the layout analysis results of the full-screen image and the screenshot trajectory. The layout analysis module performs layout analysis on the full-screen image, determining that it includes only one image area and multiple text line areas. The business module determines that the area corresponding to the screenshot trajectory 671 intersects with the image area, but the area of ​​the intersection within the image area does not reach a preset threshold (e.g., 70%). Therefore, the business module determines that the image area is not a valid area. Similarly, the business module can determine that two text line areas in the full-screen image are valid areas. Then, the business module can determine the target screenshot area 9 based on these two text line areas.

[0378] Subsequently, the business module can crop the full-screen image based on the target screenshot area 9 to obtain a partial screenshot 6102 (see reference). Figure 10e As shown in (1), the text information 6108 contained in the partial screenshot 6102 is sent to the intent recognition module. The intent recognition module performs intent recognition based on the text information 6108, obtains the intent recognition result, and determines the recommended services corresponding to the intent recognition result, generating a recommended service list 10. The icons of each recommended service included in the recommended service list 10 are the icons included in the icon area 6105.

[0379] Continue to refer to Figure 10e As shown in (1), the electronic device displays a cross-application transfer interface 610. In the cross-application transfer interface 610, the information to be transferred is the text information 6108 selected in the partial screenshot 6102. In response to the user clicking the "Notes" application icon 61051 in the icon area 6105, the electronic device transfers the text information 6108 to the "Notes" application and displays the program interface of the "Notes" application, which can be seen in the image. Figure 10e As shown in (2).

[0380] (ii) Based on the layout analysis results of the screenshot trajectory and the full-screen image, the business module determines that at least one image region and at least one text region that intersect with the screenshot trajectory area are not valid regions.

[0381] Figure 10f This example illustrates an application scenario where a user selects a region of an image on the screen for a screenshot. In this scenario, the image region selected by the user is relatively small (e.g., the area enclosed by the user's selection is less than an area threshold). In this case, the business module uses the bounding rectangle of the selection trajectory as the target screenshot area. For example, if neither the image region nor the text region intersects with the screenshot trajectory is a valid region, the business module uses the bounding rectangle of the selection trajectory as the target screenshot area.

[0382] like Figure 10f As shown in Figure (1), in response to the user's selection of the screenshot area, the user's screenshot trajectory 6121 is displayed on the user interface 612. The area corresponding to the screenshot trajectory 6121 intersects with the image area and text line area in the user interface 612.

[0383] Next, the electronic device can play an animation, and after the animation finishes playing, display the user interface 613, which can be referred to as... Figure 10f As shown in (4). The user interface 613 is the cross-application transmission interface. For example, this animation can be referred to... Figure 10f The changes in (1), (2), (3), and (4) can be explained in the previous text and will not be repeated here.

[0384] Continue to refer to Figure 10f The user interface 613 shown in (4) may include, but is not limited to, sub-interfaces 6131, partial screenshots 6132, icon areas 613, text recognition options, screenshot editing controls, etc. For an explanation of this cross-application transfer interface, please refer to the preceding text, which will not be repeated here.

[0385] In this scenario, the layout analysis module performs layout analysis on the full-screen image, determining that it contains only one image region and multiple text line regions. The business module identifies the region corresponding to screenshot trajectory 6131, which intersects with both the image region and the text line region, but the area ratio of the intersection within the image region and the text line region does not reach a preset threshold (e.g., 70%). Therefore, the business module determines that neither the image region nor the text line region is a valid region. Subsequently, the business module can use the bounding rectangle of the screenshot trajectory region as the target screenshot region. This target screenshot region contains both image and text information. Then, the business module can crop the full-screen image based on this target screenshot region to obtain a partial screenshot 6132 (refer to...). Figure 10f As shown in (4), the partial screenshot is sent to the intent recognition module. The intent recognition module performs intent recognition based on the partial screenshot 6132, obtains the intent recognition result, determines the recommended services corresponding to the intent recognition result, and generates a list of recommended services. The icons of each recommended service included in the list of recommended services are the icons included in the icon area 6133.

[0386] At this point, the user can select an application icon in the icon area 6133 to transfer the partial screenshot 6132 to that application. Since the partial screenshot 6133 contains text information, the user can also adjust the type of information to be transferred to text by clicking the text recognition option (see [reference]). Figure 8j As shown in the example, the partial screenshot can be adjusted according to the text line area. At that time, the text to be transmitted can also be increased or decreased by adjusting the position of the first cursor and / or the last cursor. For relevant examples, please refer to the previous text, which will not be repeated here.

[0387] It should be noted that the above application scenarios are all explained using the example of a user performing a single selection and screenshot operation. Understandably, in response to the user's selection and screenshot operation, when the electronic device displays the cross-application transfer interface, if the user's various adjustments fail to make the information to be transferred in the cross-application transfer interface meet the user's expectations, the user will control the electronic device to exit the display of the cross-application transfer interface and perform the selection and screenshot operation again. This embodiment will not elaborate further on this.

[0388] It should be noted that the animation effects described in the above application scenarios, such as the animation effect when an electronic device switches from the interface where the user performs a selection and screenshot operation to the cross-application transmission interface, or the animation effect when an electronic device switches from cross-application transmission interface 1 to cross-application transmission interface 2, are all illustrative examples. In practical applications, those skilled in the art can adjust the animation effects as needed, and this embodiment does not limit this.

[0389] This embodiment also provides a computer storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned related method steps to implement the partial screenshot method in the above embodiment.

[0390] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the partial screenshot method in the above embodiment.

[0391] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component or module. The apparatus may include a connected processor and a memory; wherein the memory is used to store computer execution instructions, and when the apparatus is running, the processor may execute the computer execution instructions stored in the memory to cause the chip to execute the partial screenshot method in the above method embodiments.

[0392] In this embodiment, the electronic devices (such as mobile phones), computer storage media, computer program products, or chips are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0393] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0394] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0395] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for partial screenshotting, characterized in that, include: Display the first screen; In response to a user's first operation on the first interface, a second interface is displayed. The second interface includes a first image and a first list. The first image is a partial screenshot of the first interface and includes first information. The first list includes at least one icon. In response to the user's first operation on the first icon in the first list, the first information is transmitted to the application service corresponding to the first icon.

2. The method according to claim 1, characterized in that, The first operation is to draw a trajectory on the first interface.

3. The method according to claim 2, characterized in that, The first text information is displayed on the first interface. When the trajectory surrounds a portion of the first text information, the first image is a screenshot of the first text information.

4. The method according to claim 2, characterized in that, The second image is displayed on the first interface. When the trajectory surrounds a portion of the second image, the first image is a screenshot of the second image.

5. The method according to claim 2, characterized in that, The second text information and the third image are displayed on the first interface. When the trajectory surrounds a portion of the second text information and a portion of the third image, the first image is a screenshot of the second text information, or the first image is a screenshot of the third image, or the first image is a screenshot of both the second text information and the third image.

6. The method according to claim 1, characterized in that, When displaying the second interface, it also includes: In response to a second operation by the user on the second interface, a third interface is displayed, the third interface including a fourth image and a second list, the fourth image being a partial screenshot of the first interface, the fourth image including second information, and the second list including at least one icon; In response to the user's first operation on the second icon in the second list, the second information is transmitted to the application service corresponding to the second icon.

7. The method according to claim 6, characterized in that, Display the first and last cursors on the first image; The second operation includes: adjusting the position of the first cursor and / or the last cursor; The fourth image is a partial screenshot that has been redefined in the first interface based on the adjusted positions of the first and last cursors. The second information includes the text information selected by the first cursor and the last cursor on the fourth image.

8. The method according to claim 6, characterized in that, The second interface includes a first control, and the state of the first control is a first state; Transmitting the first information to the application service corresponding to the first icon includes: transmitting the text information included in the first image to the application service corresponding to the first icon; The second operation includes: adjusting the state of the first control to the second state; Transmitting the second information to the application service corresponding to the second icon includes: transmitting the fourth image to the application service corresponding to the first icon.

9. The method according to claim 6, characterized in that, The second interface includes a second control; The method further includes: In response to a third operation by the user on the second control, a fourth interface is displayed; the fourth interface includes the first image and at least one third control, the third control being used to perform editing operations on the first image.

10. The method according to claim 1, characterized in that, The second interface also includes a first sub-interface, the content of which is the same as that of the first interface; The first sub-interface is obtained by performing a three-dimensional transformation on the first interface.

11. The method according to claim 10, characterized in that, Before displaying the second interface, the following is also included: Play animation; The animation includes at least one of the following motion effects: a motion effect of generating a screenshot frame based on a trajectory, a motion effect of cropping a partial screenshot based on the screenshot frame, a motion effect of scanning the partial screenshot, and a motion effect of transforming the first interface into the first sub-interface.

12. The method according to claim 2, characterized in that, The first operation is to draw the trajectory on the first interface using a knuckle.

13. The method according to claim 2, characterized in that, Before displaying the second interface, the following is also included: The first interface is subjected to layout analysis to obtain layout analysis results; wherein, the layout analysis results include information of image areas, information of text areas and all text information contained in the first interface. The area enclosed by the trajectory is matched with the image area and the text area to determine the first screenshot area; The first image is obtained by cropping the first screenshot area from the first interface.

14. The method according to claim 13, characterized in that, Matching the region enclosed by the trajectory with the image region and the text region to determine the first screenshot region includes: If the area enclosed by the trajectory is contained within the target image area, and the area ratio of the area enclosed by the trajectory in the target image area reaches a preset threshold, then the target image area is determined to be the first screenshot area. or, If the area enclosed by the trajectory is contained within the target text area, and the area ratio of the area enclosed by the trajectory in the target text area reaches a preset threshold, then the target text area is determined to be the first screenshot area.

15. The method according to claim 13, characterized in that, Matching the region enclosed by the trajectory with the image region and the text region to determine the first screenshot region includes: If the area enclosed by the trajectory intersects with at least one of the image areas and the text areas, then each valid area is determined in at least one of the image areas and / or the text areas; Wherein, the intersection of the effective region and the region enclosed by the trajectory has an area ratio in the effective region that reaches a preset threshold. The first screenshot area is defined as one of the effective regions contained in the first interface.

16. The method according to claim 14 or 15, characterized in that, Transmitting the first information to the application service corresponding to the first icon includes: If the first screenshot area only includes a text area, then the text information included in the first image is transmitted to the application service corresponding to the first icon; If the first screenshot area includes an image area, then the first image is transmitted to the application service corresponding to the first icon.

17. The method according to claim 14 or 15, characterized in that, Before displaying the second interface, the following is also included: If the first screenshot area only includes a text area, then intent recognition is performed based on the text information included in the first image, and the first list is determined based on the intent recognition result; If the first screenshot area includes an image area, then intent recognition is performed based on the first image, and the first list is determined based on the intent recognition result.

18. The method according to claim 6, characterized in that, Before displaying the third interface, it also includes: In response to the user's second operation on the second interface, determine the second screenshot area; The fourth image is obtained by cropping the second screenshot area from the first interface; Intent recognition is performed based on the second information included in the fourth image, and the second list is determined based on the intent recognition result.

19. The method according to claim 2, characterized in that, The first image is obtained by cropping it from the first interface based on the third screenshot area, which is the bounding rectangle area of ​​the trajectory.

20. The method according to claim 19, characterized in that, Before displaying the second interface, the following is also included: The first interface is subjected to layout analysis to obtain layout analysis results; wherein, the layout analysis results include information of image areas, information of text areas and all text information contained in the first interface. If the area enclosed by the trajectory intersects with at least one of the image areas and / or the text areas, and none of the at least one of the image areas and / or the text areas are valid areas, then the bounding rectangle area of ​​the trajectory is determined to be the third screenshot area. Wherein, the intersection of the effective region and the region enclosed by the trajectory has an area ratio in the effective region that reaches a preset threshold.

21. An electronic device, characterized in that, include: One or more processors; Memory; A display screen, used to show the interface; And one or more computer programs, wherein the one or more computer programs are stored on the memory, and when the computer programs are executed by the one or more processors, cause the electronic device to perform the partial screenshot method as described in any one of claims 1-20.

22. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the partial screenshot method as described in any one of claims 1-20.

Images