Information transmission method, device and storage medium

Abstract

Provided in the present application are an information transmission method, a device and a storage medium. In the method, after a user performs a circle-selection operation, displaying on a user interface an icon of an application and / or service matching content selected by a circle-selection trajectory corresponding to the circle-selection operation; after the user adjusts the selected content, re-recommending an application and / or service on the basis of the adjusted content, such that the recommended application and / or service can be better adapted to actual use requirements of the user; in addition, invoking the any-door service by means of the circle-selection operation enables the transmission of content in a selected area, rather than the transmission of an entire picture corresponding to a first interface.

Description

Information transmission methods, equipment and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411244812.0, filed on September 5, 2024, entitled "Information Transmission Method, Apparatus and Storage Medium", and to Chinese Patent Application No. 202411332538.2, filed on September 20, 2024, entitled "Data Transmission Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to an information transmission method, device and storage medium. Background Technology

[0003] With the continuous development of electronic information technology, smartphones, tablets, and other smart devices have become indispensable products in our daily lives, work, and entertainment. In some usage scenarios, users use electronic devices to transfer information across applications, such as transferring information from one application to another, where the second application shares or uploads that information.

[0004] However, currently, when users transfer information, such as images, between different applications, they can only transfer the entire image. If a user wants to transfer a specific area of ​​the image, they need to edit the image, save the desired area as a separate image, and then transfer the edited image to transfer the portion of the original image. In other words, it's not possible to quickly transfer a specific area of ​​an image. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of this application provide an information transmission method, device, and storage medium, aiming to enable electronic devices to efficiently and quickly complete information transmission between applications, effectively reduce the operational steps required for users to complete cross-application information transmission, improve the interaction efficiency between users and electronic devices, and thereby enhance the user experience.

[0006] In a first aspect, embodiments of this application provide an information transmission method. The method includes: receiving a first operation applied to a first interface, the first operation being a selection operation; at a first moment, in response to the first operation, displaying a second interface, the second interface including a selection trajectory corresponding to the first operation displayed on the first interface; at a second moment, displaying a third interface, the third interface including a selection display area and a service recommendation area, the selection display area including the first interface and a first selection identifier displayed on the first interface, the first selection identifier identifying the first selection area in the first interface corresponding to the selection trajectory, the service recommendation area including icons of a first group of applications and / or services determined based on the first selection area; at a third moment, receiving a second operation applied to the third interface, the second operation being an operation to adjust the first selection area; in response to the second operation, displaying a fourth interface, the fourth interface including a selection display area and a service recommendation area, the selection display area including the first interface and a second selection identifier displayed on the first interface, the second selection identifier identifying a second selection area, the second selection area being a selection area adjusted by the second operation on the first selection area, the service recommendation area including icons of a second group of applications and / or services determined based on the second selection area; wherein the second moment is later than the first moment, and the third moment is later than the second moment.

[0007] For example, the first interface can be understood as the currently displayed user interface, such as interface 10a, interface 10e, etc.

[0008] For example, a selection operation refers to a user action of circling specific content on the screen.

[0009] For example, a selection action can be a user action performed on the screen using a knuckle.

[0010] For example, a selection operation can also be a user action performed by the user using their finger on the screen.

[0011] For example, the selection operation can also be a user operation performed on the screen using an external device, such as a stylus or mouse.

[0012] For example, the second interface can be understood as a user interface that displays the selected trajectory.

[0013] For example, if the first interface is interface 10a, the second interface can be interface 10d.

[0014] For example, if the first interface is interface 10e, the second interface can be interface 20e.

[0015] For example, the selected trajectory can be a closed loop.

[0016] For example, in the scenario of the door opening animation described in the following embodiments, where the first processing is performed on the first interface in the selection display area, the third interface can be understood as interface 50d or interface 60e. In this case, the first interface displayed in the third interface can be understood as display element 50d-1 or display element 60e-1. The first selection identifier can be understood as selection box 50d-3 or selection box 60e-2. The first selection area can be understood as the area within selection box 50d-3 or selection box 60e-2. The service recommendation area displayed in the third interface can be understood as icon area 50d-2 or icon area 60e-4. The icons of the first group of applications and / or services are the icons displayed in icon area 50d-2 or icon area 60e-4.

[0017] For example, the second operation could be a user action to adjust the selection box 50d-3. This user action could include a down event (clicking the bar on the selection box 50d-3), a move event (moving the finger on the screen while holding down the bar on the selection box 50d-3), and an up event (lifting the hand).

[0018] For example, the fourth interface can be understood as the user interface displayed after the second operation is completed, i.e., after the user raises their hand, such as interface 70d. The second selection indicator can be understood as the selection box 70d-3. The second selection area can be understood as the area within the selection box 70d-3. The service recommendation area displayed in the fourth interface can be understood as the icon area 70d-2. The icons of the second group of applications and / or services are the icons displayed in the icon area 70d-2.

[0019] Therefore, after the user makes a selection, the user interface displays icons of applications and / or services that match the selected content along the selection trajectory. After the user adjusts the selected content, the user recommends applications and / or services again based on the adjusted content, so that the recommended applications and / or services can better meet the user's actual needs. Furthermore, by launching the "Anywhere Door" service through the selection operation, the user can transfer the content in the selected area, rather than transferring the entire image corresponding to the first interface.

[0020] According to the first aspect, after displaying the second interface, the method further includes: displaying a third selection identifier, the third selection identifier being used to identify the first selection area, the shape of the third selection identifier being different from that of the first selection identifier; and after displaying the third selection identifier, displaying a first animation effect, the first animation effect being an animation effect transitioning from the third selection identifier to the first selection identifier.

[0021] For example, the first motion effect can be understood as the transition motion effect described in the following embodiments.

[0022] For example, a first animation effect is displayed on the third selection icon so that during the transition from the third selection icon to the first selection icon, a second animation effect can also be displayed on the third selection icon after the first animation effect is displayed.

[0023] For example, the third selection identifier for displaying the first animation effect and the second animation effect can be different selection boxes. For instance, the third selection identifier for displaying the first animation effect can be selection box 20d-1 (or selection box 30e-2, or selection box 20f-2), and the third selection identifier for displaying the second animation effect can be selection box 30d-1 (or selection box 40e-2, or selection box 30f-2).

[0024] For example, the third selection indicator that displays the first and second animation effects can also be the same selection box. For instance, after the selection trajectory is displayed and before the first selection indicator is displayed, the third selection indicator displayed in the user interface can be fixed as selection box 30d-1 (or selection box 40e-2, or selection box 30f-2), only the animation effect displayed is different at different times.

[0025] Therefore, by displaying a third selection icon on the interface while waiting for the third interface to appear, and then displaying the first and second animation effects on the third selection icon in sequence, the fun and visual appeal of the waiting process are increased, further enhancing the user experience.

[0026] For specific implementation details of this aspect, please refer to the description of the embodiments shown in Figures 6A, 7A, and 7B in the following embodiments, which will not be repeated here.

[0027] According to the first aspect, or any implementation of the first aspect above, after displaying the first animation effect, the method further includes: displaying a second animation effect, wherein the second animation effect is an animation effect that simulates scanning the first selected area.

[0028] For example, the second motion effect can be understood as the scanning motion effect described in the following embodiments.

[0029] For example, a third selection indicator that displays the second animation is, for instance, a selection box 30d-1 displayed in interface 40d. Correspondingly, an animation that simulates scanning the first selection area is, for instance, the effect of displaying a moving scan box 30d-2 on the selection box 30d-1 displayed in interface 40d.

[0030] For example, the third selection indicator that displays the second animation is, for instance, the selection box 100d-2 displayed in interface 100d. Accordingly, the animation that simulates scanning the first selection area is, for instance, the effect of displaying a moving scan box 100d-3 on the selection box 100d-2 displayed in interface 100d.

[0031] For example, the third selection indicator that displays the second animation is, for instance, the selection box 40e-2 displayed in interface 50e. Accordingly, the animation that simulates scanning the first selection area is, for instance, the effect of displaying a moving scan box 50e-1 on the selection box 40e-2 displayed in interface 50e.

[0032] For example, the third selection indicator that displays the second animation is, for instance, the selection box 30f-2 displayed in interface 30f. Correspondingly, the animation that simulates scanning the first selection area is, for instance, the effect of displaying a moving scan box 30f-3 on the selection box 30f-2 displayed in interface 30f.

[0033] Therefore, by simulating the scanning of the first selected area during the waiting process for the fourth interface to be displayed, the fun and visual appeal of the waiting process are increased, further enhancing the user experience.

[0034] According to the first aspect, or any of the above implementations of the first aspect, the first selection area of ​​the third selection identifier cannot be adjusted.

[0035] Therefore, the first selection area identified by the third selection identifier cannot be adjusted, thus avoiding user adjustments to the first selection area during the display of the first and second animation effects, and ensuring that the icons of applications and / or services finally displayed in the service recommendation area match the content of the first selection area.

[0036] According to the first aspect, or any implementation of the first aspect above, the first interface displayed in the selected display area of ​​the third interface is the first interface after the first processing. The first interface after the first processing displays a first selection mark, and the first selection mark displays a first screenshot. The first screenshot is the screenshot corresponding to the first selection area indicated by the first selection mark. The first processing includes one or more of the following: scaling, deformation, distortion and changing transparency.

[0037] For example, the first processing performed on the first interface is, for instance, the door opening effect of the full-screen screenshot corresponding to interface 10a as described in the following embodiments, that is, the visual effect presented by display element 50d-1 during the process of displaying the interface changing from interface 40d to displaying interface 50d.

[0038] For specific implementation details of this aspect, please refer to the description of the embodiments shown in the following examples regarding interfaces 40d to 50d, or interfaces 50e to 60e, which will not be repeated here.

[0039] According to the first aspect, or any implementation of the first aspect above, the first processing is used to make the first interface displayed in the selected display area of ​​the third interface present a visual effect of pushing inward.

[0040] For example, when the third interface is interface 50d, the first interface selected for display area in the third interface is, for example, display element 50d-1. In this case, the first processing performed on the first interface is, for example, scaling, deforming, distorting, and changing the transparency of a screenshot of interface 10a, and then pushing the processed screenshot inward to the extent that display element 50d-1 is present in interface 50d. The visual effect presented in this process is the visual effect of pushing inward.

[0041] According to the first aspect, or any implementation of the first aspect above, in response to the second operation, a fourth interface is displayed, including: in response to the first part of the second operation, displaying the first interface that has undergone the second processing in the selection display area, and hiding the service recommendation area, the first part of the second operation includes the down event and / or part of the move event in the second operation, the second processing is the same type of processing as the first processing, and the processing degree of the second processing is less than that of the first processing; in response to the second part of the second operation, displaying the first interface that has undergone the first processing in the selection display area, displaying a second selection icon above the first interface that has undergone the first processing, displaying a second screenshot in the second selection icon, the second screenshot being a screenshot corresponding to the second selection area indicated by the second selection icon, and displaying the service recommendation area, the second part of the second operation includes the up event in the second operation.

[0042] For example, the second processing performed on the first interface is, for instance, performing a micro-closing motion effect on display element 50d-1 as described in the following embodiment, thus transforming it into display element 60d-1.

[0043] For example, when the second selection is identified as a selection box 60d-2, the second selection area can be understood as the area within the selection box 60d-2, and the second screenshot is, for example, the content displayed in the selection box 60d-2.

[0044] Therefore, as the user adjusts the first selection indicator, the user interface exhibits the above changes, which not only improves the user's visual experience but also avoids accidental touches on controls other than the first selection indicator during the adjustment process, further ensuring the user experience.

[0045] For specific implementation details of this aspect, please refer to the description of the embodiments shown in Figures 6A and 6D, and Figures 8A and 8B in the following embodiments, which will not be repeated here.

[0046] According to the first aspect, or any of the above implementations of the first aspect, the second processing is used to make the first interface displayed in the selected display area of ​​the third interface present a slightly outward visual effect.

[0047] For example, when the third interface is interface 50d, the first interface selected for display area in the third interface is, for example, display element 50d-1. In this case, the second processing performed on the first interface is, for example, slightly pulling display element 50d-1 in interface 50d outwards to the extent that display element 60d-1 in interface 60d. The visual effect presented during the process of display element 50d-1 becoming display element 60d-1 is the visual effect of slightly pulling outwards.

[0048] According to the first aspect, or any implementation of the first aspect above, the selection display area further includes an editing control displayed on the first interface; when the editing control is displayed, the method further includes: in response to a third operation applied to the editing control, displaying a fifth interface, the fifth interface including a first image and a control for editing the first image, the first image being a first screenshot corresponding to a first selection area, or a second screenshot corresponding to a second selection area; in response to a fourth operation applied to the first image, displaying a second image, the second image being an image after the first image has been edited by the fourth operation; in response to a fifth operation applied to the second image, displaying a third image above the second image at the position where the user performs the fifth operation, the third image being smaller than the second image and including all the content of the second image; in response to a sixth operation applied to the third image, displaying a sixth interface, the sixth interface including a selection display area and a service recommendation area, the selection display area including the fifth interface displaying the second image and a fourth image displayed on the fifth interface, the fourth image being smaller than the third image and including all the content of the third image, the service recommendation area including icons of a third group of applications and / or services determined according to the fourth image.

[0049] For example, when editing controls are displayed on a first interface, the first interface is, for instance, the first interface after the first processing. That is, the first interface with the door opening animation effect. In this case, the editing control displayed on the first interface in the display area is selected, such as editing control 50d-5, or editing control 70d-5, or editing control 110d-5, or editing control 60e-6.

[0050] Therefore, by providing editing controls on the first interface for selecting the display area, the selected content in the current interface can be edited. This eliminates the need to first save the selected content and then open dedicated image editing software, reducing the number of human-computer interactions and improving the user experience.

[0051] For specific implementation details of this aspect, please refer to the description of the embodiment shown in FIG6D in the following embodiments, which will not be repeated here.

[0052] According to the first aspect, or any implementation of the first aspect above, the selection display area further includes a text selection control displayed on the first interface; after displaying the fourth interface, the method further includes: responding to a seventh operation acting on the text selection control, displaying a seventh interface, the seventh interface including a selection display area, the selection display area including the first interface, and a fourth selection identifier displayed on the first interface, the fourth selection identifier being used to identify the second selection area, the shape of the fourth selection identifier being different from the second selection identifier; after displaying the fourth selection identifier, displaying a first animation effect, the first animation effect being a transition animation from the fourth selection identifier to the fifth selection identifier; when the seventh interface is displayed for a set time, and the second selection area includes text elements, displaying an eighth interface, the eighth interface including a selection display area and a service recommendation area, the selection display area including the first interface, and a fifth selection identifier displayed on the first interface, the fifth selection identifier being used to identify the third selection area corresponding to the text elements in the second selection area, the service recommendation area including icons of a fourth group of applications and / or services determined according to the third selection area.

[0053] For example, when a text selection control is displayed on the first interface, the first interface is, for instance, the first interface after the first processing. That is, the first interface with the door opening animation effect. In this case, the text selection control displayed on the first interface in the display area is selected, such as text selection control 50d-4, or text selection control 70d-4, or text selection control 110d-4, or text selection control 60e-5.

[0054] Therefore, by providing a text selection control on the first interface for selecting the display area, it is possible to automatically adjust the current selection marker to a selection marker for the selected text element. That is, the selection area marked by the current selection marker is adjusted to represent the area containing the text element within that selection area.

[0055] For specific implementation details of this aspect, please refer to the description of the embodiment shown in FIG6C in the following embodiments, which will not be repeated here.

[0056] According to the first aspect, or any implementation of the first aspect above, the selected display area also includes a save control displayed on the first interface; when the save control is displayed, the method further includes: responding to the eighth operation acting on the save control and saving the content within the selected trajectory.

[0057] For example, when a save control is displayed on a first interface, the first interface is, for instance, the first interface after the first processing. That is, the first interface with the door opening animation. In this case, the save control displayed on the first interface in the display area is selected, such as save control 50d-6, or save control 70d-6, or save control 110d-6, or save control 60e-7.

[0058] Therefore, by providing a save control on the first interface where the display area is selected, the needs of users who use knuckle screenshots can be accommodated.

[0059] According to the first aspect, or any implementation of the first aspect above, the service recommendation areas displayed in the third and fourth interfaces both include icons of user-defined recommended applications and / or services, and the service recommendation area displayed in the third interface also includes icons of a first group of applications and / or services determined according to the first selection area, and the service recommendation area displayed in the fourth interface also includes icons of a second group of applications and / or services determined according to the second selection area.

[0060] Therefore, it can not only meet the user's customization needs, but also match the icons of applications and / or services displayed in the service recommendation area with the content in the second selection area, thus meeting the diverse usage needs of users.

[0061] According to the first aspect, or any implementation of the first aspect above, before displaying the fourth interface, the method further includes: performing intent recognition on the second selection area to determine the user intent corresponding to the second selection area; obtaining information on a second set of applications and / or services that match the user intent based on the user intent; and determining the icons of the second set of applications and / or services to be displayed in the service recommendation area based on the information on the second set of applications and / or services.

[0062] This allows the icons of applications and / or services ultimately displayed in the service recommendation area to better match the content selected in the current interface, thereby improving the user experience.

[0063] For specific implementation details of this aspect, please refer to the description of the embodiments shown in Figures 6B and 8B in the following embodiments, which will not be repeated here.

[0064] According to the first aspect, or any of the above implementations of the first aspect, the selection operation is a user operation performed by the user using their knuckles.

[0065] For example, a user's selection operation using their knuckles could be a user operation that draws a selection trajectory 10d-2 in interface 10d, or a user operation that draws a selection trajectory 20e-1 in interface 20e, or a user operation that draws a selection trajectory 10f-1 in interface 10f.

[0066] Therefore, without integrating new selection operation recognition capabilities, the user's selection operation can be accurately identified based on the existing knuckle recognition capabilities.

[0067] According to the first aspect, or any implementation of the first aspect above, after displaying the fourth interface in response to the second operation, the method further includes: in response to a ninth operation acting on the icon of any application and / or service in the service recommendation area, transmitting the second selection area to the application or service corresponding to the selected icon.

[0068] For example, the ninth action is a user action such as clicking the icon of any application and / or service in the service recommendation area, as shown by the user in interface 70d in Figure 6B (2).

[0069] For example, the ninth action is a user action of dragging a second selection area closer to the icon of any application and / or service in the service recommendation area.

[0070] This allows the selected content to be transmitted directly to the application or service corresponding to the selected icon in a single step.

[0071] According to the first aspect, or any implementation of the first aspect above, the method further includes: displaying a ninth interface; in response to a tenth operation on the ninth interface, displaying an operation trajectory on the ninth interface, the tenth operation being a selection operation; after displaying the operation trajectory, displaying a sixth selection identifier on the ninth interface, the sixth selection identifier indicating a fourth selection area on the ninth interface, the fourth selection area being determined based on the operation trajectory; after displaying the sixth selection identifier, displaying a seventh selection identifier on the ninth interface, the seventh selection identifier indicating a fifth selection area on the ninth interface, the fifth selection area being obtained by expanding the fourth selection area; displaying a tenth interface, the tenth interface including application and / or service icons recommended based on the content in the fifth selection area; in response to an eleventh operation by the user on the fifth selection area, transmitting the content in the fifth selection area through the application and / or service corresponding to the icon selected by the user.

[0072] In the above technical solution, the user determines the corresponding selection area in the interface by making a selection operation, and then displays the matching service icon based on the content of the selection area. The user can then share the content of the selection area to the service corresponding to the service icon, thereby avoiding the misselection of data content that is not intended by the user, achieving accurate transmission of the content selected by the user, and improving the user experience.

[0073] According to the first aspect, or any implementation of the first aspect above, the method further includes: determining a first rectangle based on the operation trajectory, and using the first rectangle as a fourth selection area. The above technical solution uses the first rectangle corresponding to the operation trajectory as the fourth selection area, ensuring that the content displayed in the fourth selection area is substantially the same as the content selected by the user in the operation trajectory.

[0074] According to the first aspect, or any implementation of the first aspect above, the method further includes: obtaining the intersection content in the ninth interface that intersects with the boundary of the fourth selection area; and determining the fifth selection area based on the intersection content. The above technical solution can determine the fifth selection area based on the intersection content with the fourth selection area, thus accurately displaying the area where the content the user intends to select is located.

[0075] According to the first aspect, or any implementation thereof, determining the fifth selection area based on intersecting content includes: determining the start and end positions of the intersecting content; determining the first text area corresponding to the start position and the second text area corresponding to the end position; determining a second rectangle based on the first and second text areas, and using the second rectangle as the fifth selection area. The above technical solution can determine the second rectangle based on the text areas at the start and end positions of the fourth selection area and use it as the fifth selection area, thus accurately displaying the area where the content the user intends to select is located.

[0076] According to the first aspect, or any implementation of the first aspect above, the ninth interface includes text, and the intersecting content includes lines of text. When the fourth selection area is within the area containing the text, the fifth selection area includes the entire content of the lines of text. In the above technical solution, the fifth selection area can include the entire content of the intersecting lines of text, thus enabling the transmission of the entire content of the lines of text intersecting with the fourth selection area.

[0077] According to the first aspect, or any implementation of the first aspect above, the ninth interface includes a map, and the intersecting content includes at least one address entity of the map. When the fourth selection area is located within the map area, the fifth selection area includes at least one address entity. In the above technical solution, the fifth selection area can include the content of the intersecting address entities, thus enabling the transmission of the content of the address entities intersecting with the fourth selection area.

[0078] According to the first aspect, or any implementation of the first aspect above, determining the first text region corresponding to the starting position includes: taking the text region closest to the starting position among the intersecting content that intersects with the boundary of the fourth selection region as the first text region corresponding to the starting position, where the starting position is the position corresponding to the first character closest to the first vertex of the fourth selection region among the intersecting content, and the first vertex is the upper left vertex of the fourth selection region. The above technical solution can determine the first text region based on the character closest to the upper left vertex of the fourth selection region, thus improving the efficiency of recognizing the first text region.

[0079] According to the first aspect, or any implementation of the first aspect above, determining the second text region corresponding to the end position includes: taking the text region closest to the end position among the intersecting content that intersects with the boundary of the fourth selection region as the second text region corresponding to the end position, where the end position is the position corresponding to the second character closest to the second vertex of the fourth selection region among the intersecting content, and the second vertex is the lower right vertex of the fourth selection region. The above technical solution can determine the second text region based on the character closest to the lower right vertex of the fourth selection region, thus improving the efficiency of recognizing the second text region.

[0080] According to the first aspect, or any implementation of the first aspect above, the method further includes: using the starting position of the fourth selection area as the starting position of the fifth selection area, and displaying a first marker at the starting position of the fifth selection area; using the ending position of the fourth selection area as the ending position of the fifth selection area, and displaying a second marker at the ending position of the fifth selection area; and setting the text within the range of the first and second markers in the fifth selection area to a selected state. The above technical solution, by setting the text within the range of the first and second markers in the fifth selection area to a selected state, can further determine the content selected by the user's intent and further avoid accidental selection of data content.

[0081] According to the first aspect, or any implementation thereof, transmitting the content in the fifth selection area through the application and / or service corresponding to the icon selected by the user includes: transmitting the text in the fifth selection area that is in a selected state through the application and / or service corresponding to the icon selected by the user. The above technical solution can transmit the text in the fifth selection area that is in a selected state through the application and / or service corresponding to the icon selected by the user, thereby avoiding the transmission of misselected content and achieving accurate transmission of the content intended to be selected by the user.

[0082] According to the first aspect, or any implementation of the first aspect above, in response to the user's eleventh operation on the fifth selection area, transmitting the content of the fifth selection area through the application and / or service corresponding to the user-selected icon includes: responding to the user's long-press operation on the fifth selection area and dragging the content of the fifth selection area; responding to the user's operation of dragging the dragged content of the fifth selection area to the application and / or service icon and transmitting the content of the fifth selection area through the application and / or service corresponding to the application and / or service icon. The above technical solution achieves dragging the content of the fifth selection area by performing a long-press operation on the fifth selection area, and then transmits the dragged data by dragging the dragged data to the target application icon or target service icon.

[0083] According to the first aspect, or any implementation thereof, in response to the user's eleventh operation on the fifth selection area, transmitting the content of the fifth selection area through the application and / or service corresponding to the user-selected icon includes: in response to the user's touch operation on the application and / or service icon, transmitting the content of the fifth selection area through the application and / or service corresponding to the application and / or service icon. The above technical solution, by performing a touch operation on the application and / or service icon, enables the transmission of the content of the fifth selection area to the application and / or service corresponding to the target application icon or target service icon, thereby simplifying the user's operation of transmitting selected data across applications.

[0084] According to the first aspect, or any implementation of the first aspect above, the method further includes: displaying a text and image switching control on the tenth interface; if the text and image switching control is in an unselected state, responding to the user's touch operation on the text and image switching control, setting the text and image switching control to a selected state, and performing text recognition on the content within the fifth selection area; determining the application or service corresponding to the text recognition result; and displaying the determined application icon or service icon on the tenth interface. The above technical solution, by operating the text and image switching control, enables the selected content to be transmitted to the corresponding application or service in text form.

[0085] According to the first aspect, or any implementation of the first aspect above, the method further includes: if the text and image switching control is selected, responding to the user's touch operation on the text and image switching control, setting the text and image switching control to an unselected state, and taking a screenshot of the content in the fifth selection area as an image; determining the application or service corresponding to the image; and displaying the determined application icon or service icon on the tenth interface. The above technical solution, by operating the text and image switching control, can realize the transmission of selected content to the corresponding application or service via an image.

[0086] According to the first aspect, or any implementation of the first aspect above, the method further includes: displaying an editing control on the tenth interface; responding to the user's touch operation on the editing control, displaying an editing interface, the editing interface being used to edit the content within the fifth selection area. With the above technical solution, the user can edit the data selected in the fifth selection area through the editing control.

[0087] Secondly, embodiments of this application provide an electronic device. The electronic device includes: a memory and a processor, the memory and the processor being coupled; the memory stores program instructions, which, when executed by the processor, cause the electronic device to perform the methods of the first aspect or any possible implementation thereof.

[0088] Thirdly, embodiments of this application provide a computer-readable medium for storing a computer program, the computer program including instructions for performing the method in the first aspect or any possible implementation of the first aspect.

[0089] Fourthly, embodiments of this application provide a computer program including instructions for performing the method in the first aspect or any possible implementation thereof.

[0090] Fifthly, embodiments of this application provide a chip including a processing circuit and transceiver pins. The transceiver pins and the processing circuit communicate with each other via an internal connection path. The processing circuit executes the method in the first aspect or any possible implementation of the first aspect to control the receiving pin to receive signals and to control the transmitting pin to transmit signals. Attached Figure Description

[0091] Figure 1A is a schematic diagram of a user interface for an exemplary information transmission method;

[0092] Figure 1B is a schematic diagram of a user interface for another exemplary information transmission method;

[0093] Figure 2A is a user interface diagram illustrating another information transmission method by way of example;

[0094] Figure 2B is a user interface diagram illustrating another information transmission method by way of example;

[0095] Figure 3 is a schematic diagram of the hardware structure of an exemplary electronic device;

[0096] Figure 4 is a schematic diagram of the software structure of an exemplary electronic device;

[0097] Figures 5A and 5B are exemplary user interface diagrams illustrating a method of triggering an arbitrary door by activating a selection operation.

[0098] Figure 5C is an exemplary schematic diagram of a user interface for triggering an arbitrary door by opening a selection operation;

[0099] Figures 6A to 6D are schematic diagrams of a user interface for another exemplary information transmission method;

[0100] Figure 7A is a user interface diagram illustrating another example of an information transmission method;

[0101] Figure 7B is a user interface diagram illustrating yet another example of an information transmission method;

[0102] Figure 8A is an exemplary diagram illustrating the interaction of various functional modules in an information transmission method for implementing the scenario shown in Figure 6A.

[0103] Figure 8B is an exemplary diagram illustrating the interaction of various functional modules in an information transmission method for implementing the scenario shown in Figure 6B.

[0104] Figure 9 is a schematic diagram of a user transferring address data from a map application to a collection space according to an embodiment of this application;

[0105] Figure 10 is a user interface diagram of the function of opening or closing any door by setting the function entry provided by the application in one embodiment of this application;

[0106] Figure 11 is a schematic diagram showing the change of the selection area displayed on the interface according to the operation trajectory in a usage scenario where the selected content is text, according to an embodiment of this application.

[0107] Figure 12 is a schematic diagram illustrating the transmission of user-selected content in a usage scenario where the selected content is text, according to an embodiment of this application.

[0108] Figure 13 is a schematic diagram showing the change of the selected area displayed on the interface according to the operation trajectory in a local area usage scenario of a map selection application provided in an embodiment of this application.

[0109] Figure 14 is a flowchart of a data transmission method provided in an embodiment of this application. Detailed Implementation

[0110] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0111] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0112] The terms "first," "second," "third," "fourth," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0113] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0114] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0115] In the description of the embodiments in this application, unless otherwise stated, only one user interface can be displayed at a time. For example, if interface 10a is displayed, interface 20a will not be displayed on the current screen. Conversely, if interface 20a is displayed, interface 10a will not be displayed on the current screen.

[0116] To facilitate understanding, the relevant terms involved in the embodiments of this application will be introduced below.

[0117] (1) Cross-application information transmission

[0118] In this embodiment, cross-application information transmission can be understood as information transmission between two applications on the same device. That is, transmitting (sharing, uploading) information from one application to another. For example, sharing pictures / photos, videos, etc. managed in a gallery application with friends in an instant messaging application.

[0119] Furthermore, it should be noted that the information transmitted across applications can be not only images, but also text, video, documents, links, etc. Specifically, in this application's embodiments, images are used as an example.

[0120] (2) Anywhere Door

[0121] Any Door is an interactive method proposed in the embodiments of this application for transmitting information across applications.

[0122] In this interaction mode, when a user needs to transfer information from the interface across applications, the user can long-press the screen content and gently drag it to the side edges of the screen. The electronic device will then perform a three-dimensional transformation of the original interface displayed on the screen, creating a visual effect of the original interface pushing inward. The device will also display icons of services or applications recommended by the electronic device for the dragged content on the side of the screen. After that, the user can place the dragged content on the icon and release the device to transfer the content to the application or service corresponding to the icon and start the corresponding application or service function.

[0123] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. In practical applications, any interaction method that involves long-pressing and lifting content, and then transferring the lifted content to a recommended application or service, can be considered as the "anywhere door" interaction method described in this application embodiment. This application embodiment does not limit whether the user interface undergoes a three-dimensional transformation during information transmission based on this interaction method.

[0124] With the continuous development of electronic information technology, smartphones, tablets, and other smart devices have become indispensable products in our daily lives, work, and entertainment. In some usage scenarios, users use electronic devices to transfer information across applications, such as transferring images across applications. For ease of understanding, the following explanation of cross-application image transfer is provided with reference to accompanying diagrams.

[0125] Referring to Figure 1A, an exemplary process is shown where a user shares a picture from a gallery application to an instant messaging application.

[0126] The interface 10a shown in Figure 1A (1) is the user interface displayed on an electronic device (such as a mobile phone) after the user opens image 10a-1 in the gallery application. In addition to displaying image 10a-1, interface 10a may also include one or more controls (or options), such as a share control 10a-2, as well as favorites, edit controls, delete controls, etc.

[0127] For example, in some embodiments of this application, a user can click the share control 10a-2 to transfer the image 10a-1 across applications. Specifically, when a user clicks the share control 10a-2, the mobile phone responds to the user's operation and displays the interface 20a shown in Figure 1A (2).

[0128] The interface 20a shown in Figure 1A (2) is the user interface displayed on the mobile phone after the user shares the control 10a-2. The interface 20a may include one or more controls (or options), such as an icon for the user to select the application to which the sharing is to be made, or an icon for the service provided by the application.

[0129] Understandably, in some embodiments of this application, the icons displayed in interface 20a for the user to select can be icons of all applications installed on the mobile phone, as well as icons of services provided by the applications.

[0130] In some other embodiments of this application, the icons displayed in interface 20a for user selection may also be icons of default applications and / or services.

[0131] In the interface 20a shown in Figure 1A (2), the icons available for user selection include the icon 20a-1 for sending to friends, as well as icons for sending to Moments, notes applications, and saving to cloud storage. For example, in some embodiments of this application, a user can click icon 20a-1 to share image 10a-1 across applications with friends. Specifically, when a user clicks icon 20a-1, the mobile phone responds to the user's action and displays the interface 30a shown in Figure 1A (3).

[0132] It should be noted that interface 30a is an exemplary user interface for selecting chat partners in an instant messaging application. Interface 30a can provide a search box for users to search for target partners, and can also display recently forwarded messages and chat partners.

[0133] For example, in some embodiments of this application, the user can select any object provided in the interface 30a. For instance, after the user clicks on the chat object "Petter" displayed in the interface 30a, the mobile phone responds to the user's operation and can share the image 10a-1 with Petter.

[0134] This allows image 10a-1, managed in the gallery application, to be transferred to a friend Petter via an instant messaging application. In other words, it enables cross-application transfer of image 10a-1.

[0135] However, in some use cases, users may only need to share a portion of an image and not the entire image. For example, in scenarios where an image to be transferred across applications includes both text elements (such as "Zhang San" and "130***********1234" in image 10a-1) and image elements (such as a QR code in image 10a-1 (the part displaying "QR code" is a QR code image element; the specific QR code style is not shown in this case)), the included text or image elements may involve private information or information that the user does not wish to share. In this case, if a user wants to transfer a portion of the content in image 10a-1 across applications, they need to edit the image to capture that portion, save the captured content, and then select the corresponding portion of the image for cross-application transfer as shown in Figure 1A.

[0136] For ease of understanding, this application embodiment takes the example of a user wanting to transmit the QR code in image 10a-1 across applications, and provides a detailed explanation in conjunction with Figure 1B.

[0137] Referring to Figure 1B(1), for example, in a scenario where a user needs to transmit image elements such as QR codes across English in image 10a-1, the user needs to first click the editing control 10a-3 in interface 10a to access the editing interface of image 10a-1. That is, when the user clicks the editing control 10a-3, the mobile phone responds to the user's operation and displays interface 40a as shown in Figure 1B(2).

[0138] Referring to Figure 1B (2), for example, the interface 40a displays the image 10a-1 (image 40a-1 in interface 20a) that needs to be edited, as well as one or more controls (or options), such as a back control for returning to interface 10a, a save control 40a-2 for saving the edits made by the user to image 10a-1 in interface 40a, an adjustment bar for cropping image 10a-1, and various editing controls.

[0139] It should be noted that in some embodiments of this application, when the interface 20a is displayed, the cropping control and the free cropping control provided in the interface 20a can be selected by default, that is, the user interface presents the style of the interface 20a. Among them, when the free cropping control is selected, adjustment bars will be displayed around the image 10a-1 displayed in the interface 20a (such as the bars displayed on the four top corners of the image 10a-1, and the bars displayed on the four sides). In this way, by dragging the bars in various positions, the user can extract the image 40a-1' from the image 40a-1, removing the text elements and including only the image elements, as shown in the interface 50a in Figure 1B (3).

[0140] Referring to Figure 1B (3), for example, when a user edits image 40a-1, such as by cropping it, the image 40a-1 displayed on the current interface will not only be updated to the cropped image 40a-1', but the current interface will also display a "Reset" option. In this way, by operating the "Reset" option, the user can abandon the current modification and restore image 40a-1' back to image 40a-1, so that the user can re-edit it on the original image.

[0141] Referring again to Figure 1B (3), for example, after the user extracts the image elements from the image 40a-1 and obtains the image 40a-1' that they want to share, they click the save control 40a-2. The mobile phone responds to the user's operation and displays the interface 60a shown in Figure 1B (4).

[0142] Referring to Figure 1B (4), for example, an interface 60a displays a window for selecting how to save image 40a-1'. For example, this window may include a "Cancel" option, a "Overwrite Original Image" option, a "Save As" option 40a-3, etc.

[0143] Referring again to Figure 1B (4), for example, when the user clicks the "Save As" option 40a-3, the mobile phone responds to the user's operation, saves the image 40a-1' to the storage path corresponding to the gallery application, and displays the interface 10a' shown in Figure 1B (5).

[0144] It should be noted that the image 10a-4 displayed in interface 10a' is the image 40a-1' obtained through the user operations shown in Figure 1B (1) to Figure 1B (4). Thus, image 10a-4 is obtained by cropping from image 10a-1. In this case, when the user clicks the share control 10a-2 in interface 10a', the mobile phone responds to the user operation and displays interface 20a' shown in Figure 1B (6).

[0145] Referring to Figure 1B (6), for example, interface 20a' is similar to interface 20a, except that the selected image for sharing is different. Specifically, the selected image for sharing in interface 20a is image 10a-1, while the selected image for sharing in interface 20a' is image 10a-4.

[0146] Referring again to Figure 1B (6), for example, when a user clicks icon 20a-1 in interface 20a', the mobile phone responds to the user's operation and displays the aforementioned interface 30a. In this way, the user can share image elements in image 10a-1, such as image 10a-4, with the selected chat object displayed in interface 30.

[0147] As described in Figures 1A and 1B, the cross-application transmission method described above can only transmit the entire image, such as image 10a-1. If a user wants to transmit a specific area of ​​image 10a-1 (such as a QR code), complex and cumbersome operations are required. For example, image 10a-1 needs to be edited, and the specific area to be transmitted needs to be saved as a separate image, such as image 10a-4. Only by transmitting image 10a-4 can the transmission of the specific area of ​​image 10a-1 (such as the QR code) be achieved.

[0148] In addition, this cross-application transmission method requires users to interact with their mobile phones multiple times. Frequent interface switching not only reduces the efficiency of users in processing data, but also brings a poor user experience.

[0149] For ease of distinction, the cross-application transmission methods shown in Figure 1A and Figure 1B will be referred to as Implementation Method 1.

[0150] In view of this, this application provides a human-computer interaction method for any door. Specifically, in this embodiment, the user long-presses the information to be shared / transmitted, as shown in the image 10a-1 displayed in the interface 10a in Figure 2A (1). The mobile phone responds to the user's operation and displays the interface 70a shown in Figure 2A (2).

[0151] Referring to Figure 2A (2), for example, in interface 70a, image 10a-1 is processed (such as blurred or de-blurred) into image 10a-1' under the user's long press operation, and image 70a-1 is displayed on image 10a-1'.

[0152] Referring again to Figure 2A (2), for example, if the user does not release the finger, the image 70a-1 can move in the interface 70a with the user's drag operation, and the image 70a-1 can be displayed on top of the interface 70a. That is, when the image 70a-1 overlaps with the position of other elements in the interface 70a, it can cover the other elements displayed in the interface 70a.

[0153] It should be noted that the content shown in Figure 70a-1 can be the same as the content shown in Figure 10a-1.

[0154] Optionally, in some embodiments of this application, image 70a-1 may be a thumbnail obtained by scaling down image 10a-1 by a mobile phone at the same scale (or according to other scales). That is, image 70a-1 and image 10a-1 are identical except for their area size.

[0155] Optionally, in some other embodiments of this application, after image 10a-1 is long-pressed to form image 70a-1, the mobile phone can still display image 10a-1 in interface 70a, and blur or blur image 10a-1, i.e., image 10a-1' shown in interface 70a.

[0156] Optionally, in some embodiments of this application, the resolution of the image 10a-1' displayed in interface 70a may be lower than that of the image 10a-1 displayed in interface 10a. This serves as a notification to the user that image 10a-1 is selected and dragged, i.e., it is in the state of image 70a-1.

[0157] Referring again to Figure 2A (2), for example, the mobile phone can respond to a user operation on the image 70a-1, such as the drag operation shown in Figure 2A (2), and display the interface 80a shown in Figure 2A (3).

[0158] Optionally, in some embodiments of this application, when a user drags image 70a-1 in the direction of the arrow shown in interface 70a, if the distance between image 70a-1 and the right edge of the screen is less than a certain threshold, the mobile phone responds to the user's operation by launching the AnyDoor service and displaying interface 80a.

[0159] Referring to Figure 2A (3), for example, interface 80a may include display elements 10a-5, icon areas 10a-6, and images 10a-7.

[0160] It should be noted that the dotted line around icon area 10a-6 is for illustration only. That is, in actual use, the dotted line around icon area 10a-6 will not be displayed.

[0161] Furthermore, it should be noted that the dashed lines shown in other schematic diagrams of this application's embodiments are for illustrative purposes only. That is, they are not displayed in actual use.

[0162] Referring again to Figure 2A (3), the elements displayed in display elements 10a-5 can be the same as the elements displayed in interface 10a. That is, display elements 10a-5 can be understood as the interface obtained by the phone after performing a three-dimensional transformation of the elements of interface 10a. In interface 80a, display elements 10a-5 can create a visual effect on the screen that makes the original interface (i.e., interface 10a) appear to push inward.

[0163] It should be noted that, in the embodiments of this application, this "opening the door" visual effect metaphorically represents the act of transmitting information, which is more intuitive than copying and pasting (as shown in Figures 1A and 1B).

[0164] Optionally, in some embodiments of this application, in interface 80a, the background 80a-1 behind the display element 10a-5 can be set to the image obtained by overlaying a blur effect on the mobile phone's desktop background (i.e., desktop wallpaper).

[0165] Furthermore, it should be noted that in some other embodiments of this application, display elements 10a-5 may not require the door opening animation. In this implementation, icon areas 10a-6 can be directly overlaid on top of display elements 10a-5. For example, they can be displayed in the sidebar of the current user interface. Alternatively, a floating window (of any shape) can be displayed on top of the current interface, and application and / or service icons can be displayed within the floating window.

[0166] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. For ease of explanation, this application embodiment uses the processing of door opening animation effects by display elements 10a-5 as an example.

[0167] Continuing to refer to Figure 2A (3), icon areas 10a-6 can be used to display icons for one or more applications and / or services.

[0168] Understandably, the application icons displayed in icon areas 10a-6 can be icons of any application installed on the phone. These applications can be system applications or third-party applications.

[0169] The service icons displayed in icon areas 10a-6 can be icons for service functions provided by any application. For example, the icon for Moments is the icon for the Moments function provided by the WeChat application.

[0170] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0171] Furthermore, it should be noted that in some embodiments of this application, each icon in icon areas 10a-6 may also display corresponding text descriptions. Understandably, these text descriptions can be used to explain the specific application name or service name corresponding to the icon.

[0172] Furthermore, it should be understood that the text below each icon is only for the purpose of helping users understand the specific application or service function corresponding to the icon, and this application does not limit the specific icon style or the specific text content.

[0173] Furthermore, it should be noted that in some embodiments of this application, the content included in images 10a-7 may be the same as that included in images 70a-1 and 10a-1. That is, image 10a-7 may be a thumbnail obtained by shrinking image 70a-1 on a mobile phone. Similarly, image 10a-7 is identical to images 10a-1 and 70a-1 except for its size. Thus, when a user selects the icon corresponding to their desired application and / or service in the icon area 10a-6, image 10a-7 will not obscure a large area of ​​the icons and text in the icon area 10a-6, making it easier for the user to select the icon.

[0174] Furthermore, it should be noted that, unlike the reduction process of image 10a-1, after image 70a-1 is reduced to image 10a-7, image 70a-1 may no longer be displayed in interface 80a. That is, image 70a-1 can be considered as an image obtained by copying image 10a-1 and then reducing it, while image 10a-7 can be considered as an image obtained by directly reducing image 70a-1.

[0175] Referring again to Figure 2A (3), for example, in interface 80a, when a user drags image 10a-7 onto the corresponding icon in icon area 10a-6 and releases it, the mobile phone responds to the user's operation by transferring image 10a-7 to the application and / or service corresponding to the icon, and displays the user interface or feedback information corresponding to the icon of the application and / or service. For example, after the user drags image 10a-7 onto icon 10a-8 (as mentioned above, icon 20a-1) displayed in icon area 10a-6, the mobile phone responds to the user's operation by displaying the aforementioned interface 30a.

[0176] In addition, in order to guide users to accurately place the dragged information into the applications and / or services they need, when image 10a-7 is close to an icon in icon area 10a-6, or when the distance between image 10a-7 and an icon in icon area 10a-6 is less than a certain threshold, the icon that is close to image 10a-7 can display certain special effects. For example, the icon and the text corresponding to the icon can be snapped close to image 10a-7, and the icon and the text corresponding to the icon can be enlarged.

[0177] Taking icon 10a-8 and its corresponding text "Friends" in interface 80a as an example, when image 10a-7 is close to icon 10a-8, icon 10a-8 and text "Friends" are slightly to the left compared to other icons and text, that is, they are closer to image 10a-7 than other icons and text.

[0178] In addition, icon 10a-8 is enlarged to indicate that if the user releases their finger here, image 10a-7 (or image 10a-1) will be transferred to the application or service corresponding to icon 10a-8.

[0179] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0180] Therefore, based on the "Anywhere Door" cross-application transfer method, by long-pressing screen content, such as image 10a-1, and gently dragging it to the right edge (or left edge) of the screen, one can directly access quick services or applications. For example, image 10a-1 managed in the gallery application can be transferred to a friend Petter via an instant messaging application.

[0181] Furthermore, it should be noted that the cross-application transfer operation of image 10a-1 shown in Figure 2A is a continuous process from the user dragging image 10a-1 to placing image 10a-1 into the target application and / or service.

[0182] It should be noted that "continuous" here means that users do not need to perform operations such as copying and pasting, nor do they need to switch applications. The user's fingers do not need to leave the phone screen throughout the entire operation, making it simple and quick.

[0183] In addition, this information transmission method ensures an effective browsing area for the user interface, which can effectively improve the efficiency of users transmitting information across applications and enhance the user experience.

[0184] However, in some use cases, such as when a user only wants to share a specific area of ​​image 10a-1, it is still necessary to edit image 10a-1 first. For example, first crop the specific area of ​​image 10a-1, save the content of the cropped area, and then select the image corresponding to that specific area for cross-application transmission as shown in Figure 2A.

[0185] For ease of understanding, this application embodiment takes the example of a user wanting to transmit the QR code in image 10a-1 across applications, and provides a detailed explanation in conjunction with Figure 2B.

[0186] For example, when a user clicks on control 10a-3 in interface 10a, the mobile phone responds to the user's operation and displays interface 40a as shown in Figure 2B (1).

[0187] Referring to Figures 2B (1) to (4), an exemplary user interface is shown for editing image 10a-1 and extracting content to be shared (such as a QR code) from image 10a-4. Regarding the user interface corresponding to Figures 2B (1) to (4), and the user operations performed by the user corresponding to Figures 2B (1) to (4), the interface changes of the mobile phone in response to the user operations can be found in the description of Figures 1B (2) to (5) in the above embodiments, which will not be repeated here.

[0188] For example, after obtaining image 10a-4 by editing the image as shown in Figure 1B, the user can launch the Any Door service as shown in Figure 2A, and realize cross-application transmission of image 10a-4 (or image 70a-2, or image 10a-9) based on Any Door.

[0189] For details on launching the Any Door service in interface 10a' and performing cross-application transmission of image 10a-4 based on Any Door, please refer to the description of cross-application transmission of image 10a-1 in the embodiment shown in Figure 2A, which will not be repeated here.

[0190] Furthermore, it should be noted that in some other embodiments of this application, when a user performs a cropping operation on image 40a-1 displayed on interface 40a, thereby cropping image 40a-1 to image 40a-1', i.e., when interface 50a is displayed, the user may not need to save image 40a-1'. Instead, the user can directly long-press image 40a-1' to invoke the AnyDoor service and lift image 40a-1'. Specifically, image 40a-1' can be copied and then scaled down to obtain a thumbnail similar to image 70a-2. Then, by dragging this thumbnail to the icon of the target application or service displayed in the icon area, image 40a-1' can be transferred based on the target application or service.

[0191] It should be noted that the target application or target service mentioned in the embodiments of this application can be understood as the application or service that the user selects from the icon area for information transmission.

[0192] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0193] As can be seen from the descriptions of the scenarios shown in Figures 2A and 2B, when transmitting images using the cross-application transmission method of the "anywhere door," only the entire image can still be transmitted, such as image 10a-1. That is, the number of human-computer interactions can only be reduced in scenarios where the entire image is transmitted (for ease of distinction, the cross-application transmission methods shown in Figures 2A and 2B will be referred to as implementation method 2).

[0194] If a user wants to transmit a specific area of ​​image 10a-1 (such as a QR code), complex and cumbersome operations are still required on image 10a-1. For example, image 10a-1 needs to be edited, and the content of the area to be transmitted needs to be saved as a separate image, such as image 10a-4. Then, image 10a-4 needs to be transmitted using the cross-application transmission method of "AnyDoor" to achieve the transmission of the specific area of ​​content (such as the QR code) in image 10a-1. In other words, even in scenarios where only a portion of an entire image needs to be transmitted, the user still needs to interact with the phone multiple times.

[0195] Furthermore, in some use cases, the content on the current interface (such as the Douyin interface) cannot be hovered over. In such cases, to share content from the Douyin interface using implementation method 2, a screenshot of the content must first be taken, and then the screenshot must be shared using implementation method 2. In other words, in use cases where the content on the current interface cannot be hovered over, it is not possible to directly transfer the content across applications using implementation method 2.

[0196] In view of this, this application embodiment provides another human-computer interaction method for "Anywhere Door" (implementation method 3). Specifically, in this embodiment, the electronic device monitors whether the user makes a selection operation on the current interface. When the user makes a selection operation, the "Anywhere Door" service is launched, and a selection box that can be adjusted by the user is displayed on the current interface according to the selection area corresponding to the selection operation. In this way, the user can adjust the selection range by dragging the selection box, and at the same time, the icons of the application and / or service displayed in the icon area are updated according to the elements in the selection range. Finally, the user can directly access the quick service or application by gently dragging the content in the selection box to the edge of the screen.

[0197] Understandably, in the embodiments of this application, the selection operation refers to the user operation of circling specific content on the screen.

[0198] Optionally, in some embodiments of this application, the selection operation can be a user operation performed by the user using their knuckles on the screen.

[0199] Alternatively, in some other embodiments of this application, the selection operation can also be a user operation performed by the user using their finger on the screen.

[0200] Alternatively, in some other embodiments of this application, the selection operation can also be a user operation performed on the screen using an external device, such as a stylus or mouse.

[0201] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0202] Furthermore, in the embodiments of this application, the electronic device can be a device with a screen that has touch / touch control function, such as a mobile phone, tablet computer, wearable device (smartwatch), laptop computer, etc., which will not be listed here. This application does not impose any restrictions on the specific type of electronic device.

[0203] The hardware and software structures of the electronic device used to implement the implementation method 3 provided in the embodiments of this application can be shown in Figures 3 and 4.

[0204] Referring to Figure 3, an exemplary hardware structure for an electronic device used to implement Implementation 3 provided in the embodiments of this application is shown.

[0205] As shown in Figure 3, the electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0206] It should be noted that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0207] The processor 110 may include one or more processing units, such as an application processor (AP), a modem, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural network processing unit (NPU), etc., which will not be listed here and this application does not limit them.

[0208] The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0209] Different processing units can be independent devices or integrated into one or more processors.

[0210] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0211] The wireless communication function of the electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0212] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0213] The modem processor may include a modulator and a demodulator.

[0214] The mobile communication module 150 can provide wireless communication solutions for electronic devices 100, including 2-generation wireless telephone technology (2G), 3-generation mobile communication technology (3G), 4-generation mobile communication technology (4G), and 5-generation mobile communication technology (5G).

[0215] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0216] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0217] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some implementations, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. The electronic device 100 can implement display functions through a GPU, the display screens 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screens 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0218] In this embodiment, the GPU in the electronic device 100 can perform three-dimensional transformation on the content displayed on the display screen 194 after the user drags the information element in the interface, creating a visual effect of the original interface pushing inward on the screen, and displaying icons of recommended applications and / or services on the side of the screen of the electronic device 100.

[0219] Furthermore, in this embodiment, when a user drags up an information element in the interface, the processor 110 can identify the information element, including identifying the type of the information element and the specific content in the information element, and determine the corresponding application icon and / or application service icon based on the identification result, and generate corresponding program instructions. The program instructions can be used to change the display information of the display screen 194, for example, updating the interface in the display screen to a cross-application transmission interface. The icon displayed in the cross-application transmission interface is the application icon and / or application service icon determined by the processor 110 based on the identification result.

[0220] In this embodiment, the display screen 194 is used to display the user interface shown in the embodiments below. The display screen 194 can also be used to receive user operations performed on the display screen 194, such as a selection operation. The processor 110 is used to respond to the user operations detected by the electronic device 100, determine the location of various hot zones, and respond to the user operations performed on various hot zones to trigger the functions corresponding to the hot zones. The operations performed by each device in the electronic device 100 are specifically described in the relevant descriptions of the embodiments below, and will not be repeated here.

[0221] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0222] The internal memory 121 can be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as a screenshot function, a "door to anywhere" function, etc.). The data storage area may store data created during the use of the electronic device 100 (such as information transmitted via a cross-application transmission method based on a "door to anywhere" function). Furthermore, the internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0223] Specifically, in this embodiment of the application, the internal memory 121 is used to store a computer program that implements the information transmission method based on implementation mode 3, and the processor 110 is used to execute the computer program to implement the information transmission method.

[0224] In addition, it should be noted that the sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc., which will not be listed here, and this application does not limit them.

[0225] Specifically, in the embodiments of this application, the touch sensor can be used to collect data corresponding to input events generated when a user touches the screen, i.e., the display screen, so that the input event framework can determine whether the input event currently occurring on the screen is an event triggered by a preset selection operation. For details on this implementation, please refer to the description section of the following embodiments, which will not be repeated here.

[0226] Furthermore, it should be noted that an operating system runs on top of the aforementioned components. Examples include Apple's iOS operating system, Google's Android open-source operating system, and Microsoft's Windows operating system. These operating systems can employ layered architectures, event-driven architectures, microkernel architectures, microservice architectures, or cloud architectures.

[0227] For ease of explanation, this application uses the layered architecture of the Android system as an example to illustrate the software structure of the electronic device 100.

[0228] It should be noted that although the embodiments of this application are illustrated using the Android system as an example, the basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0229] Referring to Figure 4, it is a software structure block diagram of the electronic device 100 according to an embodiment of this application.

[0230] As shown in Figure 4, the layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments of this application, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0231] The application layer can include a series of application packages. As shown in Figure 4, the application packages can include applications such as settings, control center, any door, smart screenshot, and system interface.

[0232] Optionally, in some embodiments of this application, adjusting the selection range through a selection operation, and then enabling or disabling the information transmission function in a local area based on any door, can be achieved by setting a function entry provided in the application.

[0233] Optionally, in some other embodiments of this application, the selection range can be adjusted by a selection operation, thereby enabling or disabling the information transmission function in a local area based on any door. This can also be achieved through a dedicated any door application.

[0234] Referring to Figure 5A, an exemplary user interface diagram is shown, illustrating how to enable or disable a function through a function entry point provided by a settings application and a function entry point provided by an AnyDoor application.

[0235] Taking a mobile phone as an example, see Figure 5A (1), which shows an exemplary user interface of a mobile phone, such as interface 10b. Interface 10b displays icons for multiple applications, such as the icon 10b-1 for the settings application, the icon 10b-2 for the AnyDoor application, and icons for applications such as Messages (SMS application), Phone, Gallery, Camera, Clock, Calendar, Memo, Music, File Manager, Video, Calculator, Weather, Email, Voice Recorder, and Browser.

[0236] It should be noted that in some embodiments of this application, the interface 10b shown in Figure 5A (1) can be referred to as the main interface.

[0237] Referring again to Figure 5A (1), for example, when a user clicks the icon 10b-1 of the settings application, the phone responds to the user's operation (such as a click operation / touch operation), launches the settings application, and displays the settings interface corresponding to the settings application, as shown in the interface 20b in Figure 5A (2).

[0238] Referring to Figure 5A (2), for example, the interface 20b displays multiple settings function entries (or options, controls), such as the settings entry 20b-1 for the smart assistant function, and settings entries for functions such as WLAN, Bluetooth, mobile network, more connections, desktop and personalization, display and brightness, sound and vibration, notification and status bar.

[0239] It should be noted that the smart assistant mentioned in this application embodiment can be understood as an intelligent assistant that integrates multiple functions, aiming to provide users with a convenient living and working environment.

[0240] Optionally, in some embodiments of this application, the smart assistant may be referred to as "YOYO".

[0241] Referring again to Figure 5A (2), for example, when the user clicks the settings entry 20b-1 of the smart assistant function, the mobile phone responds to the user's operation and displays the settings interface corresponding to the smart assistant function, as shown in the interface 30b in Figure 5A (3).

[0242] Referring to Figure 5A (3), for example, the interface 30c displays multiple controls, such as the setting entry 30b-1 for the Any Door function, and the setting entry for functions such as YOYO Suggestions, YOYO Assistant, Smart Vision, Air Gesture, and Smart Sensing.

[0243] Referring again to Figure 5A (3), for example, when the user clicks the settings entry 30b-1 of the Any Door function, the mobile phone responds to the user's operation and displays the settings interface corresponding to the Any Door function, as shown in the interface 40b in Figure 5A (4).

[0244] Referring to Figure 5A (4), for example, the interface 40b may include a control 40b-1 corresponding to the "Any Door" function and a control 40b-2 corresponding to the "Knuckle Circle and Open" function. The function corresponding to control 40b-1 can be understood as the "Any Door" function corresponding to implementation method 2 above, and the function corresponding to control 40b-2 can be understood as the "Any Door" function corresponding to implementation method 3.

[0245] Referring again to Figure 5A (4), for example, when a user clicks control 40b-1, the mobile phone responds to the user's operation, opens the Any Door function corresponding to implementation method 2, and displays the interface 50b shown in Figure 5A (5), that is, the switch of the Any Door function corresponding to implementation method 2, which changes from the style of control 40b-1 to the style of control 40b-1'.

[0246] Understandably, in a scenario where the Any Door function corresponding to Implementation Method 2 needs to be turned off, the user clicks on control 40b-1', and the mobile phone responds to the user's operation, turns off the Any Door function corresponding to Implementation Method 2, and displays interface 40b, that is, the switch of the Any Door function corresponding to Implementation Method 2, and switches back from the style of control 40b-1' to the style of control 40b-1.

[0247] Referring again to Figure 5A (4), for example, when a user clicks control 40b-2, the mobile phone responds to the user's operation, opens the Any Door function corresponding to implementation method 3, and displays the interface 60b shown in Figure 5A (6), that is, the switch of the Any Door function corresponding to implementation method 3, which changes from the style of control 40b-2 to the style of control 40b-2'.

[0248] Understandably, in a scenario where the Any Door function corresponding to implementation method 3 needs to be turned off, the user clicks control 40b-2', and the mobile phone responds to the user's operation, turns off the Any Door function corresponding to implementation method 3, and displays interface 50b, that is, the switch of the Any Door function corresponding to implementation method 3 changes from the style of control 40b-2' back to the style of control 40b-2.

[0249] Furthermore, it should be noted that whether or not the "Any Door" function corresponding to Implementation Method 2 is enabled or disabled can be determined based on the user's actual usage needs. That is, the user can enable the "Any Door" function of Implementation Method 2 alone, or enable the "Any Door" function of Implementation Method 3 alone, or enable both Implementation Method 2 and Implementation Method 3 simultaneously, depending on their actual usage needs.

[0250] Thus, it was realized that the "Any Door" function corresponding to implementation method 2 and implementation method 3 could be enabled or disabled by setting the function entry provided by the application.

[0251] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. In some other embodiments of this application, control 30b-1 can also be directly set in interface 20b. In this way, when the user clicks on control 30b-1 in interface 20b, they can directly enter interface 40b, thereby reducing the number of human-computer interactions.

[0252] Referring again to Figure 5A (1), for example, when the user clicks the AnyDoor application icon 10b-2 displayed on the interface 10b, the mobile phone responds to the user's operation, launches the AnyDoor application, and displays the interface corresponding to the AnyDoor application, such as interface 40b, or interface 50b, or interface 60b (Figure 5A takes clicking icon 10b-2 to display interface 60b as an example), or interface 70b.

[0253] It should be noted that after the user clicks icon 10b-2, the displayed user interface can be determined based on the switch status recorded in the arbitrary door log document.

[0254] For example, in the Any Door Log document, the switch of Any Door corresponding to Implementation Method 2 is in the closed state (style of control 40b-1), and the switch of Any Door corresponding to Implementation Method 3 is in the closed state (style of control 40b-2). After the user clicks icon 10b-2, the user interface displayed is interface 40b.

[0255] For example, in the Any Door Log document, the switch of Any Door corresponding to Implementation Method 2 is in the open state (style of control 40b-1'), and the switch of Any Door corresponding to Implementation Method 3 is in the open state (style of control 40b-2'). After the user clicks icon 10b-2, the user interface displayed is interface 60b.

[0256] For example, in the Any Door Log document, the switch of Any Door corresponding to Implementation Method 2 is in the open state (style of control 40b-1'), and the switch of Any Door corresponding to Implementation Method 3 is in the closed state (style of control 40b-2). After the user clicks icon 10b-2, the user interface displayed is interface 50b.

[0257] For example, in the Any Door Log document, the switch of Any Door corresponding to Implementation Method 2 is in the closed state (style of control 40b-1), and the switch of Any Door corresponding to Implementation Method 3 is in the open state (style of control 40b-2'). After the user clicks icon 10b-2, the user interface displayed is interface 70b shown in Figure 5B.

[0258] Understandably, the "anywhere door" log file can be stored in the phone's internal memory. This internal memory can be read-only memory (ROM).

[0259] Thus, the function of any door corresponding to implementation method 2 and implementation method 3 can be enabled or disabled through the function entry provided by the dedicated Any Door application.

[0260] Optionally, in some other embodiments of this application, the selection range can be adjusted by the selection operation, and then the information transmission function in the local area can be turned on or off based on the arbitrary door. This can also be achieved through the function entry provided by the control center.

[0261] Referring to Figure 5C, an exemplary schematic diagram of a user interface for enabling or disabling a function through a function entry provided by the control center is shown.

[0262] For example, in some embodiments of this application, when the mobile phone interface is any interface after unlocking, the user slides his finger down from a preset position on the mobile phone screen, such as the top area, and the mobile phone responds to the user's operation by bringing up the control center interface, as shown in interface 10c in FIG5C (1).

[0263] Referring to Figure 5C (1), for example, the interface 10c displays multiple controls (or entry points, options), such as control 10c-1 for opening the Any Door function, as well as WLAN control, ringing control, auto-rotate control, wireless sharing control, airplane mode control, mobile data control, location information control, screenshot control, eye protection mode control, Bluetooth control, screen recording control, NFC control, flashlight control, dark mode control, do not disturb control, etc.

[0264] Referring again to Figure 5C (1), for example, in some embodiments of this application, in a scenario where the Any Door function needs to be opened, the user clicks control 10c-1, and the mobile phone responds to the user's operation by directly opening the Any Door function corresponding to implementation mode 2 and implementation mode 3, and displaying the interface 20c shown in Figure 5C (2), that is, the switch of the Any Door function changes from the style of control 10c-1 to the style of control 10c-1'.

[0265] Understandably, in scenarios where the Any Door function needs to be turned off, when a user clicks control 40b-1', the phone responds to the user's operation and will directly turn off the Any Door function corresponding to implementation methods 2 and 3, and display interface 10c. That is, the switch of the Any Door function changes from the style of control 10c-1' back to the style of control 10c-1.

[0266] Furthermore, it should be noted that in some other embodiments of this application, in scenarios where the "anywhere door" function needs to be opened, when the user clicks control 10c-1, the mobile phone responds to the user's operation and displays interface 40b, or interface 50b, or interface 60b, or interface 70b.

[0267] It should be noted that the user interface displayed after the user clicks control 10c-1 can be determined based on the opening and closing status recorded in the arbitrary door log document. For details, please refer to the description section of the above embodiment; it will not be repeated here.

[0268] For example, when a user performs an operation on interface 40b, or interface 50b, or interface 60b, or interface 70b, such as activating the "Anywhere Door" function corresponding to at least one implementation method, if the user slides their finger down from a preset position on the phone screen, such as the top area, the phone responds to the user's operation and re-brings back the control center interface. The control center interface will then be as shown in interface 20c, that is, control 10c-1 becomes control 10c-1'.

[0269] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0270] Thus, the "any door" function corresponding to implementation method 2 and implementation method 3 can be opened or closed through the function entry provided by the control center.

[0271] Furthermore, it should be noted that in some other embodiments of this application, the Any Door function can also be opened or closed by other set gestures or actions.

[0272] For example, users can use a three-finger long press on the screen to open the Any Door function corresponding to implementation method 3, and use a three-finger double tap on the screen to close the Any Door function corresponding to implementation method 3.

[0273] For example, users can long press the screen with two fingers to open the "Anywhere Door" function corresponding to implementation method 2, and double-tap the screen with two fingers to close the "Anywhere Door" function corresponding to implementation method 2.

[0274] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0275] With the "Anywhere Door" function corresponding to Implementation Method 3 enabled, when the user makes a selection operation on the screen, such as drawing a circle with their knuckle, the phone can take a screenshot through a smart screenshot application, and then display the selection trajectory on the current screen through a system interface application. For details on this implementation, please refer to the embodiment section for Figure 8A below, which will not be repeated here.

[0276] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. In some embodiments of this application, these programming interfaces and frameworks can be described as functions. As shown in Figure 4, the application framework layer may include an input event framework, a service management framework, an intent framework, an arbitrary gate framework, etc.

[0277] The input event framework can include an event hub module (EventHub), an event reader module (InputReader), an event publisher module (InputDispatcher), and a selection operation recognition module, etc.

[0278] Specifically, in the embodiments of this application, when the selection operation is a knuckle-triggered circle drawing operation, the selection operation recognition module can be a knuckle module.

[0279] EventHub is an event collection and processing service that can store input events reported by the touchscreen driver during the identification of selection operations, such as knuckle selection.

[0280] The InputReader is used to obtain Input events generated by the touchscreen from the EventHub and call the selection operation recognition module (or operation recognition module) to determine whether the current touch operation is a selection operation based on a preset algorithm. Specifically, in this embodiment, for example, the knuckle module is called to determine whether the current touch operation is a knuckle selection operation based on the knuckle recognition algorithm.

[0281] The selection operation recognition module is also used to return the recognition result to the InputDispatcher. The InputDispatcher, based on the recognition result, determines that the current touch operation is a selection operation and then calls the corresponding application, such as a smart screenshot application or a system interface application, to perform operations such as screenshotting, displaying the selection trajectory, the fourth selection area, and the fifth selection area, and provides feedback to the user. Specifically, in this embodiment, when the selection recognition module is a knuckle module, the knuckle module returns the knuckle recognition result to the InputDispatcher. The InputDispatcher, based on the knuckle recognition result, determines that the current touch operation is a knuckle selection operation and then calls the corresponding application, such as a smart screenshot application or a system interface application, to perform operations such as screenshotting, displaying the selection trajectory, the fourth selection area, and the fifth selection area, and provides feedback to the user.

[0282] In the scenario of opening the Any Door function, the input event framework is also used to pull up the Any Door framework when it is determined that a selection operation has occurred, and then the Any Door framework draws the corresponding door opening interface.

[0283] In the scenario of opening the "Anywhere Door" function, the input event framework is also used to transmit the information (elements) within the selected area (or fifth selection area) corresponding to the selected operation to the intent framework when it is determined that a selection operation has occurred. The intent framework then identifies the user intent corresponding to the information.

[0284] After identifying the user intent corresponding to the information, the intent framework can obtain the corresponding application and / or service information from the service management framework based on the user intent, and then feed back the obtained service and / or application information to the AnyDoor framework, which will then display it in the icon area of ​​the current interface.

[0285] Regarding the interaction logic of the input event framework, service management framework, intent framework, and any door framework in implementing the information transmission method provided in this application embodiment (based on implementation method 3), please refer to the following embodiment section with respect to Figure 8A, which will not be repeated here.

[0286] Furthermore, it should be noted that the frameworks described above, located at the application layer, can also be referred to as services or modules. For example, the input event framework, service management framework, intent framework, and any-door framework can be described as input event service, service management service (or service manager), intent service, and any-door service, etc.

[0287] The Android Runtime comprises the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0288] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0289] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0290] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0291] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0292] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0293] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0294] Understandably, the 2D graphics engine mentioned above is a 2D drawing engine.

[0295] The kernel layer is the layer between hardware and software, and includes various hardware drivers and hardware management. As shown in Figure 4, the kernel layer can include display drivers, sensor drivers, camera drivers, power management drivers, etc.

[0296] This concludes the description of the software structure of the electronic device 100. It is understood that the layers and components contained in each layer of the software structure shown in Figure 4 do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer layers than shown in the figures, and each layer may include more or fewer components; this application does not impose any limitations.

[0297] The following description uses a mobile phone as an example of an electronic device with the hardware structure shown in Figure 3 and the software structure shown in Figure 4, and the information transmission scheme provided by the embodiments of this application will be specifically explained in conjunction with the accompanying drawings.

[0298] For ease of explanation, this application embodiment takes the scenario where a user uses their knuckle to make a selection on the screen, pulls up the Any Door service, adjusts the selection of the content to be transmitted, and then transmits the selected content as an example to specifically explain the information transmission scheme based on implementation method 3.

[0299] For example, let's take the user interface of a mobile phone displaying images in a gallery application, as shown in the interface 10d in Figure 6A (1). When the user uses the "Anywhere Door" function corresponding to implementation method 3 shown in Figure 5A or Figure 5C to open the function, when the user makes a selection operation on the mobile phone screen with their knuckle, such as drawing a selection trajectory 10d-2 in the area where the QR code image element is located in image 10d-1 with their knuckle, the mobile phone recognizes that the user has made a selection operation and will respond to the user's operation by drawing a selection box in the area where the QR code image element is located according to the selection trajectory.

[0300] Optionally, the selection trajectory drawn by the user using their knuckles may or may not be a closed loop. This can be determined based on the knuckle recognition algorithm used, and this application does not impose any restrictions on it.

[0301] Optionally, the selection box can be drawn as a rectangle.

[0302] Optionally, in some embodiments of this application, the mobile phone responds to the selection operation by drawing a selection box based on the selection trajectory, for example, determined based on a layout analysis algorithm. Specifically, for this implementation, when it is detected that the user has triggered a selection operation on the current interface, a full-screen screenshot of the current interface can be taken first. Then, based on the layout analysis algorithm, the full-screen screenshot can be analyzed to divide it into different layout areas (such as image element areas, text paragraph areas, full-screen text information areas, etc.), and the relationship between these layout areas can be determined. Next, the coordinate information of the selection box is determined based on the selection area corresponding to the selection trajectory and the segmented layout areas. Finally, the selection box can be drawn based on the determined coordinate information.

[0303] Specifically, the coordinate information of the selection box is determined based on the selection area corresponding to the selection trajectory and the segmented page area. For example, it is determined by identifying whether the selection area corresponding to the selection trajectory is completely contained within a segmented page area and whether its proportion exceeds a set percentage, such as 50% or 60%. The specific setting can be determined according to business needs, and this application does not impose any restrictions on it.

[0304] For example, if the selected area corresponding to the selection trajectory is completely contained within a segmented page area, then the coordinate information of that page area can be determined as the coordinate information of the selection box to be drawn. In this way, the drawn selection box can completely display the elements (or content) in that page area.

[0305] For example, when the selected area corresponding to the selection trajectory is not completely contained within a single segmented page area, but intersects with multiple page areas, the proportion of the selected area corresponding to the selection trajectory in each intersecting page area can be further analyzed, as well as the effective overlap area between the selected area and the intersecting page areas (such as overlap areas with text elements or image elements). Then, the coordinate information corresponding to the largest page area (e.g., a page area composed of multiple intersecting page areas) that meets the set proportion and has an effective overlap area is determined as the coordinate information for drawing the selection box. In this way, the drawn selection box can not only completely display the content within the selected area corresponding to the selection trajectory, but also automatically select elements surrounding the selected area.

[0306] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. For ease of explanation, this application embodiment uses the determination of the selection box based on a layout analysis algorithm as an example.

[0307] Optionally, in some other embodiments of this application, the mobile phone responds to the selection operation by drawing a selection box based on the selection trajectory, for example, determined using the minimum bounding rectangle algorithm. Understandably, drawing the minimum bounding rectangle corresponding to the selection trajectory as the selection box, according to the minimum bounding rectangle algorithm, ensures that the content displayed in the selection box on the subsequent interface is substantially the same as the content in the user-selected area. In this use case, if the user is not satisfied with the content displayed in the selection box, they can manually adjust the selection box to change the selected content.

[0308] For details on the specific implementation of determining the minimum bounding rectangle based on the minimum bounding matrix algorithm, please refer to the relevant documents; this application will not provide specific details on this.

[0309] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0310] Optionally, in some other embodiments of this application, the mobile phone responds to the selection operation by drawing a selection box based on the selection trajectory, for example, by comprehensively determining it based on the minimum bounding rectangle algorithm and the layout analysis algorithm. Specifically, for this implementation, the mobile phone can first determine the selection box displayed on the interface based on the minimum bounding rectangle algorithm, and then adjust the size of the selection box according to the result of the layout analysis algorithm. Regarding the interface changes involved in this scheme, please refer to the description section of the embodiment shown in Figure 7A below, which will not be repeated here.

[0311] Optionally, in some other embodiments of this application, the mobile phone responds to the selection operation by drawing a selection box based on the selection trajectory, for example, determined by a minimum bounding rectangle algorithm, an OCR (Optical Character Recognition) algorithm, or a layout analysis algorithm. Specifically, for this implementation, the mobile phone can first determine the selection box displayed on the interface based on the minimum bounding rectangle algorithm, then perform optical character recognition on the content displayed within the minimum bounding rectangle using the OCR algorithm to determine whether there are characters obscured by the minimum bounding rectangle. Furthermore, when it is determined that there are folded characters, the size of the selection box is adjusted to the size that does not obscure the characters based on the result of the layout analysis algorithm. Regarding the interface changes involved in this scheme, please refer to the description of the embodiment shown in Figure 7B below, which will not be repeated here.

[0312] Optionally, when a selection box capable of accommodating the complete QR code image element is drawn according to the selection trajectory 10d-2, the mobile phone display screen can first display the interface 20d shown in Figure 6A (2). In the interface 20d, the selection box 20d-1, which is enclosed by a dashed frame (for illustration only, the dashed frame is not displayed in reality), displays the QR code element selected by the user.

[0313] It should be noted that, in the embodiments of this application, the selection box 20d-1 can be understood as a transition border displayed before the AnyDoor service is launched based on the selection operation. That is, the selection box 20d-1 cannot be adjusted by the user, and in order to improve the user's visual experience, a transition effect can be set for the selection box 20d-1. For example, by setting different states of the border color, width, style, and other attributes of the selection box 20d-1, the border of the selection box 20d-1 can change when the display interface 20d is shown.

[0314] In addition, it should be noted that when the user makes a selection operation, that is, when the conditions for launching the Any Door service are met, the content in the selection box 20d-1 can be displayed normally, while other areas will be blurred or blurred.

[0315] Optionally, after a first set time following the display of interface 20d, the phone's screen can display interface 30d as shown in Figure 6A (3). The selection box 30d-1 displayed in interface 30d shows the QR code element selected by the user.

[0316] It should be noted that in the embodiments of this application, the selection box 30d-1 is also a selection box drawn according to the selection trajectory 10d-1, and it cannot be adjusted by the user. Unlike the selection box 20d-1, the selection box 30d-1 does not have a transition effect, that is, during the display of the interface 30d, the border color, width, style, etc. of the selection box 30d-1 remain fixed.

[0317] Optionally, the first set time may be, for example, 1 second or several hundred milliseconds. The specific time may be determined based on the performance of the electronic device (such as the time required to activate the "Anywhere Door" service or the time required to recognize and process the content in the selected trajectory area, etc.), and this application does not impose any restrictions on this.

[0318] Optionally, during the display of interface 30d, in order to enhance the user experience, a scanning animation can be set on the selection box 30d-1. For example, a scanning effect can be simulated by setting a scan line or scan frame that can move up and down on the selection box 30d-1, making the user interface more vivid and interesting. For example, in the interface 40d shown in Figure 6A (4), a moving scan frame 30d-2 is set on the selection box 30d-1.

[0319] Optionally, in some embodiments of this application, after the AnyDoor service is launched and information about the application and / or service to be displayed in the icon area is received, the mobile phone display screen can display the interface 50d shown in Figure 6A (5). The interface 50d may include display elements 50d-1, icon area 50d-2, selection box 50d-3, text selection control 50d-4, editing control 50d-5, save control 50d-6, etc.

[0320] It should be noted that in this embodiment, the selection box 50d-3 is also a selection box drawn based on the selection trajectory 10d-1. Unlike selection boxes 20d-1 and 30d-3, selection box 50d-3 is a selection box that can be adjusted by the user. Thus, selection box 50d-3 is only displayed after the AnyDoor service is launched and receives information about the application and / or service that needs to be displayed in the icon area, ensuring that the icons of the application and / or service displayed in the icon area match the icons of the application and / or service selected in selection box 50d-3.

[0321] Referring again to Figure 6A (5), for example, the elements displayed in display element 50d-1 can be the same as the elements displayed in interface 10d. In some embodiments of this application, display element 50d-1 can be understood as the interface obtained by the mobile phone after performing a three-dimensional transformation on all elements of interface 10d. In interface 50d, display element 50d-1 can create a visual effect on the screen that makes the original interface (i.e., interface 10d) appear to push inward.

[0322] In some other embodiments of this application, display element 50d-1 can also be understood as interface 10d without 3D transformation. That is, display interface 50d does not have a door opening effect, and icon area 50d-2 is directly displayed on top of display element 50d-1.

[0323] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. For ease of explanation, this application embodiment uses the processing of the door opening animation effect by display element 50d-1 as an example.

[0324] Optionally, in some embodiments of this application, in the interface 50d, the background 50d-7 behind the display element 50d-1 can be set to the image obtained by overlaying a blur effect on the mobile phone's desktop background (i.e., desktop wallpaper).

[0325] Referring again to Figure 6A (5), for example, icon area 50d-2 can be used to display icons for one or more applications and / or services.

[0326] Optionally, in some embodiments of this application, the icons of applications and / or services displayed in icon area 50d-2 can be: icons obtained by the intent framework performing intent recognition processing based on selected elements, such as the content in an image held up by the user, and then performing service matching processing based on the recognized user intent; or icons of applications and / or services selected by the user based on the provided entry points for custom applications and / or services. That is, icon area 50d-2 will display icons of applications and / or services determined according to these two methods.

[0327] Optionally, in some embodiments of this application, the icons of applications and / or services displayed in icon area 50d-2 can be icons obtained separately by the intent framework performing intent recognition processing based on the selected elements, such as the content in an image held up by the user, and then performing service matching processing based on the recognized user intent. With this implementation, the icons of applications and / or services displayed in icon area 50d-2 can be different when the elements selected by the user are different. That is, the icons of applications and / or services displayed in icon area 50d-2 change as the elements selected by selection box 50d-3 change.

[0328] Optionally, in some embodiments of this application, the icons of the applications and / or services displayed in the icon area 50d-2 may simply be the icons of the applications and / or services selected by the user based on the provided entry point for the custom application and / or service. In this implementation, regardless of whether the elements selected by the user are the same, the same application and / or service icons (the icons of the applications and / or services selected by the user using the entry point for the custom application and / or service) will be displayed in the icon area 50d-2.

[0329] Optionally, in some embodiments of this application, the icons of applications and / or services displayed in the icon area 50d-2 can be icons of applications and / or services with a usage frequency greater than a certain threshold.

[0330] Optionally, in some embodiments of this application, the icons of applications and / or services displayed in the icon area 50d-2 can be all applications installed on the mobile phone, as well as icons of services provided by the applications.

[0331] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0332] Referring again to Figure 6A (5), for example, after the user clicks on the text selection control 50d-4, the mobile phone responds to the user's operation by triggering a text recognition operation, and then automatically adjusts the selection box 50d-3 (such as the position, size, shape, etc. of the selection box 50d-3) according to the recognized text. For a detailed description of this usage scenario, please refer to the description of Figure 6C in the following embodiments, which will not be repeated here.

[0333] Referring again to Figure 6A (5), for example, after the user clicks the editing control 50d-5, the mobile phone responds to the user's operation by displaying the editing interface, so that the user can complete the editing of the selected content in the editing interface. For details on the description of this use case, please refer to the description of Figure 6D in the following embodiments, which will not be repeated here.

[0334] Referring again to Figure 6A (5), for example, after the user clicks the save control 50d-6, the mobile phone responds to the user's operation by saving the QR code element displayed in the selection box 50d-3. In other words, the information transmission scheme provided in this application embodiment can not only select the content displayed on the current interface based on the selection operation, realize the lifting of the content, and then drag and drop the lifted content to the target application icon or the target service icon, or click the target application icon or the target service icon to transmit the lifted content across applications, but also take into account the user's need to use the knuckle selection operation to take screenshots.

[0335] Referring again to Figure 6A (5), for example, adjustment bars can be displayed at the four top corners of the selection box 50d-3, as well as the top, bottom, left, and right borders. Users can drag these bars to adjust the selection box 50d-3, thereby adjusting the selected elements.

[0336] Referring again to Figure 6A (5), for example, when a user drags the bar on the top border of the selection box 50d-3 to adjust the selection box 50d-3, after the user drags the bar on the top border of the selection box 50d-3 to select the text elements in the image 10d-1, such as "Zhang San" and "130***********1234", the user interface of the mobile phone displays the interface 60d shown in Figure 6A (6). That is, the selection box 50d-3 becomes the selection box 60d-2.

[0337] It should be noted that, in this embodiment, the selection box 60d-2 is an adjustable selection box. Thus, before sharing the content displayed in the selection box 60d-2, the user can adjust the selection box 60d-2 as needed, thereby changing the content they wish to share.

[0338] Furthermore, it should be noted that in some embodiments of this application, during the process of the user dragging (which can be understood as: recognizing down events and some move events, that is, recognizing the event of the user's finger clicking on the screen and moving on the screen) any of the Bars displayed on the selection box 50d-3, the display element 50d-1 slightly turns upright, performs a slight door closing animation, and becomes the display element 60d-1, while the icon area 50d-2 is hidden (i.e. not displayed).

[0339] Furthermore, it should be noted that in some other embodiments of this application, when the user clicks (which can be understood as: recognizing a down event, i.e. recognizing the user's mobile phone clicking on the screen) any of the Bars displayed on the selection box 50d-3, the display element 50d-1 slightly rotates to the center, performs a slight door-closing animation, and becomes the display element 60d-1, while the icon area 50d-2 is hidden (i.e. not displayed).

[0340] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0341] It should be noted that, in the embodiments of this application, both the "micro-closing door animation" and the "opening door animation" refer to the visual effect of pushing the displayed elements in the current interface inward. The difference is that the angle at which the displayed elements are pushed inward under the "opening door animation" is greater than the angle at which the displayed elements are pushed inward under the "micro-closing door animation".

[0342] In addition, it should be noted that the text selection control and editing control can also be hidden while the user is adjusting the selection box, thereby avoiding accidental activation of the text selection control or editing control when dragging the selection box bar.

[0343] For example, after the user adjusts the selection box 50d-3, such as adjusting it to the selection box 60d-2, and raises their hand (as shown in Figure 6B(1)), the phone responds to the user's operation, and the phone's user interface displays the interface 70d shown in Figure 6B(2). That is, the display element 60d-1 in the interface 60d continues to push inward, re-enacts the door opening animation, and becomes the display element 70d-1 in the interface 70d. After re-enacting the door opening animation, the icon area 70d-2 will also be displayed in the interface 70d.

[0344] Referring to Figure 6B(2), for example, as the element selected by the selection box changes, the icons of the applications and / or services displayed in the icon area can also change accordingly. For instance, when the selected element is a QR code image element, i.e., the selection box displayed in the user interface is selection box 50d-3, the icons of the applications and / or services displayed in the icon area can be the icons of the applications and / or services displayed in icon area 50d-2. When the selected elements are a QR code image element and a text element, i.e., the selection box displayed in the user interface is selection box 70d-3, the icons of the applications and / or services displayed in the icon area can be the icons of the applications and / or services displayed in icon area 70d-2.

[0345] Furthermore, it should be noted that the selection box 70d-3 displayed in interface 70d can be understood as the selection box 60d-2 in interface 60d, meaning that selection box 70d-3 is an adjustable selection box. Thus, before sharing the content displayed in selection box 70d-3, users can adjust selection box 70d-3 as needed, thereby changing the content they want to share.

[0346] In addition, it should be noted that the text selection control 70d-4, editing control 70d-5, and save control 70d-6 displayed in interface 70d have the same functions as the text selection control 50d-4, editing control 50d-5, and save control 50d-6 mentioned in the above embodiment, and will not be repeated here.

[0347] For example, in the case of display interface 70d, the user can directly click any icon in icon area 70d-2 to enable the application and / or service corresponding to the clicked icon, and transfer the element selected in selection box 70d-3 to the application and / or service.

[0348] For example, after a user clicks the icon 70d-7 of the "Friends" application in the icon area 70d-2, the phone responds to the user's operation and can display the interface 80d shown in Figure 6B (3).

[0349] Referring to Figure 6B (3), for example, interface 80d may include display element 80d-1 and floating window 80d-2.

[0350] The elements displayed in display element 80d-1 can be the same as the elements displayed in interface 10a.

[0351] The floating window 80d-2 can include a menu bar 80d-3 and a display area 80d-4.

[0352] Referring again to Figure 6B (3), for example, the menu bar 80d-3 may include a maximize control 80d-31, a minimize control 80d-32, and a close control 80d-33.

[0353] Specifically, when a user clicks the maximize control 80d-31, the phone will respond to the user's action by displaying the floating window 80d-2 in full screen, for example, filling the entire display. In this case, the maximize control 80d-31 will not be displayed in the menu bar 80d-3; instead, the restore control will be displayed.

[0354] Understandably, when a user clicks the restore control, the phone responds to the user's action, and the full-screen floating window will revert to the style displayed in interface 80d.

[0355] In some embodiments of this application, when a user clicks the minimize control 80d-32, the phone responds to the user's action by hiding the floating window 80d-2. In this case, a small floating ball can be displayed on the side edge of the current interface. This reduces occlusion of content on the current interface and allows the floating window 80d-2 to revert to its original state as displayed on interface 80d after the user clicks the floating ball.

[0356] Specifically, when a user clicks to close control 80d-33, the phone responds to the user's action by closing the floating window 80d-2.

[0357] Optionally, in some embodiments of this application, when the user clicks the close control 80d-33 and closes the floating window 80d-2, the phone's display interface can display the previous interface of the current interface, such as display interface 70d.

[0358] Optionally, in some embodiments of this application, when the user clicks the close control 80d-33 to close the floating window 80d-2, the mobile phone's display interface can display the original interface, such as interface 10a.

[0359] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0360] The display area 80d-4 shows the interface of the application or service corresponding to the icon selected by the user. For example, when the user selects icon 70d-7, the display area 80d-4 displays the interface corresponding to the "Friends" application, such as the interface for selecting a chat partner (as described in the above embodiment, interface 30a).

[0361] Referring again to Figure 6B (3), for example, when a user clicks on the chat object "Petter" displayed in display area 80d-4, the phone responds to the user's action, and the phone's user interface displays interface 90d as shown in Figure 6B (4). Interface 90d may include the send / edit interface 90d-1, as well as all elements in interface 80d.

[0362] Referring to Figure 6B (4), for example, the sending editing interface 90d-1 can be located above all the elements included in the interface 80d.

[0363] Referring to Figure 6B (4), for example, the sending editing interface 90d-1 may include the avatar and name of the chat object selected from the display area 80d-4, the image 90d-11 cropped from the area circled by the selection box 70d-3, the message box 90d-12, the cancel control, and the send control 90d-13, etc.

[0364] For example, after a user clicks the send control 90d-13, the mobile phone responds to the user's operation and can send the image 90d-11 (or the element circled in the selection box 70d-3) to the friend "Petter".

[0365] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0366] Furthermore, it should be noted that in some other embodiments of this application, when sharing the elements selected in the selection box 70d-3, the elements in the selection box 70d-3 can also be dragged and dropped to the corresponding application and / or service icon in the icon area, so that the elements in the selection box 70d-3 can be transferred across applications through the application or service corresponding to the icon.

[0367] Furthermore, it's worth noting that in real-world usage scenarios, the process from the user making a selection on the screen to launching the "Anywhere Door" service and displaying the icon area—that is, the change in the user interface from interface 10D to interface 50D—usually occurs within a short time, such as a few seconds, a few hundred milliseconds, or even tens of milliseconds. Users do not need to wait for extended periods. Moreover, during this process, the user interface sequentially presents animation effects of interface 10D, interface 20D, interface 30D, and interface 40D, making it more vivid and engaging.

[0368] Optionally, in some other embodiments of this application, when the user interface of the mobile phone displays the interface 40d shown in FIG6A (4) and the interface 40d displays the second set time, the current interface, such as interface 40d, may not display the scanning box 30d-2, and the displayed selection box 30d-1 may be changed to the selection box 50d-3.

[0369] Optionally, the second set time can be the same as or different from the first set time.

[0370] It should be noted that, in this embodiment of the application, when the second set time is displayed on interface 40d, the AnyDoor service may or may not be launched, and may or may not have received information about the application and / or service that needs to be displayed in the icon area. That is, after the second set time is displayed, regardless of whether the AnyDoor service is launched or not, and regardless of whether information about the application and / or service that needs to be displayed in the icon area is received, the selection box 50d-3 can be displayed on the user interface, while the scanning box 30d-2 and the selection box 30d-1 are not displayed. In this way, even when interface 50d is not displayed, the user can adjust the display of the selection box 50d-3 as needed. For this usage scenario, before displaying interface 50d, the mobile phone can re-obtain the matching application and / or service information based on the element selected by the adjusted selection box, and then display interface 50d after receiving the information about the application and / or service that needs to be displayed in the icon area. Understandably, in this case, the icons of the applications and / or services displayed in the icon area 50d-2 of the interface 50d are the icons of the applications and / or services that match the elements selected by the adjusted selection box.

[0371] Furthermore, it should be noted that the selection box that cannot be adjusted by the user in the embodiments of this application can be understood as the selection box whose size, shape, and position cannot be adjusted. That is, after the interface is displayed, the size, shape, and position of the selection box are fixed and cannot be changed. On the other hand, the selection box that can be adjusted by the user can be understood as the selection box whose size, shape, and position can be adjusted. That is, after the interface is displayed, the user can change the size, shape, and position of the selection box by dragging or other means.

[0372] Therefore, by using a selection operation, such as a knuckle selection to activate the "Anywhere Door" service, and displaying an adjustable selection box (e.g., selection box 50d-2) in the user interface, users can adjust the selected element by dragging the bar set on the selection box. In this way, through simple human-computer interaction, such as a single selection operation and dragging the bar, the element to be shared can be extracted from the entire image without frequent interface switching, greatly improving the user experience.

[0373] Furthermore, since the "Anywhere Door" service is launched based on the selection operation, a screenshot of the selected content is taken. Therefore, even if the content on the current screen cannot be scrolled, the selected content can still be transferred across applications using implementation method 3. Moreover, based on this implementation method, the shared content has no watermark, making it convenient for users to further process the shared content.

[0374] Furthermore, while the user is adjusting the selection box, the displayed elements on the current interface are turned upright with a slight closing animation, and the icon area is hidden. Once the user finishes adjusting the selection box and raises their hand, the displayed elements on the current interface are re-enacted with an opening animation, and the icon area, refreshed based on the element selected in the adjusted selection box, is displayed. This allows users to select the elements they want to transfer based on their actual needs, while ensuring that the icon area displays icons for more suitable applications and / or services, thus better meeting user requirements.

[0375] The following description, in conjunction with the accompanying diagram, illustrates the usage scenarios after the text selection control and editing control are triggered.

[0376] Referring to Figure 6C, an exemplary diagram illustrates the changes in the user interface after a user clicks on a text selection control.

[0377] Referring to Figure 6C(1) or Figure 6B(2), for example, when a user clicks the text selection control 70d-4, the mobile phone responds to the user's operation by performing text recognition processing and displaying the interface 100d shown in Figure 6C(2). That is, the display element 70d-1 is slightly rotated to the center, performs a slight closing animation, and becomes display element 100d-1. The icon area 70d-2 is hidden, as are the text selection control 70d-4, the edit control 70d-5, and the save control 70d-6. Furthermore, when the interface 100d is displayed, the selection box 70d-3 becomes the selection box 100d-2.

[0378] It should be noted that in the embodiments of this application, the selection box 100d-2 can have the same size, shape, and position on the mobile phone screen as the selection box 70d-3. The difference is that the selection box 100d-2 is a selection box that cannot be adjusted by the user, and in order to improve the user's visual experience, a scanning animation can be set on the selection box 100d-2. For example, in the interface 100d shown in Figure 6C (2), a moving scanning selection box 100d-3 is set on the selection box 100d-2.

[0379] Furthermore, it should be noted that after the text recognition processing is completed on the content circled in the selection box 70d-3, if a text element is recognized, the user interface of the mobile phone will display the interface 110d shown in Figure 6C (3). That is, the display element 100d-1 in the interface 100d continues to push inward, and the door opening animation is recreated, becoming the display element 110d-1 in the interface 110d. After the door opening animation is recreated, the interface 110d will also display the icon area 110d-2, the text selection control 110d-4, the editing control 110d-5, the save control 110d-6, and the selection box 110d-3 for the recognized text element.

[0380] It should be noted that, in this embodiment of the application, the selection box 110d-3 is a selection box that can be adjusted by the user.

[0381] Understandably, because the elements selected in selection box 110d-3 are different from those selected in selection box 70d-3—that is, the selected elements have changed—the icons for applications and / or services displayed in icon area 110d-2 can also differ from those displayed in icon area 70d-2. For example, if the elements selected in selection box 70d-3 include both text and image elements, the icons displayed in icon area 70d-2 may include icons for "Phone," "Messages," "Scan," "Moments," "Notes," and "Cloud Drive." If the elements selected in selection box 110d-3 only include text elements, the icons displayed in icon area 110d-2 may include icons for "Phone," "Messages," "Friends," "Notes," and "Cloud Drive."

[0382] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. In actual use, the icon displayed in the icon area can be customized by the user based on the provided entry, or it can be determined by intent recognition and service matching, or it can be determined by a combination of multiple implementation methods. These will not be elaborated further here; for specific implementation details, please refer to the description section of the above embodiments.

[0383] Therefore, by using a one-click text selection control, the selected elements can be automatically adjusted without the user having to manually adjust the selection box range. This reduces the number of human-computer interactions and enables users to fulfill their needs more quickly and accurately.

[0384] Referring to Figure 6D, an exemplary schematic diagram illustrates the changes in the user interface after an editing control is clicked by a user.

[0385] Referring to Figure 6D (1), for example, when a user clicks the edit control 50d-4, the mobile phone responds to the user's operation and displays the interface 120d shown in Figure 6D (2).

[0386] Referring to Figure 6D (2), for example, interface 120d can be understood as an editing selection box, such as the editing interface for the element selected in selection box 50d-3. Among them, selection box 120d-1 can be displayed in interface 120d.

[0387] It should be noted that in this embodiment of the application, the selection box 120d-1 is a selection box that can be adjusted by the user, and the content selected within it is the same as that in the selection box 50d-3.

[0388] Referring again to Figure 6D (2), for example, the interface 120d can also display multiple controls, such as control 120d-2, control 120d-3, control 120d-4, control 120d-5, as well as pen selection control, color selection control, line thickness selection control, sharing function control, graffiti function control, mosaic function control, eraser function control, scrolling screenshot function control, etc.

[0389] Control 120d-2 is used to trigger a back-to-back operation after the user touches it. That is, when the user clicks control 120d-2, the phone responds to the user's action and displays interface 50d.

[0390] Among them, control 120d-3 is used to undo the user's editing of the image elements in the selection box 120d-1 after the user touches it.

[0391] Among them, control 120d-4 is used to restore the user's editing of the image elements in the selection box 120d-1 after the user touches it.

[0392] Among them, control 120d-5 is used to save the user's edits to the image elements in the selection box 120d-1 after the user touches it, and save the edited image to the local machine.

[0393] For example, when the user selects a pen shape using the pen shape selection control and draws it on the image element in the selection box 120d-1, such as drawing the mark 130d-1 shown in interface 130d as shown in Figure 6D (3) on the image element in the selection box 120d-1, and then using implementation method 2, such as long-pressing to launch the Any Door service, when the Any Door service is launched, the user interface will change from interface 130d to interface 140d (as shown in Figure 6D (4)) and then to interface 150d (as shown in Figure 6D (5)). That is, the image element in the selection box 120d-1 will be copied to generate the corresponding thumbnail 140d-1, and the image element in the selection box 120d-1 will be blurred or blurred to the style of image element 140d-2.

[0394] For example, in interface 140d, as the user drags thumbnail 140d-1 toward the edge of the screen, such as along the direction of the arrow to the right edge of the screen, the phone responds to the user's operation (e.g., the drag operation shown in (4) of FIG6D) and displays interface 150d shown in (5) of FIG6D.

[0395] Optionally, in some embodiments of this application, when a user drags the thumbnail 140d-1 in the direction of the arrow shown in the interface 140d, if the distance from the right edge of the screen to the thumbnail 140d-1 is less than a certain threshold, the mobile phone responds to the user's operation by launching the AnyDoor service and displaying the interface 150d.

[0396] Referring to Figure 6D (5), for example, in the case of display interface 150d, in some embodiments of this application, the user can directly click any icon in the icon area 140-3 displayed in interface 150d, and then transmit the image element (i.e. graphic element 120d-1) corresponding to thumbnail 150d-2 through the application or service corresponding to the icon.

[0397] For example, in some other embodiments of this application, the user may also drag the thumbnail 150d-2 to the vicinity of an icon displayed in the icon area 150d-3, and then transmit the image element (i.e. the graphic element 120d-1) corresponding to the thumbnail 150d-2 through the application or service corresponding to that icon.

[0398] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0399] For details on the processing involved in the changes of the user interface from interface 130d to interface 140d and then to interface 150d, please refer to the description of implementation method 2 in Figure 2A, which will not be repeated here.

[0400] Therefore, by providing editing controls in the AnyDoor interface, such as Interface 50d, users can operate the editing controls to directly edit the elements selected by the selection box, without having to save the entire image or the selected elements separately and then find the image of the element in the gallery application for editing. This greatly reduces the number of human-computer interactions and simplifies the complexity of user operations.

[0401] In addition, for content edited in the editing interface accessed through the editing control, users can directly transmit it through the target service or application by long-pressing and dragging, making information transmission more convenient.

[0402] Furthermore, it should be noted that the information transmission scheme implemented based on implementation method 3 in this application embodiment can be applied to any scenario, that is, users can make a selection operation on any interface to select and take a screenshot of the currently displayed screen, thus making it more applicable to various user scenarios.

[0403] Referring to Figure 7A, an exemplary diagram illustrates the changes in the user interface in a usage scenario where the selected content is a text element.

[0404] Referring to Figure 7A (1), an example of a learning application interface 10e is shown. The interface 10e includes various elements (such as text elements) and controls (such as "Home" control, "Self-study" control, "Member Exclusive" control, "My" control, etc.).

[0405] Referring again to Figure 7A (1), for example, when a user uses their knuckle to make a selection operation on the text area 10e-1 in the interface 10e, the mobile phone responds to the user's operation by capturing the interface 10e (full-screen screenshot) and displaying the selection trajectory corresponding to the selection operation on the interface 10e.

[0406] For example, the interface that displays the selected trajectory is, for example, interface 20e shown in Figure 7A (2), and the displayed selected trajectory is, for example, selected trajectory 20e-1.

[0407] For example, in some embodiments of this application, the mobile phone can first determine the minimum bounding rectangle corresponding to the selected trajectory 20e-1 based on the minimum bounding rectangle algorithm, and display the determined minimum bounding rectangle on the user interface of the mobile phone, as shown in interface 30e in Figure 7A (3).

[0408] Referring to Figure 7A (3), for example, the interface 30e may include a display element 30e-1, a selection box 30e-2, and a text element 30e-3 displayed in the selection box 30e-2.

[0409] The elements displayed in display element 30e-1 may be the same as those displayed in interface 10e. In some embodiments of this application, display element 30e-1 may be a screen obtained by blurring or blurring the full-screen screenshot obtained above.

[0410] Among them, the selection box 30e-2 can be understood as the minimum bounding rectangle corresponding to the selection trajectory 20e-1 determined based on the minimum bounding rectangle algorithm.

[0411] It should be noted that, in the embodiments of this application, the selection box 30e-2 can be understood as a transition border displayed before the AnyDoor service is launched based on the selection operation. That is, the selection box 30e-2 cannot be adjusted by the user, and in order to improve the user's visual experience, a transition effect can be set for the selection box 30e-2. For example, by setting different states of the border color, width, style, and other attributes of the selection box 30e-2, the border of the selection box 30e-2 can change when the display interface 30e is displayed.

[0412] The text element 30e-3 can be understood as an image extracted from the full-screen screenshot based on the coordinates of the selection box 30e-2 within the full-screen screenshot.

[0413] Referring again to Figure 7A (3), for example, after a third set time in the display interface 30e, the user interface of the mobile phone can display the interface 40e shown in Figure 7A (4).

[0414] Optionally, the third set time may be, for example, 1 second or several hundred milliseconds. The specific time can be determined based on the performance of the electronic device (e.g., the time required to activate the "Anywhere Door" service or the time required to recognize and process the content within the selected trajectory area), and this application does not impose any limitations on this.

[0415] Referring to Figure 7A (4), for example, the interface 40e may include a display element 40e-1, a selection box 40e-2, and a text element 40e-3 displayed in the selection box 40e-2.

[0416] The elements displayed in display element 40e-1 may be the same as those displayed in interface 10e. In some embodiments of this application, display element 40e-1 may be a screen obtained by blurring or blurring the full-screen screenshot obtained above.

[0417] The selection box 40e-2 can also be understood as the minimum bounding rectangle corresponding to the selection trajectory 20e-1, determined by the minimum bounding rectangle algorithm, and it also cannot be adjusted by the user. Unlike the selection box 30e-2, the selection box 40e-2 does not have a transition effect, that is, during the display of interface 40e, the border color, width, style, etc. of the selection box 40e-2 remain unchanged.

[0418] The elements included in text element 40e-3 can be the same as those included in text element 30e-3.

[0419] Optionally, during the display of interface 40e, in order to enhance the user experience, a scanning animation can be set on the selection box 40e-2. For example, a scanning effect can be simulated by setting a scan line or scan frame that can move up and down on the selection box 40e-2, making the user interface more vivid and interesting. For example, in the interface 50e shown in Figure 7A (5), a moving scan frame 50e-1 is set on the selection box 40e-2.

[0420] It should be noted that, in the embodiments of this application, during the process of presenting interface 30e to interface 50e of the mobile phone user interface, the previously captured full-screen screenshot can be divided into different layout areas based on the layout analysis algorithm. For example, each icon control can be divided into a layout area, all icon controls in a row can be divided into a layout area, the text area 10e-2 can be divided into a layout area, the math control can be divided into a layout area, the Chinese language control can be divided into a layout area, the text area of ​​the unit ability test can be divided into a layout area, and the entire full-screen screenshot can be divided into a layout area, etc. This application does not limit this.

[0421] For example, after the page area is divided, the coordinate information that meets the requirements can be determined based on the selection trajectory 20e-1, such as the coordinate information of the page area corresponding to the text area 10e-2 containing the complete selection trajectory 20e-1.

[0422] For example, based on the layout analysis algorithm, the coordinate information of the selection box to be drawn is determined, and when the AnyDoor service is activated, the mobile phone display interface can display the interface 60e shown in Figure 7A (6). Among them, the interface 60e may include display elements 60e-1, selection box 60e-2, text elements 60e-3 displayed in selection box 60e-2, icon area 60e-4, text selection control 60e-5, editing control 60e-6, save control 60e-7, etc.

[0423] It should be noted that, in this embodiment, the selection box 60e-2 is a selection box drawn based on coordinate information determined by the layout analysis algorithm. Unlike the selection box 40e-2, the selection box 60e-2 is an adjustmentable selection box by the user. Thus, the selection box 60e-2 is only displayed after the AnyDoor service is launched and the coordinate information is determined according to the layout analysis algorithm, ensuring that the icons of the applications and / or services displayed in the icon area match the icons of the applications and / or services selected in the selection box 60e-2.

[0424] In the case that the selection box 60e-2 is drawn based on the coordinate information of the layout area corresponding to the text area 10e-2, the text element 60e-3 displayed in the selection box 60e-2 can be understood as the text element in the text area 10e-2.

[0425] Furthermore, it should be noted that in some other embodiments of this application, when the top, bottom, left, and right borders of the selection box determined based on the minimum bounding rectangle algorithm and the selection trajectory 20e-1, such as selection box 40e-2, obscure text elements, when determining the coordinate information of the selection box based on the layout analysis algorithm, such as the coordinate information corresponding to selection box 60e-2, the left and right borders of selection box 40e-2 can be moved outwards to the left and right ends of the row corresponding to the selected text element, while the top and bottom borders of selection box 40e-2 are moved outwards to a position where the obscured text element can be fully displayed. Based on this principle, when the selection box determined based on the minimum bounding rectangle algorithm and the selection trajectory 20e-1 is selection box 40e-2 displayed in interface 50e, the coordinate information corresponding to the selection box determined based on the layout analysis algorithm is, for example, the coordinate information corresponding to the dashed frame 60e-2' in interface 60e. That is, the selection box finally displayed in interface 60e is a dashed box 60e-2', not the selection box 60e-2.

[0426] Correspondingly, when the selection box in interface 60e is a dashed box 60e-2', the text elements displayed within it are the text elements within the dashed box 60e-2' in interface 60e, that is, excluding the text element "View Unit Knowledge Summary>".

[0427] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0428] Referring again to Figure 7A (6), for example, the elements displayed in display element 60e-1 can be the same as the elements displayed in interface 10e. In some embodiments of this application, display element 60e-1 can be understood as the interface obtained by the mobile phone after performing a three-dimensional transformation on all elements of interface 10e, or the full-screen screenshot captured above. In interface 60e, display element 60e-1 can create a visual effect on the screen that makes the original interface (i.e., interface 10e) appear to push inward.

[0429] It should be noted that in some other embodiments of this application, the scenario shown in Figure 7A can also be similar to the scenario shown in Figure 6A, without the need for an opening animation effect on the display element 60e-1. That is, the icon area 60e-4 can be directly displayed on top of the display element 60e-1.

[0430] The text selection control 60e-5, editing control 60e-6, and saving control 60e-7 have the same functions as the text selection control 50d-4, editing control 50d-5, and saving control 50d-6 in the above embodiments, and will not be described again here.

[0431] Therefore, in scenarios where users use the selection operation to launch the "Anywhere Door" service, the selection box in the interface displayed at different times can be determined based on the minimum bounding rectangle algorithm and the layout analysis algorithm, thereby improving the user experience.

[0432] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the sole limitation of this embodiment. In practical applications, the selection and cropping of image elements, as shown in the scenarios of Figures 6A and 6B, can also be determined based on the minimum bounding rectangle algorithm and the layout analysis algorithm in the interface displayed at different times, and this application does not impose any limitations on this.

[0433] Furthermore, for technical details not described in detail in this embodiment, please refer to the description section of the embodiments shown in Figures 6A to 6D, which will not be repeated here.

[0434] Referring to Figure 7B, an exemplary diagram illustrates the change of the selection box displayed on the interface according to the selection trajectory in a scenario where a local area in a map image is selected.

[0435] It should be noted that Figure 7B only shows the map image circled on the mobile phone interface; other elements in the interface containing this image are not shown.

[0436] Referring to Figure 7B(1), an interface 10f for displaying a map image is shown as an example. When a user makes a selection operation on the interface 10f using their knuckle, the mobile phone responds to the user's operation by capturing a full-screen screenshot of the interface 10f and displaying the selection trajectory 10f-1 corresponding to the selection operation on the interface 10f.

[0437] For example, in some embodiments of this application, the mobile phone can first determine the minimum bounding rectangle corresponding to the selected trajectory 10f-1 based on the minimum bounding rectangle algorithm, and display the determined minimum bounding rectangle on the user interface of the mobile phone, as shown in interface 20f in Figure 7B (2).

[0438] Referring to Figure 7B (2), for example, the interface 20f may include a display element 20f-1, a selection box 20f-2, and an image element 20f-3 displayed in the selection box 20f-2.

[0439] The elements displayed in display element 20f-1 may be the same as those displayed in interface 20f. In some embodiments of this application, display element 20f-1 may be a screen obtained by blurring or blurring the full-screen screenshot obtained above.

[0440] Among them, the selection box 20f-2 can be understood as the minimum bounding rectangle corresponding to the selection trajectory 20f-1 determined based on the minimum bounding rectangle algorithm.

[0441] It should be noted that, in the embodiments of this application, the selection box 20f-2 can be understood as a transition border displayed before the AnyDoor service is launched based on the selection operation. That is, the selection box 20f-2 cannot be adjusted by the user, and in order to improve the user's visual experience, a transition effect can be set for the selection box 20f-2. For example, by setting different states of the border color, width, style, and other attributes of the selection box 20f-2, the border of the selection box 20f-2 can change when the display interface 20f is shown.

[0442] Among them, image element 20f-3 can be understood as the image extracted from the full-screen screenshot based on the coordinate information of the selection box 20f-2 in the full-screen screenshot.

[0443] Referring again to Figure 7B (2), for example, after the fourth set time of the display interface 20f, the user interface of the mobile phone can display the interface 30f shown in Figure 7B (3).

[0444] Optionally, the fourth time setting may be, for example, 1 second or several hundred milliseconds. The specific time can be determined based on the performance of the electronic device (e.g., the time required to activate the "Anywhere Door" service or the time required to recognize and process the content within the selected trajectory area), and this application does not impose any limitations on this.

[0445] Referring to Figure 7B (3), for example, the interface 30f may include a display element 30f-1, a selection box 30f-2, and an image element 30f-3 displayed in the selection box 30f-2.

[0446] The elements displayed in display element 30f-1 may be the same as those displayed in interface 10f. In some embodiments of this application, display element 30f-1 may be a screen obtained by blurring or blurring the full-screen screenshot obtained above.

[0447] The selection box 30f-2 can also be understood as the minimum bounding rectangle corresponding to the selection trajectory 10f-1, determined based on the minimum bounding rectangle algorithm, and it also cannot be adjusted by the user. Unlike the selection box 20f-2, the selection box 30f-2 does not have a transition effect, that is, during the display of interface 30f, the border color, width, style, etc. of the selection box 30f-2 remain unchanged.

[0448] The elements included in image element 30f-3 may be the same as those included in image element 20f-3.

[0449] Optionally, during the display of interface 30f, in order to enhance the user experience, a scanning animation can be set on the selection box 30f-2. For example, a scanning effect can be simulated by setting a scan line or scan frame that can move up and down on the selection box 30f-2, making the user interface more vivid and interesting. For example, in the interface 30f shown in Figure 7B (3), a moving scan frame 30f-4 is set on the selection box 30f-2.

[0450] It should be noted that, in the embodiments of this application, during the process of the mobile phone's user interface presenting interface 10f to interface 30f, the smallest bounding rectangle (such as selection box 20f-2 or selection box 30f-2) displayed in the interface can be determined based on the OCR algorithm to determine whether it obscures the characters.

[0451] Referring again to Figure 7B (3), for example, there are characters in interface 30f that are folded by the border of selection box 30f-2, such as park, bus stop P4, taxi stop P5, bus stop P1, and taxi stop P3. In this case, when determining the selection box based on the layout analysis algorithm, it is necessary to follow the principle mentioned above (enlarge the selection box to a size that can just fully display the folded characters), and enlarge the selection box 30f-2 to the selection box 40f-2 in interface 40f shown in Figure 7B (4). In this way, the image element 40f-3 displayed in the selection box 40f-2 includes the text elements such as park, bus stop P4, taxi stop P5, bus stop P1, and taxi stop P3 that are just fully displayed.

[0452] In addition, it should be noted that the elements displayed in display element 40f-1 included in interface 40f can be the same as the elements displayed in interface 10f.

[0453] Therefore, in scenarios where users use the selection operation to launch the "Anywhere Door" service, the selection box displayed on the interface at different times is determined based on the minimum bounding rectangle algorithm, OCR algorithm, and layout analysis algorithm. This ensures that the selection box displayed on the interface, which can be adjusted by the user, can fully display the content selected by the user, while also avoiding the selection box being too large, thereby improving the user experience.

[0454] Furthermore, for technical details not described in detail in this embodiment, please refer to the description section of the embodiments shown in Figures 6A to 6D, which will not be repeated here.

[0455] The following section uses the scenarios shown in Figures 6A and 6B as examples to detail the interaction processing logic of each framework in the information transmission method based on implementation method 3.

[0456] Referring to Figure 8A, an information transmission method provided in an embodiment of this application is illustrated by way of example.

[0457] It should be noted that before implementing the information transmission method provided in the embodiment shown in Figure 8A, it is necessary to use the method shown in Figure 5A or Figure 5B to enable the arbitrary door function corresponding to implementation method 3.

[0458] For example, when the any door function corresponding to implementation method 3 is enabled, the information transmission method provided in the embodiment shown in Figure 8A specifically includes:

[0459] S101, the input event framework responds to user actions and identifies the currently received user action.

[0460] Specifically, in the embodiments of this application, after receiving a user operation, the input event framework performs identification processing on the received user operation, specifically identifying whether the user operation is a selection operation.

[0461] For example, in an embodiment of this application, the selection operation is, for instance, an operation triggered by the user's knuckle (knuckle selection operation). In this case, the input event frame for identifying whether the current operation is a selection operation is specifically a knuckle frame.

[0462] Specifically, the knuckle frame can identify and process the user's operation based on a knuckle recognition algorithm, thereby determining whether the currently received user operation is a knuckle selection operation. For the specific implementation logic of the knuckle frame in identifying whether the currently received user operation is a knuckle selection operation, please refer to the relevant documentation on knuckle recognition algorithms; this application's embodiments will not elaborate on this.

[0463] Referring again to Figure 8A, for example, when it is determined that the currently received user operation is a selection operation, the knuckle frame instructs the AnyDoor frame to launch the AnyDoor service (the process corresponding to the AnyDoor service can be called the AnyDoor service process). The AnyDoor frame (or AnyDoor service process) then performs the operation of drawing the AnyDoor interface, such as drawing interfaces 10d to 50d in Figure 6A, and notifies the smart screenshot application to take a full-screen screenshot of the current interface.

[0464] S102, if it is determined that the currently received user operation is a selection operation, the smart screenshot application responds to the notification of the input event framework, takes a screenshot of the current interface, and obtains a full-screen screenshot of the current interface.

[0465] S103, if it is determined that the currently received user operation is a selection operation, the AnyDoor framework responds to the notification from the input event framework, starts the AnyDoor service process, and creates the activity corresponding to the AnyDoor interface.

[0466] It should be noted that all operations within an activity are closely related to the user; it is a component responsible for interacting with the user and can display specified controls through appropriate interfaces (such as the interface for setting the content view, setContentView(View)). For details on using activities, please refer to the description section of the Android Application Programming Interface (API) documentation; this embodiment will not elaborate on this aspect.

[0467] Furthermore, it's worth noting that when launching the AnyDoor service process and creating the activity corresponding to the AnyDoor interface, external interfaces can be exposed. These include interfaces for interacting with the intent framework and the smart screenshot application. This allows the AnyDoor framework to interact with the smart screenshot application through these exposed interfaces, transmitting the selection trajectory corresponding to the selection operation, and obtaining the first screenshot image captured by the smart screenshot application based on the selection trajectory. It can also interact with the intent framework, transmitting the first screenshot image, and obtaining information about the application and / or service to be displayed in the icon area, as well as the parameters called.

[0468] S104, the Any Door framework draws the selection trajectory in the activity corresponding to the Any Door, and sends the coordinate information of the drawn selection trajectory to the smart screenshot application.

[0469] For example, in some embodiments of this application, the Any Door framework can call a system interface application to draw the selection trajectory corresponding to the selection operation in the activity corresponding to the Any Door interface based on the coordinate information corresponding to the selection operation collected by a sensor, such as a touch sensor. For example, the selection trajectory 10d-2 in interface 10d shown in Figure 6A (1). That is, when the user makes a knuckle selection operation on the mobile phone screen, the selection trajectory corresponding to the selection operation will be drawn based on the movement of the user's knuckle on the current interface, that is, the user interface will display the selection trajectory 10d-2 in interface 10d.

[0470] S105, the intelligent screenshot application determines the first coordinate information of the selection box based on the selection trajectory and preset algorithm, and captures the first screenshot image from the full-screen screenshot based on the first coordinate information, and sends the full-screen screenshot, the first coordinate information and the first screenshot image to the arbitrary door frame.

[0471] Optionally, in some embodiments of this application, the preset algorithm is, for example, the minimum bounding matrix algorithm. For this implementation, the determined first coordinate information includes a set (as shown in Figure 7A, a set of coordinate information for drawing selection boxes 30e-2 or 40e-2), and the captured first screenshot image includes one (such as the image corresponding to the text element 30e-3 displayed in selection box 30e-2, or the image corresponding to the text element 40e-3 displayed in selection box 40e-2).

[0472] Optionally, in some other embodiments of this application, the preset algorithm is, for example, a layout analysis algorithm. In this implementation, the determined first coordinate information includes a set, and the captured first screenshot image includes one image (as shown in Figure 7A, where a set of coordinate information for the circled selection box 60e-2 is drawn), and the captured first screenshot image includes one image (such as the image corresponding to the text element 60e-3 displayed in the circled selection box 60e-2).

[0473] Optionally, in some other embodiments of this application, the preset algorithm is, for example, the minimum bounding matrix algorithm and the layout analysis algorithm. For this implementation, the determined first coordinate information includes two sets: the first screenshot image includes two images (as shown in Figure 7A, a set of coordinate information for drawing selection boxes 30e-2 or 40e-2, and a set of coordinate information for drawing selection box 60e-2); the first screenshot image includes two images (such as the image corresponding to text element 30e-3 displayed in selection box 30e-2, or the image corresponding to text element 40e-3 displayed in selection box 40e-2, and the image corresponding to text element 60e-3 displayed in selection box 60e-2).

[0474] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0475] For ease of explanation, this application embodiment takes the layout analysis algorithm or the minimum bounding rectangle algorithm as an example, that is, the determined first coordinate information includes a set, and the first screenshot image includes one image.

[0476] S106, after receiving the full-screen screenshot, first coordinate information, and first screenshot image sent by the arbitrary gate frame, the intent frame analyzes and processes the first screenshot image to determine the first user intent, and sends a first information retrieval request to the service management frame according to the first user intent to obtain the matching first application and / or service information.

[0477] For example, in some embodiments of this application, the analysis and processing performed on the first screenshot image by the intent framework is, for example, analysis and processing implemented based on computer vision (CV) tag analysis technology.

[0478] Specifically, in the process of analyzing and processing the first screenshot image based on CV tag analysis technology, the regions where text elements and image elements are located in the first screenshot image can be identified based on CV tag analysis technology, and corresponding tag information can be added to the text elements and image elements respectively. Then, based on the tag information of the text elements and image elements in the first screenshot image, the intent recognition of the text elements and image elements in the first screenshot image can be performed, thereby determining the first user intent corresponding to the first screenshot image.

[0479] S107, the service management framework responds to the first information retrieval request by querying the first application and / or service information that matches the first user intent.

[0480] Understandably, the first information retrieval request may carry a first user intent determined by the intent framework based on the first screenshot image. Thus, upon receiving the first information retrieval request, the service management framework can, based on the first user intent parsed from the first information retrieval request, query the managed applications and / or services that match the first user intent. That is, the information of the applications and / or services matching the first user intent.

[0481] Understandably, in the embodiments of this application, application information includes, for example, the application's package name, version code, version name, application label, and icon. Service information includes, for example, the service source (e.g., which application it comes from) and its icon.

[0482] Referring again to Figure 8A, for example, after the service management framework finds the first application and / or service information that matches the first user's intent, it will feed back the first application and / or service information to the intent framework so that the intent framework can generate the first recommendation list.

[0483] S108, the intent framework generates a first recommendation list based on the first application and / or service information fed back by the service management framework, and feeds back the generated first recommendation list, as well as the parameter information for calling each application and / or service in the first recommendation list, to the arbitrary gate framework.

[0484] For example, in some embodiments of this application, when the intent framework generates a first recommendation list based on the first application and / or service information, it may sort the first application and / or service. For instance, the order of the icons corresponding to the applications and / or services in the first recommendation list may be adjusted according to certain sorting criteria.

[0485] Optionally, the sorting criteria can be determined based on user habits (Factor 1). For example, icons of apps and / or services that users frequently use can be moved to the top of the recommended list.

[0486] Optionally, the sorting criteria can be determined based on the frequency of use of the applications and / or services (Factor 2). For example, the icons of the most frequently used applications and / or services in the first category can be moved to a higher position in the recommendation list.

[0487] Optionally, the sorting criteria can be determined based on a user-defined whitelist of applications and / or services (Factor 3). For example, the icons of whitelisted applications and / or services in the first category of applications and / or services will be moved to a higher position in the recommended list.

[0488] Optionally, the sorting criteria can be determined based on the usage time of applications and / or services (Factor 4). For example, among the first applications and / or services, the icons of applications and / or services with longer usage times can be moved to a higher position in the recommendation list.

[0489] Optionally, the sorting criteria can be determined by combining any two or more of the factors 1 to 4 mentioned above.

[0490] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0491] S109, the arbitrary door frame draws the first selection box based on the first coordinate information, sets a transition effect for the first selection box, and displays the first screenshot image in the first selection box.

[0492] Taking a full-screen screenshot of interface 10d (i.e., interface 10a) excluding the selection trajectory 10d-2, and the first screenshot image being the QR code image element in image 10d-1, as an example, the first selection box is, for example, the selection box 20d-1 displayed in interface 20d. The specific effect is shown in interface 20d.

[0493] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0494] S110, after displaying the first ring selection box for a first set time, the arbitrary door frame draws the second ring selection box according to the first coordinate information, displays the scanning animation on the second ring selection box, and displays the first screenshot image in the second ring selection box.

[0495] The second selection box is a non-adjustable selection box, such as the selection box 30d-1 displayed in interface 30d.

[0496] The second selection box displays the interface corresponding to the scanning animation, such as interface 40d.

[0497] S112, the Any Door Frame creates an opening animation for the full-screen screenshot. Based on the first coordinate information, a third selection box is drawn on top of the full-screen screenshot. The first screenshot image is displayed in the third selection box, and the icons of each application and / or service in the first recommended list are displayed in the icon area at the edge of the screen.

[0498] The third selection box is an adjustable window, such as the selection box 50d-3 displayed in interface 50d.

[0499] The icon area is, for example, the icon area 50d-2 displayed in interface 50d.

[0500] In this context, a full-screen screenshot can be understood as the display element 50d-1 in the interface 50d.

[0501] It should be noted that in some other embodiments of this application, the arbitrary door frame may not include an opening animation for the full-screen screenshot.

[0502] Furthermore, it should be noted that steps S109 and S110 executed by the AnyGate framework are executed in parallel with steps S106 to S108 executed by the Intent framework and the Service Management framework; that is, the steps in the two processing operations do not need to be ordered. This is as long as the AnyGate framework receives the first recommendation list sent by the Intent framework and the parameters for calling each application and / or service before executing step S111.

[0503] S112, upon receiving a user's selection operation on any icon in the icon area, the service management framework starts the corresponding application or service based on the parameter information of the application or service corresponding to the selected icon provided by the arbitrary gate framework, and uses the started application or service to transmit the first screenshot image.

[0504] The visible area of ​​the icon region can display a maximum of N icons by default. For scenarios with more than N icons, users can swipe up and down the icon region to view icons corresponding to all applications and / or services included in the recommendation list.

[0505] Users can long-press on an icon in the icon area to move its display position.

[0506] For example, in some embodiments of this application, the selection operation of any icon in the icon area is, for example, directly clicking any icon displayed in the icon area, as shown in interface 70d.

[0507] For example, in some other embodiments of this application, the selection operation of any icon in the icon area, such as the user long-pressing the first screenshot icon in the third selection box, generates the drag content corresponding to the first screenshot image (such as a thumbnail of the first screenshot image) in the arbitrary door frame. After the user drags the drag content to a certain distance from the edge of the screen, the icon that is closer is determined to be the icon selected by the user, as shown in interface 80a.

[0508] Therefore, the technical solution provided in this application triggers the Any Door when the user makes a selection operation, and captures the elements within the selection trajectory corresponding to the selection operation. This allows the user to conveniently transmit the elements within the selection trajectory based on the recommended applications and / or services displayed in the icon area of ​​the current Any Door interface.

[0509] Referring to Figure 8B, another information transmission method provided in this application embodiment is illustrated.

[0510] It should be noted that before implementing the information transmission method provided in the embodiment shown in Figure 8B, it is necessary to enable the arbitrary door function corresponding to implementation method 3 using the method shown in Figure 5A or Figure 5B.

[0511] For example, when the any door function corresponding to implementation method 3 is enabled, the information transmission method provided in the embodiment shown in Figure 8B specifically includes:

[0512] S201, the Any Door frame creates an opening animation for the full-screen screenshot. Based on the first coordinate information, a third selection box is drawn on top of the full-screen screenshot. The first screenshot image is displayed in the third selection box, and the icons of each application and / or service in the first recommended list are displayed in the icon area at the edge of the screen.

[0513] Step S201 is the same as step S111 in Figure 8A. For specific implementation details, please refer to the description of step S111 above. It will not be repeated here.

[0514] S202: Upon receiving the user's adjustment operation on the third ring selection box, the Any Door frame applies a slight door-closing animation to the full-screen screenshot, hides the icon area, and adjusts the third ring selection box to the fourth ring selection box according to the adjustment operation.

[0515] The fourth selection box is also an adjustable selection box, such as the selection box 60d-2 displayed in interface 60d.

[0516] S203, after the user finishes adjusting the third selection box, the Any Door frame sends the coordinate information of the fourth selection box displayed on the current interface to the smart screenshot application, so that the smart screenshot application can capture the second screenshot image from the full-screen screenshot based on the coordinate information of the fourth selection box.

[0517] Referring again to Figure 8B, for example, after the smart screenshot application captures the second screenshot image, it can send the second screenshot image to the arbitrary gate frame through the interface exposed by the arbitrary gate frame, so that the arbitrary gate frame can send the second screenshot image to the intent frame.

[0518] In addition, it should be noted that when the adjustment operation is to reduce the size of the third selection box, that is, when the area selected by the fourth selection box is the area within the third selection box, the elements included in the second screenshot are the same as those in the first screenshot.

[0519] In addition, it should be noted that when the adjustment operation is to enlarge the third selection box, that is, the area selected by the fourth selection box is larger than the area selected by the third selection box (the area selected by the third selection box is a part of the area in the fourth selection box), the first element is the element in the full-screen screenshot.

[0520] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0521] S204, after receiving the second screenshot image, the Any Door frame displays the second screenshot image in the fourth selection box.

[0522] It should be noted that the fourth selection box is located above the full-screen screenshot. Therefore, before the second screenshot is received, the elements in the underlying full-screen screenshot can be displayed within the fourth selection box.

[0523] S205, the intent framework receives the second screenshot image sent by the arbitrary gate framework, analyzes and processes the second screenshot image to determine the second user intent, and sends a second information retrieval request to the service management framework according to the second user intent to obtain matching second application and / or service information.

[0524] S206, In response to the second information retrieval request, the service management framework queries the second application and / or service information that matches the second user's intent.

[0525] S207, the intent framework generates a second recommendation list based on the second application and / or service information fed back by the service management framework, and feeds back the generated second recommendation list, as well as the parameter information for calling each application and / or service in the second recommendation list, to the arbitrary gate framework.

[0526] It should be noted that steps S205 to S207 in this embodiment are similar to steps S106 to S108 in the embodiment shown in FIG8A. For specific implementation details, please refer to the description of steps S106 to S108 in the embodiment shown in FIG8A, which will not be repeated here.

[0527] Furthermore, it should be noted that if the elements included in the first and second screenshots are not identical (not completely the same), the determined user intent may or may not be the same. For scenarios where the determined user intent is the same, the first and second recommendation lists can be the same. For scenarios where the determined user intent is different, the first and second recommendation lists will be different. That is, during the user's adjustment of the selection box, the applications and / or services displayed in the icon area can be refreshed based on the elements actually selected by the user. This allows for the provision of more suitable applications and / or services to the user, making the information transmission function based on the "AnyDoor" feature more user-friendly and convenient.

[0528] S208, the Any Door Frame adds an opening animation to full-screen screenshots and displays the icons of each application and / or service in the second recommended list in the icon area at the edge of the screen.

[0529] It should be noted that in this embodiment of the application, when the user adjusts the size of the third selection box to the fourth selection box, the Any Door service process has already been launched before step S201. Therefore, during the execution of steps S205 to S207, it is not necessary to execute the operations of steps S109 and S110 as shown in Figure 8A.

[0530] S209, upon receiving a user's selection operation on any icon in the icon area, the service management framework starts the corresponding application or service based on the parameter information of the application or service corresponding to the selected icon provided by the arbitrary gate framework, and uses the started application or service to transmit the second screenshot image.

[0531] Therefore, the technical solution provided in this application embodiment allows users to conveniently select the information (elements) to be transmitted as needed by displaying a selection box in the Any Door interface when the user makes a selection operation to trigger Any Door. Then, the selected information can be transmitted through the corresponding application or service directly through the application and / or service displayed in the icon area (service and / or application recommendation area) of Any Door interface, which is convenient and fast.

[0532] Furthermore, by refreshing the application and / or service displayed in the icon area during the user's adjustment of the selection box, more suitable applications and / or services can be provided to the user, making the information transmission function based on AnyDoor more user-friendly and convenient.

[0533] In other application scenarios, the functions of mobile phones and other terminal devices are becoming increasingly diverse. For example, users may use terminal devices to transfer data across applications, such as transferring data selected in one application to another. However, when selecting data, it is easy to mistakenly select adjacent data, or users may need to repeatedly confirm or adjust the selected data. This not only affects the efficiency of data selection and transmission, but also causes considerable inconvenience to users, impacting the user experience.

[0534] Referring to Figure 9(1), in the scenario of starting the Any Door service, when the user selects the address data "Aaaaaaaaaaa", "Bbbbbbb", "Cccccc", "Dddddd", "Eee", "AAAAA" in the operation trajectory 10 by circling, the user will also select the data "BBBBBB" and "AAAAAAAAAA" near the data "Bbbbbb". Referring to Figure 9(3), when the user drags the selected screen content to the icon shown in Figure 9(3) to transmit the screen content to the application or service corresponding to the icon, the user will also share the data "BBBBBB" and "AAAAAAAAAA". This results in poor data transmission accuracy and affects the user experience.

[0535] In view of this, this application provides another data transmission method for "anywhere door". Specifically, in this embodiment, the terminal device monitors whether the user has performed a tenth operation on the current interface. If the user has performed a tenth operation, the "anywhere door" service is invoked, and a fourth selection area is displayed on the current interface according to the operation trajectory corresponding to the preset operation. A fifth selection area is determined based on the intersection of the fourth selection area and the text area within the fourth selection area. The application and / or service icon to be displayed on the interface is determined based on the content of the fifth selection area. In this way, the data content and application or service to be transmitted across applications can be accurately identified. Then, the user can directly access the corresponding quick service or application by dragging and dropping the content of the fifth selection area to the application and / or service icon.

[0536] Understandably, in the embodiments of this application, the tenth operation can be a selection operation or a sliding operation. The following uses a selection operation as an example to illustrate the data transmission method of this application's embodiments.

[0537] In some embodiments of this application, the selection operation refers to a user operation of circling specific content on the screen. In some embodiments of this application, the selection operation can be a user operation performed on the screen using a knuckle.

[0538] In some other embodiments of this application, the selection operation can also be a user operation performed by the user using their finger on the screen.

[0539] In other embodiments of this application, the selection operation can also be a user operation performed on the screen using an external device, such as a stylus or mouse.

[0540] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0541] In some embodiments of this application, the selection area is determined by a selection operation, and then the data transmission function within the selection area is enabled or disabled based on any door. This can be achieved by setting a function entry point provided in the application.

[0542] Referring to Figure 10, this is a schematic diagram of the user interface for opening or closing any door through a function entry provided by the application in one embodiment of this application.

[0543] Taking a mobile phone as an example, referring to (1) in Figure 10, an exemplary user interface of a mobile phone is shown, such as interface 10g. Interface 10g displays icons of multiple applications, such as the icon 10g-1 of the settings application, and icons of applications such as Messages (SMS application), Phone, Gallery, Camera, Clock, Calendar, Memo, Music, File Manager, Video, Calculator, Weather, Email, Voice Recorder, and Browser.

[0544] In some embodiments of this application, the interface 10g shown in (1) of FIG10 can be the main interface of a terminal device (e.g., a mobile phone).

[0545] Referring again to (1) in Figure 10, for example, when the user clicks the icon 10g-1 of the settings application, the phone responds to the user's operation (such as a click operation / touch operation), launches the settings application, and displays the settings interface corresponding to the settings application, as shown in interface 20g in (2) of Figure 10.

[0546] Referring to (2) in Figure 10, for example, the interface 20g displays multiple settings function entries, which may also be settings options or settings controls. For example, the settings entry 20g-1 for the smart assistant function, and settings entries for functions such as WLAN, Bluetooth, mobile network, more connections, desktop and personalization, display and brightness, sound and vibration, notification and status bar.

[0547] The smart assistant mentioned in this application can be understood as an intelligent assistant integrating multiple functions, designed to provide users with a convenient living and working environment. In some embodiments of this application, the smart assistant may be referred to as "YOYO".

[0548] Referring again to (2) in Figure 10, for example, when the user clicks the settings entry 20g-1 of the smart assistant function, the mobile phone responds to the user's operation and displays the settings interface corresponding to the smart assistant function, as shown in interface 30g in (3) of Figure 10.

[0549] Referring to (3) in Figure 10, for example, the interface 30g displays multiple controls, such as the setting entry 30g-1 for the Any Door function, and the setting entry for functions such as YOYO Suggestions, YOYO Assistant, Smart Vision, Air Gesture, and Smart Sensing.

[0550] When the user clicks the settings entry 30g-1 of the Any Door function, the mobile phone responds to the user's operation and displays the settings interface corresponding to the Any Door function, as shown in interface 40g in (4) of Figure 10.

[0551] Referring to (4) in Figure 10, the interface 40g may, exemplarily, include a control 40g-1 corresponding to the "Any Door" function and a control 40g-2 corresponding to the "Knuckle Circle to Open" function. Control 40g-1 corresponds to the function of launching the "Any Door" service by long-pressing (hereinafter referred to as implementation method 4). Control 40g-2 corresponds to the function of launching the "Any Door" service by knuckle circle (hereinafter referred to as implementation method 5). In other embodiments of this application, the method of launching the "Any Door" service corresponding to implementation method 5 may also be by knuckle circle or by swiping; this application does not limit this.

[0552] Referring to (4) in Figure 10, when the user clicks control 40g-1, the mobile phone responds to the user's operation, opens the Any Door function corresponding to implementation method 4, and displays the interface 50g shown in (5) of Figure 10, which is the switch of the Any Door function corresponding to implementation method 4, switching from the style of control 40g-1 to the style of control 40g-1'.

[0553] Understandably, in a scenario where the Any Door function corresponding to implementation method 4 needs to be turned off, the user clicks control 40g-1', and the mobile phone responds to the user's operation, turns off the Any Door function corresponding to implementation method 4, and displays interface 40g, that is, the switch of the Any Door function corresponding to implementation method 4, and switches back from the style of control 40g-1' to the style of control 40g-1.

[0554] Continuing to refer to (4) in Figure 10, if the user clicks control 40g-2, the mobile phone responds to the user's operation, opens the Any Door function corresponding to implementation method 5, and displays the interface 60g shown in (6) of Figure 10, that is, the switch of the Any Door function corresponding to implementation method 5, which changes from the style of control 40g-2 to the style of control 40g-2'.

[0555] Understandably, in a scenario where the Any Door function corresponding to implementation method 5 needs to be turned off, the user clicks control 40g-2', and the mobile phone responds to the user's operation, turns off the Any Door function corresponding to implementation method 5, and displays interface 50g, that is, the switch of the Any Door function corresponding to implementation method 5, and switches back from the style of control 40g-2' to the style of control 40g-2.

[0556] The ability to enable or disable the "Any Door" function corresponding to implementation method 4, and to enable or disable the "Any Door" function corresponding to implementation method 5, can be determined based on the user's actual needs. That is, the user can enable the "Any Door" function of implementation method 4 alone, or enable the "Any Door" function of implementation method 6 alone, or enable both implementation methods 4 and 5 simultaneously, depending on their actual needs.

[0557] When the "Anywhere Door" function corresponding to implementation method 5 is enabled, when the user makes a selection operation on the screen, such as drawing a circle with their knuckle, the phone can take a screenshot through a smart screenshot application, and the operation trajectory can be displayed on the current screen through a system interface application. For implementation details, please refer to the following embodiment section regarding Figure 14.

[0558] The following description uses a mobile phone as an example of a terminal device with the hardware structure shown in Figure 3 and the software structure shown in Figure 4, and provides a detailed explanation of the content sharing scheme provided in the embodiments of this application in conjunction with the accompanying drawings.

[0559] For ease of explanation, this application embodiment takes the scenario where a user uses their knuckle to make a selection on the screen, pulls up the Any Door service, adjusts the selection of the content to be transmitted, and then transmits the selected content as an example to specifically illustrate the content sharing scheme implemented based on implementation method 5.

[0560] Referring to Figure 11, this is a schematic diagram showing the change of the selection area displayed on the interface according to the operation trajectory in a usage scenario where the selected content is text, according to an embodiment of this application.

[0561] Referring to (1) in Figure 11, an example of a social application interface 10h is shown. The interface 10h includes text 10h-1, which includes various text areas and controls (such as “O” control, “P” control, “U” control, “I” control, etc.).

[0562] Referring again to (1) in Figure 11, for example, when a user uses their knuckle to make a selection operation on the text 10h-1 in the interface 10h, the mobile phone responds to the user's operation by capturing the interface 10h (full-screen screenshot) and displaying the operation trajectory corresponding to the selection operation on the interface 10h. The text 10h-1 includes the text areas corresponding to the data “Aaaaaaaaaaaaaa”, “Bbbbbbbb”, “Ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc”, and “Ddddddddddd”.

[0563] For example, an interface displaying the operation trajectory, such as interface 20h shown in (2) of Figure 11, displays an operation trajectory such as operation trajectory 20h-2. In some embodiments of this application, operation trajectory 20h-2 can be a closed loop.

[0564] In some embodiments of this application, the operation trajectory 20h-2 is displayed through a first preset animation effect. The first preset animation effect is a bolding and highlighting animation effect of the operation trajectory 20h-2. In other embodiments of this application, the first preset animation effect may also be a scanning animation effect of the operation trajectory 20h-2, and this application does not limit this. Referring to (2) in Figure 11, the interface 20h includes text 20h-1. When the user activates the Any Door function corresponding to implementation method 5, when the user makes a selection operation on the mobile phone screen using their knuckle, such as drawing the operation trajectory 20h-2 in the area where the text 20h-1 is located using their knuckle, the mobile phone also draws a fourth selection area in the area where the text 20h-1 is located based on the operation trajectory 20h-2.

[0565] In other embodiments of this application, when the "anywhere door" function corresponding to implementation method 5 is not enabled, when a user makes a selection operation on the mobile phone screen using their knuckle, such as drawing an operation trajectory 20h-2 in the area where the text 20h-1 is located using their knuckle, the mobile phone takes a screenshot based on the operation trajectory and saves the content of the screenshot in a preset file, such as in the album or gallery.

[0566] Referring to (3) in Figure 11, by way of example, in some embodiments of this application, the mobile phone can determine the first rectangle corresponding to the operation trajectory 20h-2 based on the rectangle algorithm, use the first rectangle as the fourth selection area 30h-2, and display the determined fourth selection area 30h-2 on the user interface of the mobile phone, as shown in interface 30h in (3) of Figure 11.

[0567] In some embodiments of this application, the interface 30h may display a sixth selection identifier, which may indicate the fourth selection area 20h-2 on the interface 30h. In some embodiments of this application, the sixth selection identifier may be a rectangle. Understandably, according to the rectangle algorithm, drawing the first rectangle corresponding to the operation trajectory 20h-2 as the fourth selection area 20h-2 can ensure that the content displayed in the fourth selection area 20h-2 is substantially the same as the content in the operation trajectory 20h-2 selected by the user. Referring to (3) in Figure 11, the interface 30h may, exemplarily, include text 30h-1, the fourth selection area 30h-2, and text area 30h-3. In some embodiments of this application, the fourth selection area 30h-2 is a highlighted area. The area in the interface 30h other than the fourth selection area 30h-2 may be a screen obtained by blurring or blurring a full-screen screenshot obtained from the interface 10h. In other embodiments of this application, the fourth selection area 30h-2 may also float above the interface 30h. The above display method can be adjusted according to actual application, and this application does not impose any restrictions.

[0568] In the embodiments of this application, the fourth selection area 30h-2 is a transition border displayed before the "Anywhere Door" service is launched based on a selection operation. That is, the fourth selection area 30h-2 cannot be adjusted by the user, and in order to improve the user's visual experience, a transition effect can be set for the fourth selection area 30h-2. For example, by setting different states of the border color, width, style, and other attributes of the fourth selection area 30h-2, the border of the fourth selection area 30h-2 can change when the display interface 30h is shown.

[0569] The text area 30h-3 includes one or more text lines.

[0570] Referring again to (3) in Figure 11, for example, after the first duration of 30h of displaying the interface, the user interface of the mobile phone can display the interface shown in (4) of Figure 11 for 40h.

[0571] In some embodiments of this application, the first duration is, for example, 1 second or several hundred milliseconds. The specific duration can be determined based on the performance of the terminal device (e.g., the time required to activate the "Anywhere Door" service or the time required to identify and process the content within the operation trajectory area), and this application does not impose any limitations on this.

[0572] In some other embodiments of this application, the interface 40h shown in (4) of FIG11 can be displayed directly after the display interface 30h.

[0573] Referring to (4) in Figure 11, the interface 40h includes text 40h-1, fourth selection area 40h-2, and text area 40h-3.

[0574] In some embodiments of this application, during the display of interface 30h, in order to improve the user experience, scanning animation effects can be set on the fourth selection area 30h-2. For example, a scanning effect can be simulated by setting a scan line or scan frame that can move up and down on the fourth selection area 30h-2, making the user interface more vivid and interesting. For example, in the interface 40h shown in Figure 11 (4), a moving scan frame 40h-4 is set on the fourth selection area 40h-2.

[0575] For example, the text area where text 40h-1 intersects with the fourth selection area 40h-2 is taken as the target text area, and the fourth selection area 40h-2 is expanded based on the target text area to obtain the fifth selection area. When the AnyDoor service is activated, the mobile phone's display interface can display the interface 50h shown in (5) of Figure 11. The interface 50h may include the fifth selection area 50h-2, the text area 50h-3 displayed in the fifth selection area 50h-2, the icon area 50h-4, the text and image switching control 50h-5, the editing control 50h-6, the save control 50h-7, etc.

[0576] In some embodiments of this application, the intersection content of the interface 40h with the boundary of the fourth selection area 40h-2 is obtained, the second rectangle is determined based on the intersection content, and the second rectangle is used as the fifth selection area 50h-2.

[0577] In some embodiments of this application, the fifth selection area 50h-2 is a highlighted display area. The content displayed in the fifth selection area 50h-2 does not undergo deformation (such as deformation caused by 3D transformation) relative to the original interface (e.g., interface 10h) after the door is opened. The area in interface 50h other than the fifth selection area 50h-2 can be a screen obtained by blurring or blurring the full-screen screenshot obtained above. The area in interface 50h other than the fifth selection area 50h-2 and the icon area 50h-4 undergoes deformation relative to the original interface 10h after the door is opened.

[0578] In some embodiments of this application, the fourth selection region 40h-2 can be expanded based on the entire content of the text lines of the intersecting content to obtain the fifth selection region 50h-2.

[0579] In some embodiments of this application, the interface 50h may display a seventh selection identifier, which may indicate the fifth selection area 50h-2 on the interface 50h. In some embodiments of this application, the seventh selection identifier may be a rectangle. For example, continuing to refer to (4) in FIG11, from all target text areas intersecting with the fourth selection area 40h-2, a first target text area at the starting position of the fourth selection area 40h-2 and a second target text area at the ending position of the fourth selection area 40h-2 are determined. Based on the bounding rectangle algorithm, a second rectangle is determined according to the first target text area and the second target text area, and this second rectangle is the fifth selection area 50h-2.

[0580] Referring again to (4) in Figure 11, for example, the target text region closest to any vertex (such as the top left corner) of the fourth selection region 40h-2 among all target text regions intersecting with the fourth selection region 40h-2 is taken as the first target text region at the starting position of the fourth selection region 40h-2; the target text region closest to any vertex (such as the bottom right corner) of the fourth selection region 40h-2 other than the starting position among all target text regions intersecting with the fourth selection region 40h-2 is taken as the second target text region at the ending position of the fourth selection region 40h-2; based on the bounding rectangle algorithm, a second rectangle including the first target text region and the second target text region is determined, and this second rectangle is taken as the fifth selection region 50h-2 displayed on the interface 50.

[0581] In some embodiments of this application, the starting position is the position corresponding to the text closest to the top-left corner of the fourth selection area 40h-2 within the intersecting content (such as the target text area) that intersects with the fourth selection area 40h-2. For example, this starting position is the line of text closest to the top-left corner within the intersecting content that intersects with the fourth selection area 40h-2, and the position corresponding to the text closest to the top-left corner within that line of text.

[0582] In some embodiments of this application, the end position is the position corresponding to the text closest to the lower right corner of the fourth selection area 40h-2 within the intersecting content that intersects with the fourth selection area 40h-2. For example, the end position is the line of text closest to the lower right corner within the intersecting content that intersects with the fourth selection area 40h-2, and the position corresponding to the text closest to the lower right corner within that line of text.

[0583] In some embodiments of this application, the starting position of the fourth selection area 40h-2 is used as the starting position of the fifth selection area 50h-2, and a first marker 50h-8 is displayed at the starting position of the fifth selection area 50h-2; the ending position of the fourth selection area 40h-2 is used as the ending position of the fifth selection area 50h-2, and a second marker 50h-9 is displayed at the ending position of the fifth selection area 50h-2; and text within the range defined by the starting and ending positions in the fifth selection area 50h-2 is set to a selected state. In these embodiments, setting the text within the range defined by the first and second markers in the fifth selection area 50h-2 to a selected state clearly indicates the text content circled by the user on the interface. In some embodiments of this application, the fifth selection area 50h-2 is displayed using a second preset animation effect. The second preset animation effect can be an animation effect that switches the display of the fourth selection area 40h-2 to the fifth selection area 50h-2 using a Matrus mapping animation. In this embodiment of the application, the use of a second preset animation to display the fifth selection area 50h-2 can avoid the display jump problem that occurs during the switching from the fourth selection area 40h-2 to the fifth selection area 50h-2, thus ensuring the smooth display of the fifth selection area 50h-2.

[0584] Referring again to (5) in Figure 11, for example, icon area 50h-4 can be used to display one or more application and / or service icons.

[0585] In some embodiments of this application, the application and / or service icons displayed in icon area 50h-4 can be obtained by performing intent recognition processing on the data selected by the intent framework in the fifth selection area 50h-2, such as on text within the fifth selection area 50h-2 or text in a selected state within the fifth selection area 50h-2, and then performing service matching processing based on the recognized user intent. Where the data selected in the fifth selection area 50h-2 is different, the application and / or service icons displayed in icon area 50h-4 can be different. The application and / or service icons displayed in icon area 50h-4 change as the data selected in the fifth selection area 50h-2 changes.

[0586] In some embodiments of this application, users can switch the display of application and / or service icons displayed in the icon area 50h-4 by performing swiping operations, such as swiping up or swiping down.

[0587] In some embodiments of this application, in response to the user's eleventh operation on the fifth selection area 50h-2, the data in the fifth selection area 50h-2 is dragged up, and the dragged data in the fifth selection area 50h-2 is transmitted through the application and / or service corresponding to the triggered icon. In some embodiments of this application, dragging up the data in the fifth selection area 50h-2 means displaying the data in the fifth selection area 50h-2 in a floating manner, for example, floating it on the interface 50h.

[0588] For example, referring to (5) in Figure 11, in response to a user's long-press operation on the fifth selection area 50h-2, the data in the fifth selection area 50h-2 is dragged up and the dragged content is displayed in the floating frame, and the floating frame is displayed in the user interface, as shown in interface 60h in Figure 11 (6). For example, interface 60h includes the fifth selection area 60h-2, the floating frame 60h-3, and the icon area 60h-4. The floating frame 60h-3 can adjust its display position in real time following the user's drag operation on the screen. Referring to (6) in Figure 11, after displaying the floating frame 60h-3, in response to the user's operation of dragging the floating frame 60h-3 to an application and / or service icon in the icon area 60h-4, the application and / or service corresponding to the application and / or service icon is triggered to transmit the data in the floating frame 60h-3.

[0589] In this embodiment of the application, by long-pressing the fifth selection area, the content of the fifth selection area can be dragged up, and then the dragged data can be transferred by dragging and dropping it to the target application icon or the target service icon.

[0590] In some other embodiments of this application, referring to FIG12, when the display interface 50 is in the case of a user touch operation on an application and / or service icon in the icon area 50h-4, the application and / or service corresponding to the application and / or service icon is triggered to transmit data in the fifth selection area 50h-2, such as transmitting text that is selected in the fifth selection area 50h-2.

[0591] This application embodiment enables the transfer of content from the fifth selection area to the application and / or service corresponding to the target application icon or target service icon through touch operation on the application and / or service graph, thereby simplifying the user's operation of transferring selected data across applications.

[0592] Referring again to (5) in Figure 11, for example, if the text and image switching control 50h-5 is in an unselected state and is clicked by the user, the mobile phone responds to the user's operation by displaying the text and image switching control 50h-5 as selected, performing text recognition on the data in the fifth selection area 50h-2, and automatically displaying the matching application and / or service icon in the icon area 50h-4 based on the recognized text recognition result. If the text and image switching control 50h-5 is in a selected state and is clicked by the user again, the mobile phone responds to the user's operation by displaying the text and image switching control 50h-5 as unselected, and taking a screenshot of the data in the fifth selection area 50h-2 as an image, and then automatically displaying the matching application and / or service icon in the icon area 50h-4 based on the image. In this embodiment, the default state of the text and image switching control 50h-5 is the selected state.

[0593] Referring again to (5) in Figure 11, for example, after the user clicks the editing control 50h-6, the mobile phone responds to the user's operation and displays the editing interface so that the user can complete the editing of the data circled in the fifth selection area 50h-2 in the editing interface.

[0594] Referring again to (5) in Figure 11, for example, after the user clicks the save control 50h-7, the mobile phone responds to the user's operation by taking a screenshot of the data in the fifth selection area 50h-2 and saving the image in a preset file, such as an album or gallery. In other words, the information transmission scheme provided in this application embodiment can not only transmit the selected content across applications based on the selection operation, but also take into account the user's need to take screenshots by using the knuckle selection operation.

[0595] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0596] Referring again to (5) in Figure 11, exemplarily, interface 50h can be the interface obtained by the mobile phone after performing a three-dimensional transformation on all elements of interface 10h, or the aforementioned full-screen screenshot. In some embodiments of this application, interface 50h can display a visual effect of the original interface (i.e., interface 10h) being pushed inward on the screen. In other embodiments of this application, interface 50h can display a visual effect of the original interface undergoing transparency changes or scaling on the screen. This application does not limit the visual effect displayed by interface 50h.

[0597] It should be understood that in the embodiments of this application, the interface 50h shown in FIG11 (5) and FIG12, and the interface 60h shown in FIG12, display a status bar, such as displaying information such as time, network connection status, signal, and battery level. In other embodiments of this application, the interface 50h and interface 60h do not include a status bar.

[0598] It should be understood that interfaces 10h, 20h, 30h, and 40h are the same interface (such as interface nine) displayed at different points in time. Interfaces 50h and 60h are the same interface (such as interface ten) displayed at different points in time.

[0599] It should be understood that, in the embodiments of this application, the dashed lines shown in interface 50h and interface 60h are only for marking the icon display area, and there are no dashed lines in the actual interface displayed on the terminal device.

[0600] In this application embodiment, in a scenario where a user uses a selection operation to launch the Any Door service, the selection area corresponding to the selection operation in the interface is determined based on the bounding rectangle algorithm. Then, a matching service icon is displayed based on the content of the selection area. Through user operation (such as the eleventh operation mentioned above), the content of the selection area is shared to the service corresponding to the service icon, thereby avoiding the misselection of data content that is not intended by the user, realizing the accurate transmission of the content selected by the user's intention, and improving the user experience.

[0601] Referring to Figure 13, this is a schematic diagram showing the change of the selected area displayed on the interface according to the operation trajectory in a local area usage scenario of a map selection application provided in an embodiment of this application.

[0602] Referring to (1) in Figure 13, an exemplary interface 10i for displaying a map application is shown. When a user makes a selection operation on interface 10i using their knuckle, the mobile phone responds to the user's operation by capturing a full-screen screenshot of interface 10i and displaying the corresponding operation trajectory 10i-2 on interface 10i. In some embodiments of this application, the operation trajectory can be a closed loop.

[0603] In some embodiments of this application, the operation trajectory 10i-2 is displayed using a first preset animation effect. The first preset animation effect is a highlighting effect that bolds the operation trajectory 10i-2. In other embodiments of this application, the first preset animation effect may also be a scanning effect of the operation trajectory 10i-2, and this application does not limit this to that.

[0604] For example, in some embodiments of this application, the mobile phone can determine the rectangle corresponding to the operation trajectory 10i-2 as the fourth selection area based on the circumscribed rectangle algorithm, and display the determined rectangle on the user interface of the mobile phone, as shown in interface 20i in (2) of FIG13.

[0605] Referring to (2) in Figure 13, exemplarily, interface 20i may include a fourth selection area 20i-2 and a text area 20i-3. In some embodiments of this application, the text area of ​​the map application may be an address entity.

[0606] In some embodiments of this application, the fourth selection area 20i-2 is a highlighted area, and the area in interface 20i other than the fourth selection area 20i-2 can be a screen obtained by blurring or blurring the full-screen screenshot obtained above. In other embodiments of this application, the fourth selection area 20i-2 can also float above interface 20i. The above display methods can be adjusted according to actual applications, and this application does not impose any limitations.

[0607] Among them, the fourth selection region 20i-2 can be the rectangle corresponding to the operation trajectory 10i-2 determined based on the circumscribed rectangle algorithm.

[0608] It should be noted that, in the embodiments of this application, the fourth selection area 20i-2 can be a transition border displayed before the Any Door service is launched based on the selection operation. To improve the user experience, a transition effect can be set for the fourth selection area 20i-2. For example, by setting different states of the border color, width, style, and other attributes of the fourth selection area 20i-2, the border of the fourth selection area 20i-2 can change when the display interface 20i is displayed.

[0609] The text area 20i-3 may include one or more text lines.

[0610] Referring again to (2) in Figure 13, by way of example, after a second duration of displaying interface 20i, the user interface of the mobile phone may display interface 30i as shown in (3) in Figure 13.

[0611] In some embodiments of this application, the second duration is, for example, 1 second or several hundred milliseconds. The specific duration can be determined based on the performance of the terminal device (e.g., the time required to activate the "Anywhere Door" service or the time required to identify and process the content within the operation trajectory area), and this application does not impose any limitations on this.

[0612] Referring to (3) in Figure 13, for example, the interface 30i may include a second area 30i-2 and a text area 30i-3.

[0613] The fourth selection region 30i-2 can also be the rectangle corresponding to the operation trajectory 10i-2 determined based on the circumscribed rectangle algorithm, and it cannot be adjusted by the user.

[0614] In some embodiments of this application, during the display of interface 30i, in order to enhance the user experience, scanning animation effects can be set on the fourth selection area 30i-2. For example, a scanning effect can be simulated by setting a scan line or scan frame that can move up and down on the fourth selection area 30i-2, making the user interface more vivid and interesting. For example, in the interface 30i shown in (3) of Figure 13, a moving scan frame 30i-4 is set on the fourth selection area 30i-2.

[0615] For example, the text area 30i-3 that intersects with the fourth selection area 30i-2 in the interface 30i is taken as the target text area, and the fourth selection area 30i-2 is expanded based on the target text area to obtain the fifth selection area. When the AnyDoor service is activated, the mobile phone display interface can display the interface 40i shown in (4) of Figure 13. The interface 40i may include the fifth selection area 40i-2, the text area 40i-3, the icon area 40i-4, the text and image switching control 40i-5, the editing control 40i-6, the save control 40i-7, etc.

[0616] In some embodiments of this application, the intersection content of the interface 30i with the boundary of the fourth selection area 30i-2 is obtained, the second rectangle is determined according to the intersection content, and the second rectangle is used as the fifth selection area 40i-2.

[0617] In some embodiments of this application, the fifth selection area 40i-2 is a highlighted display area. The content displayed in the fifth selection area 40i-2 does not undergo deformation relative to the original interface (such as interface 10i) after any door is opened (e.g., deformation caused by 3D transformation). The area in interface 40i other than the fifth selection area 40i-2 can be a screen obtained by blurring or blurring the full-screen screenshot obtained above. The area in interface 40i other than the fifth selection area 40i-2 and the icon area 40i-4 undergoes deformation relative to the original interface 10i after any door is opened.

[0618] In some embodiments of this application, the fourth selection region 30i-2 can be expanded based on the content of the address entities in the intersecting content to obtain the fifth selection region 40i-2.

[0619] In some embodiments of this application, interface 40i may display a seventh selection identifier, which may indicate the fifth selection area 40i-2 on interface 40i. In some embodiments of this application, the seventh selection identifier may be a rectangle.

[0620] For example, referring to (3) in Figure 13, a first target text region at the starting position of the fourth selection region 30i-2 and a second target text region at the ending position of the fourth selection region 30i-2 are determined from all target text regions intersecting with the fourth selection region 30i-2. Based on the bounding rectangle algorithm, a second rectangle is determined according to the first target text region and the second target text region, and this second rectangle is the fifth selection region 40i-2.

[0621] Referring again to (3) in Figure 13, for example, the target text region closest to the upper left corner of the fourth selection region 30i-2 among all target text regions intersecting with the fourth selection region 30i-2 (such as the data corresponding to the data “Dddddd”) is taken as the first target text region at the starting position of the fourth selection region 30i-2; the target text region closest to the lower right corner of the fourth selection region among all target text regions intersecting with the fourth selection region 30i-2 (such as the data corresponding to the data “AAAAA”) is taken as the second target text region at the ending position of the fourth selection region 30i-2; based on the bounding rectangle algorithm, a second rectangle including the first target text region and the second target text region is determined, and this second rectangle is taken as the fifth selection region 40i-2.

[0622] In some embodiments of this application, the starting position is the position corresponding to the text closest to the top-left corner of the fourth selection area 30i-2 within the intersecting content (such as the target text area) that intersects with the fourth selection area 30i-2. This starting position is the line of text closest to the top-left corner within the intersecting content that intersects with the fourth selection area 30i-2, and the position corresponding to the text closest to the top-left corner within that line of text.

[0623] In some embodiments of this application, the end position is the position corresponding to the text closest to the lower right corner of the fourth selection area 30i-2 within the intersecting content that intersects with the fourth selection area 30i-2. This end position is the line of text closest to the lower right corner within the intersecting content that intersects with the fourth selection area 30i-2, and the position corresponding to the text closest to the lower right corner within that line of text.

[0624] In some embodiments of this application, the starting position of the fourth selection area 30i-2 is used as the starting position of the fifth selection area 40i-2, and a first mark 40i-8 is displayed at the starting position of the fifth selection area 40i-2; the ending position of the fourth selection area 30i-2 is used as the ending position of the fifth selection area 40i-2, and a second mark 40i-9 is displayed at the ending position of the fifth selection area 40i-2; and the text within the range of the first and second marks in the fifth selection area 40i-2 is set to a selected state. Referring to (4) in FIG13, in some embodiments of this application, the interface 40i further includes a text area 40i-3, an icon area 40i-4, a text and image switching control 40i-5, an editing control 40i-6, and a save control 40i-7 displayed in the fifth selection area 40i-2.

[0625] In some embodiments of this application, the fifth selection area 40i-2 is displayed using a second preset animation effect. This second preset animation effect can be a Matrus mapping animation that switches the display of the fourth selection area 30i-2 to the fifth selection area 40i-2. By using this second preset animation effect to display the fifth selection area 40i-2, this embodiment avoids display jumps during the switch from the fourth selection area 30i-2 to the fifth selection area 40i-2, ensuring smooth display of the fifth selection area 40i-2.

[0626] In other embodiments of this application, it is determined whether there is a text region intersecting with the fifth selection region 40i-2. If there is a text region intersecting with the fifth selection region 40i-2, the text region intersecting with the fifth selection region 40i-2 is taken as the third target text region, and a third rectangle is determined based on the bounding rectangle algorithm according to the fifth selection region 40i-2 and the third target text region, and the fifth selection region 40i-2 is updated according to the third rectangle. In the above embodiments of this application, after the fifth selection region 40i-2 is displayed, if the fifth selection region 40i-2 intersects with a text region, the fifth selection region 40i-2 is further expanded according to the intersecting text region.

[0627] Referring again to (4) in Figure 13, icon area 40i-4 may be used to display one or more application and / or service icons, for example.

[0628] In some embodiments of this application, the application and / or service icons displayed in icon area 40i-4 can be obtained by performing intent recognition processing on the data selected by the intent framework based on the fifth selection area 40i-2, such as text within the fifth selection area 50i-2 or text in a selected state within the fifth selection area 40i-2, and then performing service matching processing based on the recognized user intent. Where the data selected in the fifth selection area 40i-2 is different, the application and / or service icons displayed in icon area 40i-4 can be different. The application and / or service icons displayed in icon area 40i-4 change as the data selected in the fifth selection area 40i-2 changes.

[0629] In some embodiments of this application, users can switch the display of application and / or service icons displayed in the icon area 40i-4 by performing swiping operations, such as swiping up or swiping down.

[0630] In some embodiments of this application, in response to the user's eleventh operation on the fifth selection area 40i-2, the data of the fifth selection area 40i-2 is dragged up, and the dragged data of the fifth selection area 40i-2 is transmitted through the application and / or service icon.

[0631] In some embodiments of this application, referring to (4) in FIG13, in response to a user's long-press operation on the fifth selection area 40i-2, the data in the fifth selection area 40i-2 is dragged up and the dragged content is displayed in a floating frame, and the floating frame is displayed in the user interface, as shown in interface 50i in FIG13 (5). Exemplarily, interface 50i includes the fifth selection area 50i-2, the floating frame 50i-3, and the icon area 50i-4. The floating frame 50i-3 can adjust its display position in real time following the user's drag operation on the screen. Referring to (5) in FIG13, in response to a user's operation of dragging the fifth selection area 50i-2 to an application and / or service icon in the icon area 50i-4, the application and / or service corresponding to the application and / or service icon is triggered to transmit the data in the fifth selection area 50i-2.

[0632] In some other embodiments of this application, referring to (6) in FIG13, in the case of display interface 40i, in response to the user's touch operation on an application and / or service icon, the application and / or service corresponding to the application and / or service icon is triggered to transmit the content in the fifth selection area after being dragged.

[0633] In this embodiment of the application, by long-pressing the fifth selection area, the data in the fifth selection area 50i-2 can be dragged up, and then the dragged data can be transferred by dragging and dropping it to the target application icon or the target service icon.

[0634] Referring again to (4) in Figure 13, exemplarily, if the text and image switching control 40i-5 is in an unselected state and is clicked by the user, the mobile phone responds to the user's operation by displaying the text and image switching control 40i-5 as selected, and performs text recognition on the content of the fifth selection area. Based on the recognized text recognition result, the matching application and / or service icon is automatically displayed in the icon area 40i-4. If the text and image switching control 40i-5 is in a selected state and is clicked by the user again, the mobile phone responds to the user's operation by displaying the text and image switching control 40i-5 as unselected, and takes a screenshot of the data in the fifth selection area 40i-2 as an image, and then automatically displays the matching application and / or service icon in the icon area 40i-4 based on the image. In this embodiment, the default state of the text and image switching control 40i-5 is the selected state.

[0635] Referring again to (4) in Figure 13, for example, after the user clicks the editing control 40i-6, the mobile phone responds to the user's operation and displays the editing interface so that the user can complete the editing of the data selected in the fifth selection area in the editing interface.

[0636] Referring again to (4) in Figure 13, for example, after the user clicks the save control 40i-7, the mobile phone responds to the user's operation by taking a screenshot of the data in the fifth selection area 40i-2 as an image and saving the image in a preset file, such as an album or gallery.

[0637] Referring again to (4) in Figure 13, exemplarily, interface 40i can be the interface obtained by the mobile phone after performing a three-dimensional transformation on all elements of interface 10i, or the aforementioned full-screen screenshot. Interface 40i can display a visual effect of the original interface (i.e., interface 10i) being pushed inward on the screen. In other embodiments of this application, interface 40i can display a visual effect of the original interface undergoing transparency changes or scaling on the screen. This application does not limit the visual effect of the display of interface 40i.

[0638] It should be understood that in the embodiments of this application, the interfaces 40i shown in (4) and (6) of FIG13 and the interfaces 50i shown in (5) and (6) of FIG13 display status bars, such as displaying information such as time, network connection status, signal, and battery level. In other embodiments of this application, the interfaces 40i and 50i do not include status bars.

[0639] It should be understood that, in the embodiments of this application, the dashed lines shown in interface 40i and interface 50i are only for marking the icon display area, and there are no dashed lines in the actual interface displayed on the terminal device.

[0640] In this application embodiment, when a user uses a selection operation to launch the "Anywhere Door" service, the selection area in the interface is determined based on the bounding rectangle algorithm. This ensures that the selected content is fully displayed on the interface while avoiding an excessively large selection area, thus improving the user experience. Furthermore, this application embodiment can determine the selection area corresponding to the selection operation in the interface, display a matching service icon based on the content of the selection area, and share the content of the selection area to the service corresponding to the service icon through user interaction. This avoids misselection of data content not intended by the user, achieving accurate transmission of the user's intended selection and improving the user experience.

[0641] The following section uses the scenario shown in Figure 11 as an example to detail the interaction processing logic of each framework in the data transmission method based on implementation method 5.

[0642] Referring to Figure 14, a flowchart of a data transmission method provided in an embodiment of this application is shown.

[0643] It should be noted that before implementing the data transmission method provided in the embodiment shown in Figure 14, the arbitrary door function corresponding to implementation method 5 needs to be enabled using the method shown in Figure 10.

[0644] For example, when the arbitrary door function corresponding to implementation method 5 is enabled, the data transmission method provided in the embodiment shown in Figure 14 specifically includes:

[0645] S301, the input event framework responds to user actions and identifies the currently received user action.

[0646] Specifically, in the embodiments of this application, after receiving a user operation, the input event framework performs identification processing on the received user operation, specifically identifying whether the user operation is a selection operation.

[0647] For example, in an embodiment of this application, the selection operation is, for instance, an operation triggered by the user's knuckle (knuckle selection operation). In this case, the input event frame for identifying whether the current operation is a selection operation is specifically a knuckle frame.

[0648] Specifically, the knuckle frame can identify and process the user's operation based on a knuckle recognition algorithm, thereby determining whether the currently received user operation is a knuckle selection operation. For the specific implementation logic of the knuckle frame in identifying whether the currently received user operation is a knuckle selection operation, please refer to the relevant documentation on knuckle recognition algorithms; this application embodiment will not elaborate on it.

[0649] Referring to Figure 14, if the currently received user operation is determined to be a selection operation, the knuckle frame instructs the AnyDoor frame to launch the AnyDoor service (the process corresponding to the AnyDoor service can be called the AnyDoor service process). The AnyDoor frame (or AnyDoor service process) then performs the operation of drawing the AnyDoor interface, such as drawing interfaces 10h to 60h in Figure 11, and notifies the smart screenshot application to take a full-screen screenshot of the current interface.

[0650] S302, if it is determined that the currently received user operation is a selection operation, the smart screenshot application responds to the notification of the input event frame, takes a screenshot of the current interface, and obtains a full-screen screenshot of the current interface.

[0651] S303, if it is determined that the currently received user operation is a selection operation, the AnyDoor framework responds to the notification from the input event framework, starts the AnyDoor service process, and creates the activity corresponding to the AnyDoor interface.

[0652] It should be noted that all operations within an activity are closely related to the user; it is a component responsible for interacting with the user and can display specified controls through appropriate interfaces (such as the interface for setting the content view, setContentView(View)). For details on using activities, please refer to the description section of the Android™ Application Programming Interface (API) documentation; this embodiment will not elaborate on this aspect.

[0653] Furthermore, it's worth noting that when launching the AnyDoor service process and creating the activity corresponding to the AnyDoor interface, external interfaces can be exposed. These include interfaces for interacting with the intent framework and the smart screenshot application. This allows the AnyDoor framework to interact with the smart screenshot application through these exposed interfaces, transmitting the operation trajectory corresponding to the selection action, and obtaining the screenshot image captured by the smart screenshot application based on the operation trajectory. It also allows interaction with the intent framework to transmit screenshot images of the operation trajectory, obtain information about the application and / or service to be displayed in the icon area, and the parameters called.

[0654] S304, the Any Door framework draws the operation trajectory in the activity corresponding to the Any Door, and sends the coordinate information of the drawn operation trajectory to the smart screenshot application.

[0655] For example, in some embodiments of this application, the Any Door framework can call a system interface application to draw the operation trajectory corresponding to the selection operation in the activity corresponding to the Any Door interface based on the coordinate information corresponding to the selection operation collected by a sensor, such as a touch sensor. For example, the operation trajectory 20h-2 in the interface 20h shown in Figure 11 (2). That is, when the user makes a knuckle selection operation on the mobile phone screen, the operation trajectory corresponding to the selection operation will be drawn based on the movement of the user's knuckle on the current interface, that is, the user interface will display the operation trajectory 20h-2 in the interface 20h.

[0656] S305, based on a preset algorithm, the intelligent screenshot application determines the fourth selection area according to the coordinate information of the operation trajectory, and captures the first screenshot image of the fourth selection area from the full-screen screenshot, and sends the fourth selection area and the first screenshot image to the arbitrary door frame.

[0657] In some embodiments of this application, the preset algorithm is, for example, the bounding rectangle algorithm. Based on the bounding rectangle algorithm, the fourth selection area is determined according to the coordinate information of the operation trajectory (as shown in Figure 11, the fourth selection area 30h-2 or the fourth selection area 40h-2 is drawn), and the captured first screenshot image includes at least one text area (such as the text area 30h-3 displayed in the fourth selection area 30h-2, or the text area 40h-3 displayed in the fourth selection area 40h-2).

[0658] S306, draw the fourth selection area using the arbitrary door frame, and set a transition effect for the fourth selection area.

[0659] The fourth selection area is, for example, the fourth selection area 30h-2 displayed in interface 30h. The specific effect is shown in interface 40h.

[0660] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0661] S307, after displaying the fourth selection area for a first duration, the arbitrary door frame displays a scanning animation on the fourth selection area and displays the first screenshot image in the fourth selection area.

[0662] The fourth selection area is a non-adjustable selection box, such as the fourth selection area 40h-2 shown in the interface 50h in Figure 11.

[0663] The interface corresponding to the scanning animation is displayed in the fourth selection area, such as interface 50h shown in Figure 11.

[0664] S308, the arbitrary door frame identifies the text region in the first screenshot image, and determines the fifth selection region based on the fourth selection region and the text region in the first screenshot image.

[0665] In some embodiments of this application, the arbitrary gate frame determines a first target text region at the starting position of the fourth selection region and a second target text region at the ending position of the fourth selection region from all target text regions intersecting with the fourth selection region. Based on the bounding rectangle algorithm, the arbitrary gate frame determines a second rectangle based on the first and second target text regions, which is the fifth selection region. For example, referring to (4) in Figure 11, the target text region closest to the upper left corner of the fourth selection region 40h-2 among all target text regions intersecting with the fourth selection region 40h-2 (such as the data corresponding to the data “Aaaa”) is taken as the first target text region at the starting position of the fourth selection region 40h-2. The target text region closest to the lower right corner of the fourth selection region 40h-2 among all target text regions intersecting with the fourth selection region 40h-2 (such as the data corresponding to the data “Bbbb”) is taken as the second target text region at the ending position of the fourth selection region 40h-2. Based on the bounding rectangle algorithm, a second rectangle including the first target text region and the second target text region is determined, and this second rectangle is taken as the fifth selection region 60h-2.

[0666] S309, the Any Door frame takes a screenshot of the fifth selection area to obtain a second screenshot image of the fifth selection area, and sends the second screenshot image to the Intent frame.

[0667] S310, the intent frame receives the second screenshot image of the fifth selection area sent by the arbitrary gate frame, identifies the data of the second screenshot image, determines the user intent based on the identified data, and sends an information retrieval request to the service management frame according to the user intent to obtain matching application and / or service information.

[0668] For example, in some embodiments of this application, the analysis and processing of the second screenshot image by the intent framework is, for example, analysis and processing based on computer vision (CV) tag analysis technology.

[0669] Specifically, in the process of analyzing and processing the second screenshot image based on CV tag analysis technology, the text elements (such as text) in the text area of ​​the second screenshot image can be identified based on CV tag analysis technology, and corresponding tag information can be added to the text elements. Then, based on the tag information of the text elements in the second screenshot image, the intent of the text elements in the second screenshot image can be identified, thereby determining the user intent corresponding to the second screenshot image.

[0670] S311, the service management framework responds to information retrieval requests by querying application and / or service information that matches the user's intent.

[0671] Understandably, the information retrieval request can carry the user intent determined by the intent framework based on the second screenshot image. Thus, after receiving the information retrieval request, the service management framework can query application and / or service information matching the user intent from the managed applications and / or services, based on the user intent parsed from the request.

[0672] Understandably, in the embodiments of this application, application information includes, for example, the application's package name, version code, version name, application label, and icon. Service information includes, for example, the service source (e.g., which application it comes from) and its icon.

[0673] Referring again to Figure 14, for example, after the service management framework finds application and / or service information that matches the user's intent, it will feed the application and / or service information back to the intent framework so that the intent framework can generate a recommendation list.

[0674] S312, the intent framework generates a recommendation list based on the application and / or service information fed back by the service management framework, and feeds back the generated recommendation list, as well as the parameter information for calling each application and / or service in the recommendation list, to the arbitrary gate framework.

[0675] For example, in some embodiments of this application, when the intent framework generates a recommendation list based on application and / or service information, it may sort the first application and / or service. For instance, the order of the icons corresponding to the applications and / or services in the recommendation list may be adjusted according to certain sorting criteria.

[0676] In some embodiments of this application, the sorting criteria may be determined based on user habits (Factor 1). For example, icons of frequently used applications and / or services may be moved to the front of the recommended list.

[0677] In some embodiments of this application, the sorting criteria may be determined based on the frequency of use of the application and / or service (factor 2). For example, the icons of applications and / or services that are used more frequently may be moved to a higher position in the recommendation list.

[0678] In some embodiments of this application, the sorting criteria may be determined based on a user-defined whitelist of applications and / or services (3). For example, the icons of whitelisted applications and / or services may be moved to a higher position in the recommended list.

[0679] In some embodiments of this application, the sorting criteria may be determined based on the usage duration of the application and / or service (factor 4). For example, the icons of applications and / or services with longer usage durations may be moved to a higher position in the recommended list.

[0680] In some embodiments of this application, the sorting criteria may also be determined based on any two or more of the factors 1 to 4 mentioned above.

[0681] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0682] S313, the Any Door frame draws the fifth selection area on top of the full-screen screenshot, and displays the second screenshot image in the fifth selection area.

[0683] In some embodiments of this application, the fifth selection area is, for example, the fifth selection area 40i-2 shown in the interface 40i of FIG12.

[0684] It should be understood that the above description is merely an example provided to better understand the technical solution of this embodiment, and is not intended to be the only limitation of this embodiment.

[0685] S314, the Any Door Frame adds an opening animation to full-screen screenshots and displays the icons of each application and / or service in the recommended list in the icon area at the edge of the screen.

[0686] The icon area is, for example, the icon area 40h-4 displayed in the interface 40h shown in Figure 11.

[0687] The full-screen screenshot can be understood as the interface content in interface 10h-1.

[0688] It should be noted that in some other embodiments of this application, the arbitrary door frame may not include an opening animation for the full-screen screenshot.

[0689] S315, upon receiving a user's selection operation on any icon in the icon area, the service management framework starts the corresponding application or service based on the parameter information of the application or service corresponding to the selected icon provided by the arbitrary gate framework, and uses the started application or service to transmit the data of the second screenshot image corresponding to the fifth selection area.

[0690] The visible area of ​​the icon region can display a maximum of N icons by default. For scenarios with more than N icons, users can swipe up and down the icon region to view icons corresponding to all applications and / or services included in the recommendation list.

[0691] Users can long-press on an icon in the icon area to move its display position.

[0692] For example, in some embodiments of this application, the selection operation of any icon in the icon area is, for example, directly clicking any icon displayed in the icon area.

[0693] The technical solution provided in this application, by way of example, triggers the Any Door when the user makes a selection operation, and determines the selection area and the content within the selection area based on the operation trajectory. This allows the user to conveniently transmit the content within the selection area based on the recommended applications and / or services displayed in the icon area of ​​the current Any Door interface, thereby improving the accuracy of data transmission.

[0694] Furthermore, embodiments of this application also provide an electronic device comprising: one or more processors and a memory. The memory is coupled to the one or more processors and is used to store computer program code, which includes computer instructions. The one or more processors invoke the computer instructions to cause the electronic device to execute the information transmission method shown in the foregoing embodiments.

[0695] It is understood that, in order to achieve the above-mentioned functions, an electronic device may include hardware, software, firmware, or any combination thereof to perform the respective functions. Whether a function is performed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0696] Furthermore, it should be noted that in practical application scenarios, the information transmission methods provided in the above embodiments implemented by electronic devices can also be executed by a chip system included in the electronic device, wherein the chip system may include a processor. The chip system can be coupled to a memory, enabling the chip system to call a computer program stored in the memory during runtime to implement the steps executed by the electronic device. The processor in the chip system can be an application processor or a non-application processor.

[0697] In addition, this application embodiment also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the above-mentioned related method steps to implement the information transmission method in the above embodiment.

[0698] In addition, this application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform the aforementioned related steps to realize the information transmission method described above.

[0699] In addition, embodiments of this application also provide a chip (which may also be a component or module), the chip may include one or more processing circuits and one or more transceiver pins; wherein, the transceiver pins and the processing circuits communicate with each other through internal connection paths, the processing circuits execute the above-mentioned related method steps to implement the information transmission method in the above embodiments, so as to control the receiving pin to receive signals and control the transmitting pin to transmit signals.

[0700] Furthermore, as can be seen from the above description, the electronic devices, computer-readable storage media, computer program products, or chips provided in the embodiments of this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0701] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An information transmission method, characterized in that, include: Receive a first operation applied to the first interface, wherein the first operation is a selection operation; At the first moment, in response to the first operation, a second interface is displayed, the second interface including the selection trajectory corresponding to the first operation displayed on the first interface; At the second moment, a third interface is displayed, the third interface including a selection display area and a service recommendation area. The selection display area includes the first interface and a first selection identifier displayed on the first interface. The first selection identifier is used to identify the first selection area in the first interface corresponding to the selection trajectory. The service recommendation area includes a first group of application and / or service icons determined according to the first selection area. At the third moment, a second operation is received on the third interface, the second operation being an operation to adjust the first selection area; In response to the second operation, a fourth interface is displayed, the fourth interface including the selection display area and the service recommendation area. The selection display area includes the first interface and a second selection identifier displayed on the first interface. The second selection identifier is used to identify the second selection area. The second selection area is the selection area after the first selection area has been adjusted by the second operation. The service recommendation area includes a second set of application and / or service icons determined according to the second selection area. Wherein, the second time is later than the first time, and the third time is later than the second time.

2. The method according to claim 1, characterized in that, After displaying the second interface, the method further includes: A third selection identifier is displayed, which is used to identify the first selection area, and the shape of the third selection identifier is different from that of the first selection identifier; After the third selection identifier is displayed, a first animation effect is displayed, which is a transition animation from the third selection identifier to the first selection identifier.

3. The method according to claim 2, characterized in that, After displaying the first animation effect, the method further includes: The second animation effect is displayed, which is a simulation of scanning the first selected area.

4. The method according to claim 3, characterized in that, The first selection area of ​​the third selection identifier cannot be adjusted.

5. The method according to any one of claims 1 to 4, characterized in that, The first interface displayed in the selection display area in the third interface is the first interface after the first processing. The first selection icon is displayed on the first interface after the first processing. The first screenshot is displayed in the first selection icon. The first screenshot is the screenshot corresponding to the first selection area indicated by the first selection icon. The first processing includes one or more of the following: scaling, deformation, distortion, and changing transparency.

6. The method according to claim 5, characterized in that, The first processing is used to make the first interface displayed in the selected display area of ​​the third interface present an inward pushing visual effect.

7. The method according to claim 6, characterized in that, The response to the second operation, displaying a fourth interface, includes: In response to the first part of the second operation, the first interface that has undergone the second processing is displayed in the selection display area, and the service recommendation area is hidden. The first part of the second operation includes the down event and / or part of the move event in the second operation. The second processing is the same type as the first processing, but the degree of processing of the second processing is less than that of the first processing. In response to the second part of the second operation, a first interface that has undergone the first processing is displayed in the selection display area, a second selection identifier is displayed on the first interface that has undergone the first processing, a second screenshot is displayed in the second selection identifier, the second screenshot is a screenshot corresponding to the second selection area indicated by the second selection identifier, and the service recommendation area is displayed. The second part of the second operation includes the up event in the second operation.

8. The method according to claim 7, characterized in that, The second processing is used to make the first interface displayed in the selected display area of ​​the third interface appear slightly pulled outwards.

9. The method according to any one of claims 1 to 8, characterized in that, The selection display area also includes editing controls displayed on the first interface; When the editing control is displayed, the method further includes: In response to a third operation applied to the editing control, a fifth interface is displayed. The fifth interface includes a first image and a control for editing the first image. The first image is a first screenshot corresponding to the first selection area or a second screenshot corresponding to the second selection area. In response to a fourth operation applied to the first image, a second image is displayed, wherein the second image is an image edited by the first image through the fourth operation; In response to a fifth operation performed on the second image, a third image is displayed above the second image at the location where the user performs the fifth operation. The third image is smaller than the second image and includes all the content of the second image. In response to a sixth operation applied to the third image, a sixth interface is displayed. The sixth interface includes the selection display area and the service recommendation area. The selection display area includes a fifth interface displaying the second image and a fourth image displayed on the fifth interface. The fourth image is smaller than the third image and includes all the content of the third image. The service recommendation area includes icons for a third group of applications and / or services determined based on the fourth image.

10. The method according to any one of claims 1 to 9, characterized in that, The selection display area also includes a text selection control displayed on the first interface; After displaying the fourth interface, the method further includes: In response to a seventh operation applied to the text selection control, a seventh interface is displayed. The seventh interface includes the selection display area, which includes the first interface and a fourth selection identifier displayed on the first interface. The fourth selection identifier is used to identify the second selection area, and the shape of the fourth selection identifier is different from that of the second selection identifier. After the fourth selection icon is displayed, a first animation effect is displayed, which is a transition animation from the fourth selection icon to the fifth selection icon; When the seventh interface is set to a time and the second selection area includes text elements, an eighth interface is displayed. The eighth interface includes the selection display area and the service recommendation area. The selection display area includes the first interface and a fifth selection identifier displayed on the first interface. The fifth selection identifier is used to identify the third selection area corresponding to the text elements in the second selection area. The service recommendation area includes icons of a fourth group of applications and / or services determined according to the third selection area.

11. The method according to any one of claims 1 to 10, characterized in that, The selection display area also includes a save control displayed on the first interface; When the save control is displayed, the method further includes: In response to the eighth operation applied to the save control, the content within the selected trajectory is saved.

12. The method according to any one of claims 1 to 11, characterized in that, The service recommendation areas displayed in both the third and fourth interfaces include icons of user-defined recommended applications and / or services. The service recommendation area displayed in the third interface also includes icons of the first group of applications and / or services determined based on the first selection area, and the service recommendation area displayed in the fourth interface also includes icons of the second group of applications and / or services determined based on the second selection area.

13. The method according to claim 12, characterized in that, Before displaying the fourth interface, the method further includes: Perform intent recognition on the second selection area to determine the user intent corresponding to the second selection area; Based on the user intent, obtain information on the second group of applications and / or services that match the user intent; Based on the information of the second group of applications and / or services, determine the icons of the second group of applications and / or services to be displayed in the service recommendation area.

14. The method according to any one of claims 1 to 13, characterized in that, The selection operation is a user action performed by the user using their knuckles.

15. The method according to any one of claims 1 to 14, characterized in that, After displaying the fourth interface in response to the second operation, the method further includes: In response to a ninth operation applied to the icon of any application and / or service in the service recommendation area, the second selection area is transmitted to the application or service corresponding to the selected icon.

16. The method according to any one of claims 1 to 15, characterized in that, The method further includes: Display the ninth interface; In response to a tenth operation on the ninth interface, an operation trajectory is displayed on the ninth interface, wherein the tenth operation is a selection operation; After displaying the operation trajectory, a sixth selection indicator is displayed on the ninth interface. The sixth selection indicator is used to indicate the fourth selection area on the ninth interface. The fourth selection area is determined according to the operation trajectory. After the sixth selection indicator is displayed, a seventh selection indicator is displayed on the ninth interface. The seventh selection indicator is used to indicate the fifth selection area on the ninth interface. The fifth selection area is obtained by expanding the fourth selection area. The tenth screen is displayed, which includes application and / or service icons recommended based on the content in the fifth selection area; In response to the user's eleventh operation on the fifth selection area, the content in the fifth selection area is transmitted through the application and / or service corresponding to the icon selected by the user.

17. The data transmission method as described in claim 16, characterized in that, The method further includes: The first rectangle is determined based on the operation trajectory, and the first rectangle is used as the fourth selection area.

18. The data transmission method as described in claim 16, characterized in that, The method further includes: Obtain the intersection content in the ninth interface that intersects with the boundary of the fourth selection area; The fifth selection region is determined based on the intersecting content.

19. The data transmission method as described in claim 18, characterized in that, Determining the fifth selection region based on the intersecting content includes: Determine the start and end positions of the intersecting content; Determine the first text region corresponding to the starting position, and determine the second text region corresponding to the ending position; A second rectangle is determined based on the first text region and the second text region, and the second rectangle is used as the fifth selection region.

20. The data transmission method as described in claim 18 or 19, characterized in that, The ninth interface includes text, the intersecting content includes the text lines, and when the fourth selection area is located in the area where the text is located, the fifth selection area includes the entire content of the text lines.

21. The data transmission method as described in claim 18 or 19, characterized in that, The ninth interface includes a map, and the intersecting content includes at least one address entity of the map. If the fourth selection area is located in the area where the map is located, the fifth selection area includes the at least one address entity.

22. The data transmission method as described in claim 21, characterized in that, Determining the first text region corresponding to the starting position includes: The text region closest to the starting position among the intersecting contents that intersect with the boundary of the fourth selection region is taken as the first text region corresponding to the starting position. The starting position is the position corresponding to the first character closest to the first vertex of the fourth selection region among the intersecting contents. The first vertex is the upper left corner vertex of the fourth selection region.

23. The data transmission method as described in claim 21, characterized in that, The step of determining the second text region corresponding to the end position includes: The text region closest to the end position among the intersecting contents that intersect with the boundary of the fourth selection region is taken as the second text region corresponding to the end position. The end position is the position corresponding to the second text closest to the second vertex of the fourth selection region among the intersecting contents. The second vertex is the lower right corner vertex of the fourth selection region.

24. The data transmission method as described in claim 21, characterized in that, The method further includes: The starting position of the fourth selection area is used as the starting position of the fifth selection area, and a first mark is displayed at the starting position of the fifth selection area; The end position of the fourth selection area is taken as the end position of the fifth selection area, and a second mark is displayed at the end position of the fifth selection area; Set the text within the range of the first and second marks in the fifth selection area to the selected state.

25. The data transmission method as described in claim 24, characterized in that, The step of transmitting the content in the fifth selection area through the application and / or service corresponding to the icon selected by the user includes: The text selected in the fifth selection area is transmitted through the application and / or service corresponding to the icon selected by the user.

26. The data transmission method as described in claim 16, characterized in that, The step of responding to the user's eleventh operation on the fifth selection area by transmitting the content in the fifth selection area through the application and / or service corresponding to the icon selected by the user includes: In response to the user's long-press operation on the fifth selection area, the content of the fifth selection area is dragged up; In response to a user dragging the content of the fifth selection area to the application and / or service icon, the content of the fifth selection area is transmitted through the application and / or service corresponding to the application and / or service icon.

27. The data transmission method as described in claim 16, characterized in that, The step of responding to the user's eleventh operation on the fifth selection area by transmitting the content in the fifth selection area through the application and / or service corresponding to the icon selected by the user includes: In response to the user's touch operation on the application and / or service icon, the content of the fifth selection area is transmitted through the application and / or service corresponding to the application and / or service icon.

28. The data transmission method as described in claim 16, characterized in that, The method further includes: A text and image switching control is displayed on the tenth interface; If the text and image switching control is in an unselected state, in response to the user's touch operation on the text and image switching control, the text and image switching control is set to a selected state, and text recognition is performed on the content in the fifth selection area; Identify the application or service corresponding to the text recognition results; The selected application icon or service icon will be displayed on the tenth screen.

29. The data transmission method as described in claim 28, characterized in that, The method further includes: If the text and image switching control is selected, in response to the user's touch operation on the text and image switching control, the text and image switching control is set to an unselected state, and the content in the fifth selection area is captured as an image; Determine the application or service corresponding to the image; The selected application icon or service icon will be displayed on the tenth screen.

30. The data transmission method as described in claim 16, characterized in that, The method further includes: Editing controls are displayed on the tenth interface; In response to the user's touch operation on the editing control, an editing interface is displayed, which is used to edit the content in the fifth selection area.

31. An electronic device, characterized in that, The electronic device includes: a memory and a processor, the memory and the processor being coupled; the memory stores program instructions, which, when executed by the processor, cause the electronic device to perform the information transmission method as described in any one of claims 1 to 30.

32. A computer-readable storage medium, characterized in that, It includes a computer program that, when run on an electronic device, causes the electronic device to perform the information transmission method as described in any one of claims 1 to 30.

Images