Information transfer method and electronic device

Abstract

The present application provides an information transfer method and an electronic device. In the method, directly in response to an operation of a user on information, the electronic device can display multiple application service icons in an interface, so that the user can put the information in a target application service, enabling the electronic device to quickly and efficiently accomplish information transfer between applications; the number of operation steps required for the user to accomplish cross-application transfer of information is effectively reduced, and the efficiency of interaction between the user and the electronic device is improved, thereby enhancing the user experience.

Description

Information transmission method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on June 16, 2023, with application number 202310725366.4 and application name “Information Transmission Method and Electronic Device”, and the Chinese patent application filed with the China Patent Office on June 29, 2023, with application number 202310788791.8 and application name “Data Sharing Method and Related Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to an information transmission method and an electronic device. Background Art

[0003] With the continuous development of electronic information technology, smart devices such as mobile phones and tablets have become indispensable products in our daily lives, work, and entertainment. When using smart devices, users often open multiple applications on the device simultaneously to work together. Users also often use smart devices to transfer information across applications, that is, to share or upload information from one application through another application.

[0004] However, when users need to transfer information across applications, they often have to perform complex and tedious operations to complete the process. For example, when a user needs to forward text or images from a webpage in a browser to a WeChat friend, they typically need to copy or save the text, image, and other information in the browser first, then switch to WeChat before sharing or uploading the information. This information transmission method makes the interaction between users and smart devices inefficient and seriously affects the user experience.

[0005] Therefore, it is necessary to study more efficient and convenient information transmission methods.

[0006] Summary of the Invention

[0007] The purpose of this application is to provide an information transmission method and electronic device. By implementing this method, an electronic device can directly respond to a user's operation on information by displaying multiple application service icons on its interface, so that the user can place information into a target application service, enabling the electronic device to efficiently and quickly complete information transmission between applications, effectively reducing the number of steps required for the user to complete cross-application information transmission, improving the efficiency of interaction between the user and the electronic device, and thereby enhancing the user experience.

[0008] The above-mentioned and other objects are achieved by the features of the independent claims. Further implementations are given in the dependent claims, the description and the drawings.

[0009] In a first aspect, the present application provides an information transmission method, the method comprising: displaying a first interface, the first interface comprising a first sub-interface and a second sub-interface, the first sub-interface being used to display an application interface of a first application, the second sub-interface being used to display an application interface of a second application, the first sub-interface comprising first information, the second sub-interface being used to transmit the first information to the second application under a first operation on the first information; displaying a second interface in response to a second operation on the first information; the second interface comprising the first information and at least one application service icon; and transmitting the first information to the application service corresponding to the target icon in response to a third operation on the first information, the target icon being any one of the at least one application service icons.

[0010] In this method, the first interface can be a smart multi-window interface displayed by an electronic device, that is, there are multiple windows in the first interface, each of which corresponds to a sub-interface, and the applications corresponding to the application interfaces displayed by each sub-interface can be different. The first sub-interface and the second sub-interface are two sub-interfaces among the multiple sub-interfaces corresponding to the multiple windows. Optionally, the number of the multiple windows can be 2, and the sub-interfaces corresponding to these two windows are the first sub-interface and the second sub-interface.

[0011] In the first interface, information elements displayed in any of the sub-interfaces corresponding to the multiple windows can be moved to another sub-interface by dragging it by the user, thereby completing cross-application information transfer. For example, when the user drags the first information in the first sub-interface to the second sub-interface, the information element can be transferred from the application corresponding to the first sub-interface to the application corresponding to the second sub-interface.

[0012] However, due to the limited display area of ​​the electronic device, the number of windows (sub-interfaces) in the first interface is limited, and the number of application interfaces that can be displayed is limited. In addition, the user needs to open these applications in advance through corresponding user operations and need to display the application interfaces of these applications in split screen. When the user wants to share the element dragged in the first interface to an application that is not opened (i.e., the application interface is not displayed in the first interface), the sub-interface displayed in the first interface cannot meet this need.

[0013] Therefore, in this method, the second operation can be used to cause the electronic device to display the "cross-application transfer interface" provided in this application, that is, the second interface is used to share the information element of the first interface to an application that is not opened (that is, the application interface is not displayed in the first interface). In the "cross-application transfer interface", the side of the interface can display the service icon of the application recommended by the electronic device for the information element; then, the user places the dragged information element on the icon and releases it, and the information element can be transferred to the application service corresponding to the icon and the corresponding application service function is activated.

[0014] The second operation can be used to select the first information. Specifically, the second operation can be an operation of long pressing the first information and then dragging it to the side of the first interface (i.e., the side of the display screen). If the user does not let go, the dragged first information moves in the first interface with the user's fingertips. When the user drags the first information to a distance less than a certain threshold from the right or left edge of the screen, the electronic device can display the second interface. In the second interface, the application services corresponding to the at least one application service icon are all application services that can receive the above-mentioned first information. For example, when the above-mentioned first information is an image, The friend chat service can send the image, The Moments service can be used to share the image. The beautification service provided by the electronic device can beautify the image, and the search service provided by the electronic device can perform intelligent search on the image, etc. Therefore, the at least one application service icon may include application service icons corresponding to these services.

[0015] Optionally, the at least one application service icon may include multiple application service icons corresponding to one application. It should be noted that the at least one application service icon is an application service icon rather than an application icon, and the at least one application service icon may not be displayed in the first interface. When a user drags the dragged information (such as the first information) onto one of the at least one application service icons, the information may be directly received by the application service corresponding to the icon.

[0016] In the second interface, if the user associates the first information with the target icon through the third operation, the electronic device can transmit the first information to the application service corresponding to the target icon in response to the third operation. Specifically, the third operation can be an operation of dragging the first information close to the target icon, or an operation of dragging the first information onto the target icon and releasing it, or other operations of associating the first information with the target icon, as long as the third operation can select the target icon required by the user from the at least one application service icon and associate the first information with the target icon.

[0017] It can be understood that the information transmission process provided by this application is continuous from the time the user drags up the first information to the time the first information is placed into the application service corresponding to the target icon. The "continuous" here means that the user does not need to perform operations such as copying and pasting, nor does he need to switch applications. The user's fingers do not need to leave the screen of the electronic device during the entire operation process. The operation is simple and fast, which can effectively improve the user's efficiency in transmitting information across applications and enhance the user experience.

[0018] Optionally, the second interface may also display a third sub-interface. The element content displayed in the third sub-interface is the same as the element content displayed in the first interface (except the status bar). In fact, the third sub-interface displayed in the second interface can be considered as the interface obtained by the electronic device after performing a three-dimensional transformation of all the element content of the first interface (except the status bar). In the second interface, the sub-interface can create a visual effect on the screen that the original interface (i.e., the first interface) is pushed inward. It is understandable that this "opening the door" visual effect metaphorically represents the behavior of transmitting elements (i.e., the first information), which is more intuitive than copying and pasting, and this information transmission method ensures the effective browsing area of ​​the user interface, and can transmit the first information to other application services while viewing the historical application interface (i.e., the first interface) through the sub-interface, which can further enhance the user experience. Optionally, in the second interface, the background behind the sub-interface can be set to the desktop background of the electronic device (i.e., the desktop wallpaper) superimposed with a blurred effect to provide users with better visual effects.

[0019] In combination with the first aspect, in one possible embodiment, the first sub-interface and the second sub-interface are displayed in a top-bottom split screen or a left-right split screen; or, the first sub-interface and the second sub-interface are displayed in a picture-in-picture format.

[0020] In this embodiment, when the first sub-interface and the second sub-interface are displayed in a vertical split-screen or a left-right split-screen manner, this application does not limit the relative positions of the first sub-interface and the second sub-interface within the first sub-interface. For example, the first sub-interface can be displayed above the second sub-interface or below the second sub-interface.

[0021] When the first sub-interface and the second sub-interface are displayed in the form of picture-in-picture, the first sub-interface is an interface displayed in full screen, then the second sub-interface can be an interface displayed in a floating window (small window); the first sub-interface can also be an interface displayed in a floating window (small window), then the second sub-interface can be an interface displayed in full screen, and this application does not limit this. Accordingly, when information is transmitted between the two applications corresponding to the first sub-interface and the second sub-interface, the user can drag the information elements in the interface displayed in the floating window (small window) to the interface displayed in full screen, or drag the information elements in the interface displayed in full screen to the interface displayed in the floating window (small window). When using the "cross-application transmission interface" provided by this application to share the information elements in the first interface to other applications that do not display application interfaces in the first interface, the user can also drag the information elements in the interface displayed in the floating window (small window) or the information elements in the interface displayed in full screen to the side of the screen so that the electronic device displays the "cross-application transmission interface", that is, the second interface.

[0022] In combination with the first aspect, in a possible implementation manner, the second operation is an operation of associating the first information with the target icon.

[0023] In combination with the first aspect, in a possible implementation manner, the second operation is an operation of moving the first information to the target icon.

[0024] Specifically, in this embodiment, moving the first information to the target icon may be moving the first information around the target icon so that the target icon is the icon closest to the first information among the at least one application service icon; or, moving the first information to the target icon may be moving the first information above the target icon. It should be understood that "above" here means that the first information overlaps with the target icon, and the first information is displayed over the target icon.

[0025] Optionally, the second operation may also include leaving the first information on (or around) the target icon for a period of time (for example, 120ms), or dragging the first information onto (or around) the target icon and then directly releasing it, so as to trigger the transmission of the first information to the application service corresponding to the target icon.

[0026] In combination with the first aspect, in one possible implementation, in the process of moving the first information to the target icon, the target icon is enlarged and / or the position of the target icon in the first interface is moved, and the direction of moving the target icon is opposite to the direction of moving the first information.

[0027] In this embodiment, in order to guide the user to accurately place the first information into the application service they need, in the process of dragging the first information to the target icon, when the first information is close to the target icon (here it is assumed that the target icon is always the icon closest to the first information among the at least one application service icon during the entire dragging process), the target icon can be enlarged, and the target icon can be moved in the second interface to make the target icon closer to the first information, creating a visual effect in the second interface that the target icon is absorbed by the first information (that is, the moving direction of the first information is opposite to the moving direction of the target icon, and both are approaching each other), so as to prompt the user that the first information has been associated with the target icon. If the user lets go here, the first information will be transmitted to the application service corresponding to the target icon.

[0028] Optionally, the electronic device will enlarge the target icon in the second interface and / or move the position of the target icon in the first interface only when the distance between the first information and the target icon is less than a first threshold.

[0029] In combination with the first aspect, in one possible implementation, the at least one application service icon includes at least one first icon, and the at least one first icon is an application service icon recommended based on the first information.

[0030] In this embodiment, the electronic device can recommend icons to the user based on the content contained in the first information. Specifically, for this part of the icons recommended by the electronic device to the user, the electronic device can first identify the first information that is dragged up (the electronic device has not yet displayed the second user interface at this time), and obtain the application service icon that meets the user's needs in the second interface based on the identification result. The identification result may include the type of the first information and the element content contained in the first information. For example, assuming that the first information is a picture containing a QR code image, the electronic device can identify the QR code image in the first information, and display the application service icon that can scan the QR code to the user in the second interface, such as WeChat-Scan, Alipay-Scan, etc.; assuming that the first information is a picture containing a face image, the user can identify the face image in the first information, and display the application service icon that can beautify the image or can be used to share the image in the second interface, such as Meitu-Portrait Beautification, WeChat-Friends Circle, etc. For another example, assuming that the first information is text containing location information, the electronic device can recognize the location information in the text and display application service icons for the user to navigate or take a taxi to the location information in the second interface, such as map application A-Navigation, map application A-One-click taxi, etc.; assuming that the above-mentioned first information is text containing time and location information, the electronic device recommends schedule-related application service icons to the user, such as Schedule-Add Schedule, etc.

[0031] That is, when a user drags two information elements in succession, even if the two information elements are of the same type (e.g., both are text or both are images), if the content contained in the two information elements is different (e.g., different image content or different text characters), then the at least one application service icon displayed on the electronic device based on the two elements may also be different; it should be understood that the "different" here may include different icon types and different icon orders. Optionally, the icon of the application service recommended to the user (i.e., the at least one first icon) is displayed in the icon area; among the at least one first icon, the icon corresponding to the application service that is more in line with the current user's intention will be located at the top of the icon area. The current user's intention can be determined based on the content of the elements contained in the dragged information. For example, when the information is an image and contains a QR code image, the icons corresponding to application services such as WeChat-Scan and Alipay-Scan can be displayed at the top of the at least one application service icon; when the information is text and contains location information, the icons corresponding to application services such as Map Application A-Navigation and Map Application A-One-Click Taxi can be displayed at the top of the at least one application service icon.

[0032] In combination with the first aspect, in one possible embodiment, after transmitting the first information to the application service corresponding to the target icon, the method also includes: displaying a third interface, the third interface including all the contents of the first interface, and the third interface also includes any one of a floating window and an information feedback bar, the floating window is used to display the target icon or the application service interface corresponding to the target icon, and the information feedback bar is used to prompt the electronic device of the transmission result of the first information.

[0033] In this embodiment, after the information is transmitted across applications through the second interface, the electronic device can also provide the user with different prompt methods and display different interface effects according to the type of application service entered by the information and the result of the information transmission, so as to prompt the user to know the transmission result of the first information and proceed to the next step.

[0034] In combination with the first aspect, in one possible embodiment, the third interface includes a floating window, which is used to display the target icon. The third interface also includes second information. After displaying the third interface, the method also includes: in response to a fourth operation on the second information, transmitting the second information to the application service corresponding to the target icon, and the fourth operation is an operation of associating the second information with the floating window.

[0035] In this embodiment, it is understood that, except for the floating window, the content of the third interface is the same as that of the first interface. The floating window is a minimized floating window that may not display the application interface but only the target icon. The floating window may be displayed close to the right or left edge of the third interface to reduce obstruction of the third interface, so that the user can continue to transfer information in the third interface (such as the second information) to the application service corresponding to the target icon.

[0036] Optionally, in this embodiment, the floating window can be expanded in response to a user's click operation. After being expanded, the floating window can be displayed in the third interface (at this time, the floating window can no longer be displayed edge to edge, but it is still not displayed in full screen), and it can display an expansion control, which can respond to user operations to display the application interface in the floating window in full screen on the screen.

[0037] In combination with the first aspect, in one possible embodiment, the third interface includes a floating window, which is used to display the application service interface corresponding to the target icon, and the floating window includes an expansion control. After displaying the third interface, the method also includes: in response to a fifth operation on the expansion control, displaying the application service interface displayed in the floating window in full screen.

[0038] When the target icon corresponds to an application service such as navigation or search, which requires displaying a new application service interface, the electronic device needs to immediately display the application service interface corresponding to the target icon to the user. Therefore, in this embodiment, the floating window is a floating window that is displayed on top of the third interface. The third interface can still display the content of the first interface, while the floating window can display the application service interface corresponding to the target icon.

[0039] In combination with the first aspect, in one possible embodiment, the third interface includes an information feedback bar, the information feedback bar includes a jump control, and after displaying the third interface, the method further includes: in response to a sixth operation on the jump control, displaying a fourth interface, the fourth interface being the application service interface corresponding to the target icon.

[0040] When the target icon corresponds to an application service such as a global favorite, after such an application service receives the first information, the user may not need to understand and browse the interface of the application service, but only needs to know whether the electronic device has successfully received the first information. Therefore, in this embodiment, after the first information is transmitted to the application service corresponding to the target icon, the electronic device does not need to directly jump to its corresponding application interface, but only needs to inform the user of the result of the information transmission. Therefore, the information feedback bar can be displayed in the third interface to prompt the user of the result of the first information transmission. The information feedback bar is a jump bar, which can include the jump control, so that the user can quickly enter the application service interface corresponding to the target icon by clicking the jump control to view the processing result of the application service corresponding to the target icon on the first information.

[0041] In combination with the first aspect, in one possible embodiment, the first interface also includes a first trigger area and a second trigger area, the first trigger area is larger than the second trigger area, the second operation includes a first sub-operation and a second sub-operation, the first sub-operation is an operation of moving the first information to the first trigger area, and the second sub-operation is an operation of moving the first information to the second trigger area, and the second interface is displayed in response to the second operation on the first information in the first interface, including: in response to the first sub-operation, when the first information stays in the first trigger area for a first time length, displaying a light band guide special effect in the first interface, the light band guide special effect being used to indicate that the first information is moved to the second trigger area; in response to the second sub-operation, displaying the second interface.

[0042] Since dragging information between different sub-interfaces in the first interface can also complete cross-application transmission of information, and in the daily use of electronic devices, users often drag some elements, these dragging operations may not be for the purpose of transferring the dragged elements across applications (for example, dragging text in document editing to change the position or paragraph of text), therefore, how to combine user operations and user operating habits to guide users to use the transmission method provided in this application, and set the trigger rules of the "cross-application transmission interface" in this method for users, is the key to further improving the efficiency of user interaction with electronic devices.

[0043] Therefore, in this embodiment, the first interface may include the first trigger area and the second trigger area. Optionally, the specific size and position of the first trigger area and the second trigger area may not be displayed in the user interface, that is, the user cannot directly observe the specific outlines of the two areas in the first interface. The size and position of the two areas can only be stored in the electronic device in a form that is perceptible to the electronic device (such as program code). When the user drags the first information into the first trigger area, the electronic device may display a special effect in the first interface (such as the edge of the first interface) to prompt the user to drag the image further in the direction closer to the second trigger area; the special effect may be a user-visible special effect such as a luminous special effect, a shadow special effect, or other user-perceptible special effect, which is not limited in this application; optionally, the electronic device may display a special effect in the first interface only after the user drags the first information into the first trigger area and keeps the first information in the first trigger area for a period of time T. Specifically, T can be set to 600 milliseconds.

[0044] When the user continues to drag the first information to the second trigger area, the electronic device can immediately respond to the operation and display the second interface.

[0045] In an optional embodiment, the second interface further includes a door-closing trigger area. When the user drags the first information into the door-closing trigger area, the electronic device may stop displaying the second interface and redisplay the first interface. If the user does not release the first information, the first information remains in the state of being dragged by the user in the first interface when the first interface is redisplayed, so that the user can reselect a method for transferring the information across applications.

[0046] In combination with the first aspect, in one possible embodiment, when the first sub-interface and the second sub-interface are displayed in a split-screen manner, the first trigger area exists in both the first interface and the second sub-interface, and the second trigger area exists in both the first interface and the second sub-interface.

[0047] In this embodiment, no matter which sub-interface the element dragged by the user belongs to, when the first trigger area exists in both the first interface and the second sub-interface, and the second trigger area exists in both the first interface and the second sub-interface, the electronic device can respond to the user operation in a timely manner to guide the opening of the "cross-application transmission interface".

[0048] In combination with the first aspect, in one possible embodiment, the second interface also includes an icon area, and the at least one application service icon is displayed in the icon area. The at least one application service icon also includes at least one second icon, and the at least one second icon is a custom-displayed application service icon. The at least one first icon is displayed above the at least one second icon.

[0049] In this embodiment, the at least one application service icon is displayed in the icon area, and the at least one application service icon in the second interface can be divided into two parts, one of which is an icon that is fixedly displayed on the right (i.e., the at least one second icon), and the other part of the icon can be an application service icon recommended to the user based on the first information. Optionally, a dividing line can be displayed between the two parts of icons for distinction. Among them, the at least one second icon is an icon that the user customizes based on his or her own usage habits and frequency of use. Regardless of what information the user shares based on the electronic device, the electronic device can include this part of icons in the icons displayed based on the information to meet the user's needs as much as possible; and the at least one first icon is recommended by the electronic device based on the elements contained in the first information. When the user shares other information through this method, the recommended application service icon can be different from the icon in the at least one first icon.

[0050] In combination with the first aspect, in one possible embodiment, the at least one application service icon includes at least one icon displayed in the icon area and at least one icon not displayed in the icon area, the target icon is not displayed in the icon area, and the second interface also includes a third trigger area. Before responding to the second operation on the first information, the method also includes: in response to a seventh operation on the first information, updating the at least one icon displayed in the icon area until the target icon is displayed in the icon area, and the seventh operation is an operation to move the first information to the third trigger area.

[0051] The electronic device can recommend multiple application services that can receive the first information to the user based on the first information, and display the icons of these application services on the right side of the second interface. Specifically, the upper limit of the number of these application service icons can be 20. In this embodiment, when the number of these application service icons exceeds a certain threshold (for example, 7), in order to ensure the neatness and visibility of the interface, the electronic device can only display a part of these application service icons (for example, 7) in the second interface at the same time; when the icons currently displayed on the second interface are not the application services expected by the user, the user can also place the first information in the third trigger area to enable the electronic device to update the icons in the icon area. Specifically, the third trigger area can be set directly above and / or directly below the icon area.

[0052] In combination with the first aspect, in a possible implementation manner, an area of ​​the first information in the second interface is smaller than an area of ​​the first information in the first interface.

[0053] It is understood that in the method provided in this application, the first information in the second interface contains the same content as the first information in the first interface. However, in order to reduce the large area of ​​​​the at least one application service icon blocked by the first information in the second interface, in this embodiment, after entering the second interface, the area of ​​​​the first information can be reduced to facilitate the user's selection of the application service icon.

[0054] In combination with the first aspect, in a possible implementation manner, the first information is any one of an image, text, an audio file, a video file, and a document file.

[0055] In second aspect, the present application provides a method for information transmission, including: in response to any one of a global screenshot operation, a local screenshot operation, and a long screenshot operation on a fifth interface, displaying a sixth interface, wherein the sixth interface includes a first image obtained by screenshotting the fifth interface; in response to an eighth operation on the first image in the sixth interface, displaying a seventh interface; the seventh interface includes the first image and at least one application service icon; in response to a ninth operation on the first image in the seventh interface, transmitting the first image to the application service corresponding to the target icon, wherein the target icon is any one of the at least one application service icon.

[0056] Currently, most smart devices provide users with a quick screenshot function. For example, in some scenarios, users can double-click the screen or swipe down with three fingers to take a full-screen screenshot of the current interface of the electronic device. In some embodiments, the electronic device can also temporarily display the image captured by the user through the full-screen screenshot operation as a thumbnail in the interface. Therefore, in this embodiment, after the user captures an image through the full-screen screenshot operation, the image is displayed as a thumbnail in the first interface. The user can use the quick screenshot function and the transmission method provided in this application to more quickly transfer the captured image to the application service corresponding to the target icon.

[0057] In combination with the second aspect, in a possible embodiment, the first image is an image obtained by performing a global screenshot operation on the fifth interface, and the display of the seventh interface in response to the eighth operation on the first image in the six interfaces includes: in response to a click operation on the first image in the sixth interface, an editing interface is displayed, the editing interface includes the first image and an editing option, and the editing option is used to edit the first image; in response to the editing operation on the first image through the editing option, after the image content of the first image in the editing interface is updated, in response to the move operation on the first image in the editing interface, the seventh interface is displayed, and the eighth operation includes the click operation, the editing operation and the move operation.

[0058] It is understandable that after using the quick screenshot function to capture an image, the user may also need to edit the obtained image, such as further capturing the captured image to change its image area, or marking important content in the image with a brush, and then sharing or uploading the edited image to other applications. Therefore, in this embodiment, after the electronic device briefly displays the image captured by the user through the full-screen screenshot operation in the form of a thumbnail (i.e., the first image) in the sixth interface, the user can enter the image editing interface through the click operation; in the editing interface, the user can edit the first image accordingly. After the editing operation is completed, the electronic device still displays the editing interface, and the user can directly use the move operation in the editing interface to make the electronic device display the second interface, and transmit the edited image (i.e., the first information) to the application service corresponding to the target icon through the ninth operation.

[0059] In the third aspect, the present application provides a method for information transmission, including: in response to a partial screenshot operation on an eighth interface, displaying a ninth interface, the ninth interface including a second image obtained by performing a partial screenshot operation on the eighth interface and a text recognition control, the text recognition control being used to recognize the text contained in the second image; in response to a click operation on the text recognition control, displaying a tenth interface, the tenth interface including first text information obtained by recognizing the second image; in response to a tenth operation on the first text information in the tenth interface, displaying an eleventh interface; the eleventh interface including the first text information and at least one application service icon; in response to the eleventh operation on the first text information in the eleventh interface, transmitting the first text information to the application service corresponding to the target icon, the target icon being any one of the at least one application service icon.

[0060] After the user takes a screenshot, some smart devices also provide the user with a recognition function in this scenario, which is used to identify the image captured by the user (for example, extracting text from the image or translating text, etc.). Therefore, with respect to the screenshot function provided by the electronic device, in this method, the electronic device can quickly obtain the image selected by the user using the knuckle frame through the partial screenshot operation, and display the ninth interface. In the ninth interface, the user can identify the text information contained in the image selected by the partial screenshot frame by clicking the text recognition control; thereafter, the user can perform the tenth operation on the first text information obtained by the recognition to make the electronic device display the "cross-application transmission interface" provided by this application, that is, the tenth interface, and transmit the text information (that is, the first information) to the application service corresponding to the target icon through the eleventh operation.

[0061] Optionally, after using the text recognition control to identify the text information, the text information contained in the image selected by the local screenshot box can be selected by the first cursor and the last cursor box. The user can also adjust the position of the first cursor and the last cursor to further filter the text information to obtain the first text information that he needs to transmit.

[0062] In a fourth aspect, the present application provides an electronic device, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, the one or more processors calling the computer instructions to enable the electronic device to execute the method in the first aspect or any possible implementation of the first aspect, the method in the second aspect or any possible implementation of the second aspect, and the method in the third aspect.

[0063] In a fifth aspect, the present application provides a chip system, which is applied to an electronic device, and the chip system includes one or more processors, and the processors are used to call computer instructions to enable the electronic device to execute the method in the first aspect or any possible implementation of the first aspect, the method in the second aspect or any possible implementation of the second aspect, and the method in the third aspect.

[0064] In a sixth aspect, the present application provides a computer-readable storage medium comprising instructions, which, when the instructions are executed on an electronic device, enables the electronic device to execute the method in the first aspect or any possible implementation of the first aspect, the method in the second aspect or any possible implementation of the second aspect, and the method in the third aspect.

[0065] In the seventh aspect, the present application provides a computer program product comprising instructions, which, when the above-mentioned computer program product is run on an electronic device, enables the above-mentioned electronic device to execute the method in the first aspect or any possible implementation of the first aspect, the method in the second aspect or any possible implementation of the second aspect, and the method in the third aspect.

[0066] In an eighth aspect, a data sharing method is provided, which may include: displaying a first interface, the first interface including a first element; detecting a drag operation on the first element;

[0067] When the drag operation is a first drag operation, the touch start point of the first drag operation is on the first element and located in the first area of ​​the display screen, and the touch end point is located in the second area of ​​the display screen, a first menu bar is displayed, and the first menu bar includes an icon of the first application and an icon of the second application, and the first area and the second area do not overlap; after detecting that the first element is continued to be dragged into the area where the icon of the first application is located and released, first data corresponding to the first element is shared with the first application;

[0068] When the drag operation is a second drag operation, the touch start point of the second drag operation is on the first element and located in the second area, the touch end point is located in the second area of ​​the display screen, the track of the second drag operation is located in the second area, and the first menu bar is not displayed;

[0069] When the drag operation is the third drag operation, the touch start point of the third drag operation is on the first element and located in the second area, the touch end point is located in the second area, and the track of the third drag operation passes through the first area, displaying the first menu bar.

[0070] When implementing the method of the eighth aspect, the electronic device will display the first menu bar after detecting the operation of dragging the first element from the first area to the second area, or detecting the operation of dragging the first element from the second area to the first area and then to the second area, and the electronic device will not display the first menu bar after detecting the operation of dragging the first element within the second area. This can provide users with a convenient, simple, fast, smooth and intuitive data sharing solution, which is convenient for users to share data when needed, and can avoid displaying the first menu bar when the touch point is first located in the second area, so as to achieve an anti-mistouch effect.

[0071] In combination with the eighth aspect, in some embodiments, when the drag operation is a first drag operation, the first menu bar is displayed in response to the first drag operation; when the drag operation is a second drag operation, the first menu bar is displayed without responding to the second drag operation; when the drag operation is a third drag operation, the first menu bar is displayed in response to the third drag operation.

[0072] In conjunction with the eighth aspect, in some embodiments, when the drag operation is a first drag operation, the first menu bar is displayed after the touch endpoint of the first drag operation remains in the second area for a first duration; when the drag operation is a second drag operation, the first menu bar is not displayed after the touch endpoint of the second drag operation remains in the second area for a first duration; and when the drag operation is a third drag operation, the first menu bar is displayed after the touch endpoint of the third drag operation remains in the second area for a first duration. If the touch endpoint remains in the second area for the first duration, it can be determined that the user has a need to share data, and thus the electronic device displays the first menu bar again, thereby more accurately providing the user with data sharing services and avoiding accidental triggering of the first menu bar.

[0073] In conjunction with the eighth aspect, in some implementations, the first drag operation and the operation of continuing to drag the first element into the area where the icon of the first application is located and releasing it are continuous and uninterrupted.

[0074] In conjunction with the eighth aspect, in some embodiments, before displaying the first interface, the method may further include: displaying a second interface, the second interface including text; detecting an operation of selecting first text in the text, and displaying a toolbar on the first text, the toolbar including multiple controls for processing the first text; and displaying the first interface, including: detecting a long press operation on the first text, stopping displaying the toolbar, and displaying the first interface; wherein the first element includes a preview image of the first text, and the first data includes the first text. This allows the electronic device to conveniently share the text selected by the user.

[0075] In conjunction with the eighth aspect, in some embodiments, before displaying the first interface, the method may further include: detecting a screenshot operation while displaying the second interface; and displaying the first interface, including: displaying the first interface in response to the screenshot operation, wherein the first element includes a preview of the screenshot image of the second interface, and the first data includes the screenshot image. This allows the electronic device to conveniently share screenshots.

[0076] In conjunction with the eighth aspect, in some embodiments, before displaying the first interface, the method may further include: displaying a second interface, the second interface including an icon for the first image and an icon for the second image; detecting an operation of selecting the icon for the first image; and displaying the first interface, including: detecting a long press operation on the icon for the first image and displaying the first interface; wherein the first element includes a preview of the icon for the first image, and the first data includes the first image. This allows the electronic device to conveniently share images, which may include pictures, animated images, videos, and other types.

[0077] In conjunction with the eighth aspect, in some embodiments, the first menu bar and the second area where the touch endpoint of the first drag operation or the third drag operation is located are on the same side of the display screen. That is, the electronic device displays the first menu bar in the direction in which the user drags the first element, thereby providing a better user experience.

[0078] In conjunction with the eighth aspect, in some embodiments, when the drag operation is the first drag operation or the third drag operation, when the first menu bar is displayed, the first interface is further adjusted to a perspective style of the first side, where the first side is the side of the first menu bar on the display screen. In this way, the perspective style of the first interface and the first menu bar on the first side can visually create an "open door" effect.

[0079] In combination with the eighth aspect, in some embodiments, the first interface also includes a video being played, and the method may further include: when the drag operation is a first drag operation, or when the drag operation is a third drag operation, when the first menu bar is displayed, a screenshot image of the first interface is displayed, and the screenshot image of the first interface is displayed in a perspective style of the first side, where the first side is a side of the first menu bar on the display screen.

[0080] In combination with the eighth aspect, in some embodiments, the second area includes: a rectangular area at the left edge of the display screen, and a rectangular area at the right edge of the display screen, and the height of the rectangular area is the same as the height of the display screen.

[0081] In conjunction with the eighth aspect, in some embodiments, when the drag operation is a first drag operation, the trajectory of the first drag operation passes through a third area, and the third area includes the second area; the method may further include: in response to the operation of dragging the first element to the third area in the first drag operation, displaying a light strip, wherein the light strip and the first menu bar are located on the same side of the display screen; displaying the first menu bar includes: in response to the operation of dragging the first element to the third area and then to the second area in the first drag operation, stopping displaying the light strip and displaying the first menu bar. The electronic device displays the light strip to prompt the user that continuing to drag in the same direction will trigger the display of the first menu bar, thereby sharing data.

[0082] In some embodiments, the third area includes: a rectangular area at the left edge of the display screen, and a rectangular area at the right edge of the display screen, and a height of the third area is the same as a height of the display screen.

[0083] In combination with the eighth aspect, in some embodiments, the first menu bar further includes an icon for a third application and an icon for a fourth application, and the icon for the first application, the icon for the second application, the icon for the third application, and the icon for the fourth application are arranged in sequence from top to bottom; after displaying the first menu bar, the method may further include: detecting a fourth drag operation of continuing to drag the first element within a fourth area, the fourth area including the second area where the touch end point of the first drag operation or the third drag operation is located, the touch starting point of the fourth drag operation is the same as the touch end point of the first drag operation or the third drag operation, and the touch end point of the fourth drag operation is closest to the icon of the second application in the first menu bar; in response to the fourth drag operation, displaying the icon of the fourth application as the base size, displaying the icon of the second application as a size enlarged by a first ratio on the base size, and displaying the icon of the first application and the icon of the third application as a size enlarged by a second ratio on the base size; the first ratio is greater than the second ratio.

[0084] According to the above embodiment, the closer the user's touch point is to the application icon in the first menu bar, the larger the corresponding application icon can be. This can create an effect where the application icon in the first menu bar is attracted by the user's finger, and the user can focus on the corresponding application icon by enlarging the size of the application icon.

[0085] In combination with the previous embodiment, the fourth area includes: a rectangular area including the second area where the touch endpoint of the first drag operation or the third drag operation is located, and the height of the fourth area is the same as the height of the display screen.

[0086] In combination with the eighth aspect, in some embodiments, after detecting a fourth drag operation of continuing to drag the first element within the fourth area, the method may further include: detecting a fifth drag operation of continuing to drag the first element to the fifth area, and the fifth area and the fourth area do not overlap; in response to the fifth drag operation, displaying each icon in the first menu bar as the basic size.

[0087] In combination with the eighth aspect, in some embodiments, when the drag operation is the first drag operation, or when the drag operation is the third drag operation, after displaying the first menu bar, the method may further include: detecting a sixth drag operation of continuing to drag the first element to the sixth area, the sixth area and the second area do not overlap, and the sixth area is located on the opposite side of the first menu bar in the display screen; in response to the sixth drag operation, stopping displaying the first menu bar.

[0088] Through the above embodiment, the electronic device provides a sixth area for stopping displaying the first menu bar. After the user mistakenly triggers the electronic device to display the first menu bar, the user can stop the mistaken operation simply, conveniently and quickly.

[0089] In some embodiments, the touch start point of the sixth drag operation is the same as the touch end point of the first or third drag operation, that is, the sixth drag operation and the first or third drag operation are continuous and uninterrupted. This allows the user to maintain continuous contact with the display screen without having to lift their finger from the electronic device, thus conveniently and quickly avoiding accidental operations.

[0090] In some embodiments, the sixth region and the fifth region do not overlap.

[0091] In combination with the eighth aspect, in some embodiments, the method may further include: displaying a third interface, the third interface including a second element; detecting a drag operation on the second element; when the drag operation is a seventh drag operation, the touch starting point of the seventh drag operation is on the second element and located in the seventh area of ​​the display screen, the touch end point is located in the eighth area of ​​the display screen, and the second menu bar is displayed, and the seventh area and the eighth area do not overlap; when the drag operation is the eighth drag operation, the touch starting point of the eighth drag operation is on the second element and located in the eighth area, the touch end point is located in the eighth area of ​​the display screen, the trajectory of the eighth drag operation passes through the seventh area, and the second menu bar is displayed; when the drag operation is not the seventh drag operation or the eighth drag operation, the second menu bar is not displayed; wherein the eighth area is a partial area of ​​the second area.

[0092] Through the above embodiment, it is equivalent to reducing the area for triggering the display of the menu bar from the second area to the eighth area. By reducing the area, the probability of the user accidentally triggering the menu bar can be avoided.

[0093] In some embodiments, the eighth area includes: a rectangular area at the left edge of the display screen, and a rectangular area at the right edge of the display screen; the height of the eighth area is the same as that of the second area, and the width of the eighth area is half the width of the second area.

[0094] In some implementations, the third interface is provided by a fifth application, which includes a text editing application.

[0095] In some embodiments, the application icons included in the second menu bar may be different from the application icons included in the first menu bar. Specifically, the icons included in the first menu bar may include icons of applications that support receiving the first data, and the icons included in the second menu bar may include icons of applications that support receiving the second data corresponding to the second element.

[0096] In conjunction with the eighth aspect, in some embodiments, the second area when the display screen is in the landscape state is different from the second area when the display screen is in the portrait state.

[0097] In conjunction with the eighth aspect, in some implementations, the first interface is provided by a sixth application, which is different from the first application. This method can achieve cross-application data transmission.

[0098] In the ninth aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method provided in the eighth aspect or any one of the embodiments of the eighth aspect.

[0099] In a tenth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the eighth aspect or any one of the embodiments of the eighth aspect is implemented.

[0100] In the eleventh aspect, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements the method provided in the eighth aspect or any one of the embodiments of the eighth aspect.

[0101] In a twelfth aspect, a method for recommending services is provided, which is applied to an electronic device. The method may include: obtaining first content; converting the first content into a first entity, the first entity including an entity category and an entity value; identifying a corresponding first intent based on the first entity; determining a service that supports the first intent; and displaying an icon of the service on a display screen.

[0102] By implementing the method of the twelfth aspect, after the user inputs content into the electronic device, the electronic device can recommend the services the user wants based on the user's personal intentions, making it easier for the user to use these services.

[0103] In conjunction with the twelfth aspect, in some embodiments, the electronic device may determine the entity category of the first entity and / or the first intent category corresponding to the modality to which the first content belongs based on a mapping relationship between the entity category and / or modality and the intent category; and determine the first intent based on the first intent category and the entity value. In this way, the electronic device can determine the user's intent category based on the entity category and / or modality.

[0104] In conjunction with the twelfth aspect, in some embodiments, the electronic device can determine an intent based on a global intent system. Specifically, the electronic device can find a first intent category in the global intent system that has an entity category labeled as a first entity; and fill the entity value of the first entity into the slot indicated by the first intent category to obtain the first intent. The global intent system includes information on multiple intent categories, and each intent category includes the following items for the corresponding category: a label and a slot name.

[0105] In some embodiments, the information of each type of intent further includes one or more of the following for the corresponding category: identification, level, category of the value filled in the slot, and necessity of the slot.

[0106] In some implementations, the global intent framework is pre-stored in the electronic device, or downloaded from a network by the electronic device.

[0107] In some embodiments, the electronic device may also search for services based on a global service system. Specifically, the electronic device may find a service in the global service system whose supported intent category is identified by a first identifier, where the first identifier is an identifier of a category to which the first intent belongs; and determine the service in the global service system whose supported intent category is identified by the first identifier as a service that supports the first intent. The global service system includes information on multiple services, and the information on each service includes an identifier of the intent category supported by the corresponding service.

[0108] In some embodiments, the information of each service also includes one or more of the following for the corresponding service: ID, name, icon, type, slot name, package name of the application to which it belongs, component name, URL, and parameter name for transmitting data.

[0109] In some implementations, the global service system is pre-stored in the electronic device, or downloaded from the network by the electronic device.

[0110] In some embodiments, the information of each service also includes: a slot name, a parameter name for transmitting data, and the method of the twelfth aspect may also include: receiving a user operation of the icon of the first service among the icons acting on the services; passing the value of the first slot in the first intention to the first parameter of the first service, the first slot being the slot indicated by the slot name of the information of the first service, and the first parameter being the parameter indicated by the parameter name of the information of the first service; and starting the first service based on the first parameter.

[0111] In combination with the twelfth aspect, in some embodiments, the electronic device can obtain the first content through the entrance of the first service; before obtaining the first content, the method may also include: subscribing to the intent category supported by the first service, and the first intent belongs to the intent category supported by the first service.

[0112] In combination with the twelfth aspect, in some embodiments, the electronic device can obtain the first content through the entrance of the first service; before obtaining the first content, the method may also include: subscribing to a service supported by the first service, and the first service belongs to the services supported by the first service.

[0113] In combination with the twelfth aspect, in some embodiments, after determining the service that supports the first intention and before displaying the service icon on the display screen, the method may also include: sorting multiple services that support the first intention, and arranging the service icons displayed on the display screen in sequence according to the sorting of the multiple services.

[0114] In some embodiments, there may be multiple rules for sorting multiple services, such as priority for most recently used, priority for most frequently used, priority for intent categories with high confidence (i.e., high level) corresponding to the intent, user's application preference order, etc.

[0115] In some embodiments, before sorting the multiple services supporting the first intent, the method may further include: sorting the multiple first intents corresponding to the first entity, and the sorting of the multiple services is obtained according to the sorting of the multiple first intents.

[0116] In some embodiments, the manner in which the electronic device ranks multiple first intentions corresponding to the first entity may include: using a click-through rate prediction model to rank multiple first intentions corresponding to the first entity; wherein the click-through rate prediction model is trained based on multiple groups of first historical behavior data, and a group of first historical behavior data includes: time features, content features, and click-through rate features; the time features indicate one or more of the following: whether the time when the electronic device historically acquired content was a weekday, a holiday, or a time period; the content features indicate one or more of the following: the source application of the content acquired by the electronic device, the length of the content, the entity category of the entity corresponding to the content, and the modality of the content; the click-through rate features indicate: after the electronic device acquires the content and recommends the service, the user's click rate on the recommended service.

[0117] In conjunction with the twelfth aspect, in some embodiments, the electronic device can also learn the mapping relationship between undefined entity categories and / or modalities and intent categories. Specifically, the electronic device can obtain multiple sets of second historical behavior data; based on the multiple sets of second historical behavior data, determine that the first entity category and / or first modality has a mapping relationship with the second intent category; wherein, a set of second historical behavior data includes: after the electronic device displays the icon of the recommended service on the display screen according to the content corresponding to the first entity category and / or first modality input by the user, it does not receive a user operation acting on the icon of the service, and within the first time period after stopping to display the icon of the service, the electronic device starts the service under the trigger of the user; among the multiple sets of second historical behavior data, the K services with the highest startup frequency support the second intent category.

[0118] In this way, for the case where the mapping relationship from entity category and / or modality to intent category is incomplete, a mapping relationship mining method is provided. The electronic device can mine the mapping relationship from subsequent behaviors that are not clicked after the recommendation, thereby supplementing the recall of the mapping relationship and better providing recommendation services for users.

[0119] In conjunction with the twelfth aspect, in some embodiments, the electronic device can also learn undefined intent categories. Specifically, the electronic device can obtain multiple sets of second historical behavior data; determine a third intent category based on the multiple sets of second historical behavior data; wherein, a set of second historical behavior data includes: after the electronic device displays the icon of the recommended service on the display screen according to the content corresponding to the first entity category and / or the first modality input by the user, does not receive a user operation acting on the service icon, and within a first period of time after stopping displaying the service icon, the electronic device starts the service under the trigger of the user; the third intent category is defined based on the N services with the highest startup frequency in the multiple sets of second historical behavior data.

[0120] In this way, in response to the situation where the user's actual intention exceeds the scope of the intention categories defined by the decision cloud platform, a method for mining new intention categories is provided. The electronic device can mine potential undefined new intention categories from subsequent behaviors that are not clicked after the recommendation, and define the mapping relationship between the new intention category and the entity category and / or modality, thereby supplementing the recall of the intention category and mapping relationship to better provide recommendation services for users.

[0121] In some embodiments, the method may further include: determining a mapping relationship between the first entity category and / or the first modality and the third intent category.

[0122] In some implementations, after acquiring the first content, the electronic device can also preload resources needed for subsequent processing. For example, the electronic device can load mappings between entity categories and / or modalities and intent categories, or load a global intent system, or load a global service system. This allows the electronic device to preload the required resources, speeding up the processing of subsequent steps.

[0123] In conjunction with the twelfth aspect, in some embodiments, the electronic device can extract features from multimodal content in parallel. Specifically, the electronic device can launch a first thread and a second thread in parallel, with the first thread used to extract features of the content of the first modality and the second thread used to extract features of the content of the second modality; based on the features of the content of the first modality and the features of the content of the second modality, a first entity is obtained. By extracting features concurrently from multiple channels, the overall business congestion problem caused by anomalies in the extraction of single modal information can be effectively reduced, thereby improving business stability.

[0124] In some embodiments, the first thread includes one or more threads, the second thread includes one or more threads, and one thread is used to call an entity recognition capability to recognize an entity.

[0125] In the thirteenth aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method provided in the twelfth aspect or any one of the embodiments of the twelfth aspect.

[0126] In the fourteenth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method provided in the twelfth aspect or any one of the embodiments of the twelfth aspect is implemented.

[0127] In the fifteenth aspect, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, it implements the method provided in the twelfth aspect or any one of the embodiments of the twelfth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] FIG1 is a schematic diagram of a process for transmitting information across applications according to an embodiment of the present application;

[0129] FIG2 is a schematic diagram of a process for transmitting information across applications according to an embodiment of the present application;

[0130] FIG3A is a schematic diagram of a process for transmitting an image across applications according to an embodiment of the present application;

[0131] FIG3B is a structure of a framework for recommending services provided in an embodiment of the present application;

[0132] FIG3C is another structure of a framework for recommending services provided in an embodiment of the present application;

[0133] FIG3D is a flowchart of a method for an electronic device to perform self-learning of intentions during a service recommendation process according to an embodiment of the present application;

[0134] FIG4 is a schematic diagram of a process for transmitting text across applications according to an embodiment of the present application;

[0135] FIG5 is a schematic diagram of a process for transmitting information across applications in a horizontal screen state according to an embodiment of the present application;

[0136] FIG6 is a schematic diagram of a process for transmitting a global screenshot image across applications according to an embodiment of the present application;

[0137] FIG7 is a schematic diagram of a process for transmitting a partial screenshot image across applications according to an embodiment of the present application;

[0138] FIG8 is a schematic diagram of a process for transmitting text in a partial screenshot image across applications according to an embodiment of the present application;

[0139] FIG9 is a schematic diagram of a process of triggering a “cross-application transmission interface” operation according to an embodiment of the present application;

[0140] FIG10 is a schematic diagram of an operation process of closing a “cross-application transmission interface” provided in an embodiment of the present application;

[0141] FIG11 is a schematic diagram of an operation process of closing a “cross-application transfer interface” provided in an embodiment of the present application;

[0142] FIG12 is an interface diagram of some cross-application transmissions through the split-screen function provided in an embodiment of the present application;

[0143] FIG13 is an interface diagram of cross-application transmission via the “cross-application transmission interface” in some split-screen states provided by an embodiment of the present application;

[0144] FIG14 is an interface diagram of some picture-in-picture functions for cross-application transmission provided in an embodiment of the present application;

[0145] FIG15 is an interface diagram of cross-application transmission via the “cross-application transmission interface” in some picture-in-picture states provided by an embodiment of the present application;

[0146] FIG16 is an interface diagram of a snack bar for feedback after some information is transmitted across applications according to an embodiment of the present application;

[0147] FIG17 is a schematic diagram of an interface for providing feedback in the form of capsules after information is transmitted across applications according to an embodiment of the present application;

[0148] FIG18 is a schematic diagram of a process of performing secondary transmission through a capsule and then expanding the capsule according to an embodiment of the present application;

[0149] FIG19 is a schematic diagram of an interface for providing feedback in a floating window after some information is transmitted across applications according to an embodiment of the present application;

[0150] FIG20 is a schematic diagram of an interface for providing feedback via a snackbar after some information is transmitted across devices according to an embodiment of the present application;

[0151] FIG21 is a schematic diagram of an interface for providing toast feedback after some information cross-device transmission fails according to an embodiment of the present application;

[0152] Figures 22A-22N are a set of user interfaces for implementing data sharing on an electronic device, provided in an embodiment of the present application;

[0153] FIG23 is a schematic diagram of various hot zones of a mobile phone in portrait mode provided by an embodiment of the present application;

[0154] FIG24 is a schematic diagram of hot spots in a mobile phone in landscape mode according to an embodiment of the present application;

[0155] Figures 25A-25F are a set of user interfaces for sharing screenshots provided in an embodiment of the present application;

[0156] FIG25G is a user interface for preventing accidental touches provided in an embodiment of the present application;

[0157] 25H-25L are another user interface for preventing accidental touches provided in an embodiment of the present application;

[0158] Figures 25M-25R are another user interface for preventing accidental touches provided in an embodiment of the present application;

[0159] FIG26A-FIG26B are a set of user interfaces for moving a floating window provided in an embodiment of the present application;

[0160] Figures 26C-26L are user interfaces for preventing accidental touches in a floating window scenario provided by an embodiment of the present application;

[0161] FIG27 is a schematic diagram of a process for updating icons in an icon area according to an embodiment of the present application;

[0162] FIG28 is an interface diagram showing some application service icons for users based on location information provided by an embodiment of the present application;

[0163] FIG29 is an interface diagram showing some application service icons for users based on location and time information provided by an embodiment of the present application;

[0164] FIG30 is an interface diagram of some application service icons displayed to users based on QR code images provided in an embodiment of the present application;

[0165] FIG31 is an interface diagram showing some application service icons for users based on facial images provided in an embodiment of the present application;

[0166] FIG32 is an interface diagram of some information transmission via chat interface thumbnails in the “cross-application transmission interface” provided by an embodiment of the present application;

[0167] FIG33 is an interface diagram of some cross-application information transmission via sidebar icons provided in an embodiment of the present application;

[0168] FIG34 is an interface diagram of some custom sidebar icons provided in an embodiment of the present application;

[0169] FIG35 is a schematic diagram of a process for transmitting a screen recording file across applications according to an embodiment of the present application;

[0170] FIG36 is a schematic diagram of a process for transmitting an edited screenshot image across applications according to an embodiment of the present application;

[0171] FIG37 is a schematic diagram of a process for transmitting a long screenshot image across applications according to an embodiment of the present application;

[0172] FIG38 is a diagram of a system setting interface related to some “smart transfer” functions provided in an embodiment of the present application;

[0173] FIG39 is a diagram of a system setting interface related to some “smart transfer” functions provided in an embodiment of the present application;

[0174] FIG40 is a diagram illustrating an electronic device according to an embodiment of the present application;

[0175] Figure 41 is a flowchart of an information transmission method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0176] The terms used in the following examples of the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in the specification and appended claims of the present application, the singular expressions "a," "an," "said," "above," "the," and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in the present application refers to and encompasses any or all possible combinations of one or more of the listed items.

[0177] To facilitate understanding, the following first introduces the relevant terms involved in the embodiments of this application.

[0178] (1) Information transmission across applications and devices

[0179] Cross-app information transfer refers to the transfer of information between two apps on the same device. This means obtaining target information through one app and sharing, saving, or editing it through another app. For example, you can forward text or images from a webpage in a browser to a WeChat friend, or send a picture sent by a WeChat friend to a QQ friend.

[0180] Cross-device information transmission occurs when two devices establish a communication connection via Wi-Fi, Bluetooth, or a cellular network. For example, a user might use Bluetooth to transfer a picture between two devices, or share a picture from a mobile phone to a PC on the same network.

[0181] Specifically, the above information can be pictures, text, video, voice, links, documents, etc.

[0182] (2) Cross-application transmission interface

[0183] In this application, the "cross-application transmission interface" (or cross-application transmission interface) is an interactive method provided by an electronic device based on the user's demand for cross-application and cross-device information transmission. In some embodiments of this application, this function can also be called "smart flow". As its name suggests, in any interface displayed by an electronic device, when the user needs to transmit information in the interface across applications or across devices, the user only needs to long press the information element in the interface and drag it to the edge of the screen. The electronic device will transform the original interface displayed on the screen into three dimensions, creating a visual effect of pushing the original interface inward on the screen, and display the electronic device's recommended icon for the element on the side of the screen (for details, please refer to the relevant description of the subsequent embodiments, which will not be repeated here). The recommended icon can be the service icon of the application; after that, the user places the dragged information element on the icon and lets go, and the information element can be transferred to the application service corresponding to the icon and start the corresponding application service function. Specifically, in this application, an application APP installed in an electronic device may have multiple application services. In the cross-application transmission interface provided by this application, the service icons of the applications displayed may include some or all of the application service icons provided by the same application APP. This application does not limit this.

[0184] (3) Small windows, capsules, snack bars, toast feedback, and side app bars

[0185] "Small window" is a system tool of smart devices. It can float the operation window of an application above the interface or on the desktop, allowing users to easily switch between recently used applications. As a reduced application interface, the floating window can be displayed on top of other application interfaces (such as the application interface displayed in full screen), and the small window can still display the interface style and function list inherent to its corresponding application. Users can directly operate the controls in the floating window interface to use the corresponding functions without closing or exiting the application interface of other applications. For example, when playing games or watching videos in full screen, you suddenly receive a message from WeChat, QQ, etc., and its corresponding chat interface can be directly above the game interface in the form of a floating window. Users can view and reply to messages in the chat interface displayed in the small window without switching or exiting the currently open application.

[0186] "Capsule" is a form of mini-window after minimization, named because of its capsule-like shape. Unlike mini-windows, it generally cannot display the inherent interface style and function list of its corresponding application, and will only be displayed as a small icon on the side of the electronic device screen. Users can expand the capsule by clicking on it. After expansion, the capsule will be displayed at the top of the screen in the form of an application interface (such as a mini-window).

[0187] "Toast feedback" is a lightweight feedback method and a form of non-modal pop-up window. Common toast feedback is expressed as text or a combination of icon and text. The location of the page will be designed at the top, middle or bottom of the page according to different scenarios. It will automatically disappear after a certain time interval (3s to 5s), or it will automatically disappear when the user clicks elsewhere. Toast feedback will not interrupt the user's current operation process and will cause minimal disturbance to the user. Its disadvantage is that it is easily ignored by users and is not suitable for displaying too much information. It may disappear before the user has finished reading the information.

[0188] "Snack bar" is a lightweight feedback method that is slightly heavier than toast. It contains a line of text related to the ongoing operation and may contain corresponding jump controls to facilitate users to browse the application interface related to the aforementioned ongoing operation. Snack bar is also a non-modal pop-up window, generally used to notify users that the program is undergoing or is about to undergo a process. Its content must be related to the user's current operation. It usually appears briefly at the bottom of the screen, does not interrupt the user experience, does not require any user operation, and will automatically disappear after a period of time. In addition, in some embodiments, if a snack bar appears in the interface, the user can make the snack bar disappear by clicking an area outside the snack bar.

[0189] The "Sidebar" is a quick-access area displayed on the side of an electronic device screen that can hold multiple app icons. Users can customize the types of apps in the sidebar and the order of the app icons. In any interface, users can swipe inward from the edge of the screen and pause to call up the sidebar. Click an app icon to open the corresponding app, and once opened, the app will generally be displayed at the top of the interface in the form of a small window.

[0190] (4) Global Collection

[0191] Global Collection is an application service provided by embodiments of the present application. It can be used to collect elements such as pictures, text, audio, and web pages that users have viewed on electronic devices. After adding elements to the global collection application service, users can also edit the collected elements, such as adding tags to highlight key points in the collected videos, so that users can efficiently review the key information contained in the elements when replaying them.

[0192] The user can add an element to the global collection application service by swiping down with three fingers, or by using the cross-application transfer interface provided in this application. After successfully adding an element to the global collection application service, the electronic device can display the capsule corresponding to the global collection application service in the interface, making it easier for the user to add other elements in the interface to the global collection application service. Please refer to the relevant description of the subsequent embodiments for details, which will not be repeated here.

[0193] With the continuous development of electronic information technology, smart devices such as mobile phones and tablets have become indispensable products in our daily lives, work, and entertainment. When using smart devices, users often open multiple applications simultaneously to work together. Users also frequently use smart devices to transfer information across applications, processing, sharing, and uploading information from one application through another. However, when users need to transfer information across applications, they need to perform complex and tedious operations to complete the process.

[0194] FIG1 shows a specific process in which a user adds text from the application “Novel Reading” to the application “Notes”.

[0195] The user interface 01 shown in (A) of FIG1 is the interface displayed by the electronic device after the user opens a novel in the "Novel Reading" application. The user interface 01 includes a status bar 011, text information 012, a text option list 013, and a navigation bar 014, where:

[0196] The status bar 011 may include the name of the operator (eg, China Mobile), time, WI-FI icon, signal strength, and current remaining power.

[0197] The text information 012 can be divided into two parts. The text information "ABCDEFGHIJKLMN" is the text that the user wishes to transfer to another application (for example, to copy to a note-taking application), so this part of the text has been selected by the user and is in the dark area. The text information "OPQRSTUVWXYZ" has not been selected by the user and is therefore not in the dark area. It should be understood that although not shown in Figure 1, when selecting text, the user can long-press the text information 012, and after the first cursor 0121 and the last cursor 0122 appear, adjust the position of the first cursor 0121 and the last cursor 0122 to complete the text selection.

[0198] Text option list 013 includes a copy option, a select all option, a search option, a translate option, and a share option. The copy option is used to copy the text information selected by the user. After copying, the user can paste the selected text information into other text boxes on the electronic device; the select all option is used to select all text information in interface 01 with one click; the search option is used to call the search function provided by the electronic device to search for the text selected by the user; the translate option is used to translate the text selected by the user, such as from Chinese to English or from English to Chinese; and the share option is used to share the text selected by the user. Any option can be used to respond to a user operation, such as a click operation, causing the electronic device to perform the corresponding operation.

[0199] The navigation bar 014 may include system navigation keys such as a return button 0141, a home screen button 0142, and an outgoing task history button 0143. When it is detected that the user clicks the above-mentioned return button 0141, the electronic device may display the previous user interface of the current user interface. When it is detected that the user clicks the above-mentioned home screen button 0142, the electronic device may display the home screen interface set by the user for the electronic device. When it is detected that the user clicks the outgoing task history button 0143, the electronic device may display the tasks that the user has recently opened. The names of the navigation keys may also be other, for example, the above-mentioned return button may be called Back Button, the above-mentioned home screen button may be called Home button, and the above-mentioned outgoing task history button may be called Menu Button, and this application does not impose any restrictions on this. In addition, the navigation keys in the above-mentioned navigation bar are not limited to virtual buttons, and may also be implemented as physical buttons.

[0200] After the user selects the text, the user can first copy the selected text information by clicking the copy option as shown in (A) of Figure 1. Afterwards, in order to put the copied text information into the excerpt page of the note application, the user clicks the main interface button 0142 as shown in (A) of Figure 1 to make the electronic device exit the current user interface 01 and display the main interface. The specific style of the main interface can refer to the user interface 02 shown in (B) of Figure 1. The user interface 02 may include a status bar 021, a calendar and weather widget 022, an application area 023, a dock area 024, and a navigation bar 025, wherein:

[0201] The specific functions of the status bar 021 and the navigation bar 025 can be referred to the aforementioned description of the user interface 01 and will not be repeated here.

[0202] The calendar weather widget 022 can be used to indicate the current time, such as date, day of the week, hour and minute information, etc., and can be used to indicate the weather type, such as cloudy to sunny, light rain, etc. It can also be used to indicate information such as temperature and can also be used to indicate location.

[0203] The application area 023 may include, for example, an application icon for "Novel Reading" or an application icon 0231 for "Notes," and may also include icons of other applications, which are not limited in this embodiment of the present application. Any application icon can be used to respond to a user operation, such as a touch operation, to enable the electronic device to launch the application corresponding to the icon.

[0204] The dock area 024 may include, for example, an icon for "Phone," an icon for "SMS," an icon for "Browser," and an icon for "Camera," and may also include icons for other applications, which are not limited in this embodiment of the present application. Any application icon may be used to respond to a user operation, such as a touch operation, so that the electronic device launches the application corresponding to the icon. Regardless of how the user switches the screen interface, the size of the dock area 024 and the applications it contains on the screen interface never change and are always displayed on the current interface.

[0205] To place previously selected and copied text information into the excerpt page of the app note, as shown in FIG1(B), the user first needs to open the "Note" app by clicking on the app icon 0231. In response to the user clicking on the app icon 0231, the electronic device may display the user interface 03 shown in FIG1(C). In the user interface 03, the user may first long-press a blank area in the input area 031 to bring up the input option box 032, and then click the paste option in the input option box 032 to paste the previously selected text information into the input area 031.

[0206] After the text information is pasted, the interface displayed by the electronic device can refer to user interface 04 shown in (D) of Figure 1. As shown in user interface 04, the text information selected by the user in the "Novel Reading" application has been successfully entered into the "Notes" application. Subsequently, the electronic device can save the user's newly created note in response to the user clicking the confirmation control 042.

[0207] From the specific process of text cross-application transmission shown in Figure 1, it can be seen that this method of information cross-application transmission requires multiple operation steps when executed, and involves jumping between multiple applications and pages. The interaction efficiency between the user and the electronic device is very low, which seriously affects the user experience.

[0208] FIG2 shows a specific process in which a user uses the split-screen function provided by an electronic device to add a picture in the application “Album” to the application “Notes”.

[0209] As shown in (A) of FIG2 , the user interface 05 includes a first display area 051 and a second display area 052. The first display area 051 is used to display the application interface of the application "Notes", and the second display area 052 is used to display the application interface of the application "Album". Specifically:

[0210] The first display area 051 includes an input area 0511 and a function list 052, wherein:

[0211] The input area 0511 is used to receive and display note information such as text, images, etc. input by the user.

[0212] The function list 052 may include a share option, a favorite option, a delete option, and more options. Any of the options may respond to a user's click operation, causing the electronic device to execute the function corresponding to the option for the current note.

[0213] The second display area 052 includes an image display area and an image thumbnail set 0522, wherein:

[0214] The image display area 0521 is used to display images saved by the user in the gallery;

[0215] Image thumbnail set 0522 is a thumbnail set formed after the user selects multiple images from the image display area 0521 and drags them up (that is, the images that the user currently wants to put into the note). The number "3" displayed in the superscript in the upper left corner indicates that the user has selected three images in total.

[0216] As shown in FIG2(A), a user can press and hold image thumbnail set 0522, drag it to first display area 051, and then release it to add multiple images corresponding to image thumbnail set 0522 to the application "Notes". For details, please refer to user interface 06 shown in FIG2(B). It should be noted that in user interface 06, due to the limited area of ​​first display area 061, although the three images selected by the user (i.e., the three images corresponding to image thumbnail set 0522) have all been added to the application "Notes", they are not fully displayed in first display area 061.

[0217] Although the process of information transmission across applications shown in FIG2 simplifies the user's operation steps to a certain extent, the electronic device needs to keep two application interfaces displayed at the same time during the process, and the user's reading effect is poor; in addition, the user also needs to operate the electronic screen to display the two application interfaces in split screen. The operations required for this process are also very cumbersome, and the interaction efficiency between the user and the electronic device has not been improved, and the user experience is still poor.

[0218] In response to the defects of the information cross-application transmission method in Figures 1 and 2, the present application provides an information transmission method and an electronic device, which can be used to implement the method. When implementing the method, the electronic device can display a cross-application transmission interface, so that the electronic device can efficiently and quickly complete the information transmission between applications, effectively reducing the number of operating steps required for users to complete cross-application information transmission, improving the interaction efficiency between users and electronic devices, and thereby improving user experience.

[0219] Specifically, in any interface displayed by an electronic device, when the user needs to transfer information in the interface across applications or across devices, the user can long press the information element in the interface and drag it to the edge of the screen. The electronic device will transform the interface originally displayed on the screen into three dimensions, creating a visual effect of the original interface being pushed inward on the screen, and display the recommended application service icon on the side of the screen of the electronic device. At this time, the user interface displayed by the electronic device is the cross-application transmission interface. Afterwards, the user places the dragged information element on the application service icon and lets go, and the information can be transferred to the application service corresponding to the icon and the corresponding application service function can be started. It should be noted that the above-mentioned information elements can be elements such as pictures, text, video, voice, links, documents, etc., and can also be other elements, which are not limited in this application. In addition, "cross-application transmission interface" is just some names used in the embodiments of this application. The meanings they represent have been recorded in the embodiments of this application, and their names do not constitute any limitations on this embodiment.

[0220] The following describes the user interface provided by the embodiments of the present application.

[0221] First, the user interface involved in the triggering process of the above-mentioned cross-application transmission interface is introduced.

[0222] FIG3A is a schematic diagram of the process of changing the interface of an electronic device when a user transfers an image across applications based on the cross-application transfer interface.

[0223] As shown in (A) of FIG3A , the user interface 07 may be an application interface of an application “browser” in an electronic device, or an application The application interface of the present invention may be an application interface of another application, which is not limited in this application. The user interface 07 may include an image 071, text 072, text 073, and a status bar 074. The electronic device may respond to a user operation on the image 071, such as a long press operation shown in (A) of FIG. 3A, and display the user interface 08 shown in (B) of FIG. 3A.

[0224] As shown in (B) in FIG3A , in the user interface 08, the original image 071 forms an image 081 under the user's long press operation. If the user does not let go, the image 081 can move with the user's fingertip in the user interface 08, and the image 081 can be displayed on top of the user interface 08 (i.e., the image 081 can cover any element in the user interface 08). It should be understood that the image content displayed by the image 081 is the same as the image content displayed by the image 071. Optionally, the image 081 can be a thumbnail obtained by the electronic device after reducing the image 071, that is, the image 081 and the image 071 have no difference except the size. Optionally, after the image 071 is dragged up to form the image 081, the electronic device can still display the image 071 in the user interface 08 and perform a virtual or blurred processing on the image 071, i.e., the clarity of the image 071 in the user interface 08 can be lower than the clarity of the image 071 in the user interface 07, to prompt the user that the image 071 has been selected and dragged up.

[0225] The electronic device may display the user interface 09 as shown in (C) in FIG. 3A in response to a user operation on the image 081 , such as a drag operation as shown in (B) in FIG. 3A .

[0226] Optionally, when the user drags image 081 to a point where the distance from the right or left edge of the screen is less than a certain threshold, the electronic device displays user interface 09, which will be described in detail later and will not be repeated here. As shown in (C) in Figure 3A, user interface 09 is the cross-application transmission interface mentioned in the above description, which may include sub-interface 091, icon area 092, and image 093. Among them:

[0227] The element content (such as text, pictures, etc.) displayed in the sub-interface 091 is the same as the element content displayed in the user interface 07 (except the status bar). In fact, the sub-interface 091 can be considered as the interface obtained by the electronic device after performing a three-dimensional transformation on all the element contents (except the status bar) of the original user interface 07. In the user interface 09, the sub-interface 091 can create a visual effect on the screen that the original interface (i.e., the user interface 07) is pushed inward. It is understandable that this visual effect of "opening the door" is a metaphor for the behavior of transmitting elements (information), which is more intuitive than copying and pasting. Optionally, in the user interface 09, the background behind the sub-interface 091 can be set to the desktop background of the electronic device (i.e., the desktop wallpaper) with a blurred effect superimposed on it.

[0228] The icon area 092 can be used to display application service icons recommended by one or more electronic devices based on the information element selected by the user (ie, image 071 in the embodiment of the present application).

[0229] Optionally, each icon in the icon area 092 has a corresponding text description below, which can be used to explain the specific application name or application service name corresponding to the icon. When the icon 0921 is an application, the text 0922 can be "WeChat"; when the icon 0921 is an application When the application service icon of the application service "Friends Circle" is used, the text 0922 can be "Friends Circle" or "WeChat Friends Circle". When the application service icon of the application service "Scan" is displayed, the text 0922 can be "Scan" or "WeChat Scan". It should be understood that the text below each application service icon is only for the purpose of helping users understand the specific application or specific application function corresponding to the icon, but this application does not limit the specific icon style and specific text content.

[0230] The image content displayed by image 093 is the same as that displayed by image 071 and image 081. Alternatively, image 093 may be a thumbnail of image 081 reduced by the electronic device. Similarly, image 093 is identical to images 081 and 071 except for their size. This way, when a user selects a desired application service in icon area 092, image 093 will not obstruct the icons and text in icon area 092, making it easier for the user to select the icon.

[0231] It should be noted that, unlike the reduction process of image 071, after image 081 is reduced to image 093, image 081 may no longer be displayed in user interface 09. That is, image 081 can be considered an image obtained by the electronic device copying image 071 and then reducing it, while image 093 can be considered an image obtained by the electronic device directly reducing image 081.

[0232] In the user interface 09, the user places the dragged information element (i.e., image 093) on the corresponding icon in the icon area 092 and lets go. The electronic device can transmit the information element to the application service corresponding to the icon in response to the user operation, and display the application interface or feedback information corresponding to the application service icon. This will be explained in more detail later and will not be repeated here.

[0233] In addition, in order to guide the user to accurately place the dragged element into the application service they need, when image 093 is close to an icon in icon area 092, or when the distance between image 093 and an icon in icon area 092 is less than a certain threshold, the icon approached by image 093 can display certain special effects, such as the icon and the text corresponding to the icon can be attracted and drawn close by image 093, and the icon and the text corresponding to the icon can be enlarged. Taking icon 0923 and its corresponding text 0924 in user interface 09 as an example, when image 093 is close to icon 0923, icon 0923 and text 0924 are further to the left than other icons and text, that is, closer to image 093 than other icons and text; in addition, icon 0923 is also enlarged to remind the user that if they let go here, image 093 (or image 071) will be transferred to the application service corresponding to icon 094.

[0234] It can be understood that the cross-application information sharing process shown in Figure 3A is continuous from the user dragging up the element to the element placing the element into the target application service. The "continuous" here means that the user does not need to perform operations such as copying and pasting, nor does he need to switch applications. The user's fingers do not need to leave the screen of the electronic device during the entire operation process, and the operation is simple and fast. This information transmission method ensures the effective browsing area of ​​the user interface, which can effectively improve the efficiency of the user's cross-application information transmission and enhance the user experience.

[0235] In the aforementioned cross-application transmission method, the electronic device can determine the recommended services in icon area 092. This is not limited to the electronic device recommending services in icon area 092 in the "Smart Flow" function; electronic devices can also recommend services in other scenarios. For example, after receiving an event in which a user copies text, the electronic device can also display an icon area on the display screen to recommend services. For another example, after receiving an event in which a user drags content, the electronic device can also display an icon area on the display screen to recommend services.

[0236] The following uses the copy scenario as an example to introduce a process for an electronic device to implement the recommended service for determining the recommended icon 092 area recommendation in the aforementioned cross-application transmission method. The process may include:

[0237] 1. The electronic device first monitors a copy event, which can trigger subsequent steps. In a specific implementation, the electronic device can determine that a copy event has been monitored when it confirms that there is new content in the clipboard (also known as a cutboard, pasteboard, etc.).

[0238] 2. The electronic device filters the copied content. Filtering refers to identifying and ignoring copied content that is not necessary for service recommendations, retaining only the copied content that is necessary for service recommendations. For example, if the copied content is a verification code, then service recommendations are not necessary, and subsequent steps, such as de-duplication and content refinement, are not necessary.

[0239] 3. Entity recognition.

[0240] An entity refers to an object, concept, or item. Examples of entity types include passwords, addresses, film and television drama titles, dates, product names, food names, email addresses, flight numbers, express delivery numbers, and phone numbers. Each entity contains a specific value, which is the entity's specific content. For example, if a user copies the text "XX City, XX District, XX Road, XX Building, XX Road," the text is an entity whose entity type is address and whose specific content is "XX City, XX District, XX Road, XX Building, XX Road." Another example is a date-type entity whose entity value is a specific date, such as "April 18, 2024."

[0241] The purpose of step 3 is to identify the entity in the filtered copy content and obtain the type and value of the entity.

[0242] If the copied content is a foreign language text, then in step 3, the electronic device may first translate the foreign language text into the local language and then perform entity recognition.

[0243] 4. Entity sorting.

[0244] When multiple entities are identified, the electronic device can sort the multiple entities according to a preset rule. This application does not limit the preset rule, for example, it can be a preset order, the order of identification, etc.

[0245] 5. Get a list of services based on the identified entity.

[0246] Typically, there are preset mappings between different types of entities and service lists. For example, an address entity corresponds to navigation services and taxi services, a flight number entity corresponds to flight query services, and an email entity corresponds to an email application. Based on these preset mappings, electronic devices can identify the service list corresponding to an entity.

[0247] 6. Sort the services in the obtained service list. There are many sorting rules, which are not limited here.

[0248] 7. Display the icons of each service in the menu bar (eg, icon area 092) according to the sorting.

[0249] 8. Service execution, that is, in response to a user operation on a service icon, jumping to the user interface of the service.

[0250] The service recommendation process described above can be executed by a service recommendation engine in the electronic device. The service recommendation process in other similar scenarios is similar to the above process. For example, when a user drags other data (such as screenshots or text), services can also be recommended according to the above process.

[0251] The service recommendation process has the following shortcomings:

[0252] 1. The entire service recommendation process depends on specific events (such as replication events) and will only be triggered after the replication event is detected;

[0253] 2. Different services have different execution methods, lacking unified standards, and unable to reuse engine capabilities. In different scenarios, such as "smart flow" and replication recommendation services, the corresponding services have different implementation standards, which is complex and redundant.

[0254] 3. For different entities, service acquisition, service ranking, and service execution are all customized processes for each entity. The capabilities of electronic devices cannot be reused, resulting in a high workload when making service recommendations.

[0255] 4. If you need to access a third-party service, that is, enable the third-party service to be launched through the menu displayed on the electronic device, then you need to customize the third-party service. Because the execution parameters of each third-party service are different, multiple customizations are required, and the access cycle of the third-party service is too long.

[0256] In addition, there are other ways to implement service recommendations in the existing Android system, but these solutions usually have the following shortcomings: they only support applications that access Android shared content (share intent) and cannot reuse existing services of third-party applications (such as services launched using deep links); they do not support applications that access non-Android ecosystems, such as light apps and Alipay mini-programs; they can only recommend services based on fixed correspondences based on entities, do not take into account the user's personal intentions, and cannot well meet user needs.

[0257] The embodiments of the present application provide a framework for recommending services, which can solve the above problems.

[0258] 3B , which exemplarily illustrates the structure of a framework for recommending services, may include two parts, one located on the electronic device side and the other located on the cloud device side.

[0259] As shown in Figure 3B, the electronic device side may include: a display portal, a software development kit (SDK), a multimodal recommendation service, and a service management framework. The multimodal recommendation service, also known as a multimodal service recommendation engine, may include the following modules: a service subscription module, a multimodal content understanding module, an intent recognition module, a service recall module, a service ranking module, a service notification module, and an intent self-learning module.

[0260] The display entry on the electronic device side can be located in the application layer, and the remaining modules can be located in the framework layer or service layer, which is not limited here. The display entry can refer to the entry of each business that can provide service recommendation functions, such as the entry for copying recommendation services, the entry for dragging content recommendation services in the "Smart Flow" function, which is used to receive user input content, etc. The relationship between the display entry and the interaction methods mentioned above is not limited to the interaction methods mentioned above.

[0261] The SDK is used to provide an interface for the display portal to use the multi-modal recommendation service. The framework shown in Figure 3B defines a unified multi-modal content input interface (i.e., the display portal). The display module display portal can obtain input content from the clipboard and drag events, and then transmit the input content to the SDK, which in turn triggers the subsequent service recommendation execution of the multi-modal recommendation service. In this way, the multi-modal recommendation service does not need to obtain content from the clipboard or drag events, which improves the efficiency of information processing.

[0262] The multimodal recommendation service is used to recommend services for various businesses at the application layer. Specifically, the service subscription module is used to subscribe to the services and / or intent categories supported by each business, the multimodal content understanding module is used to identify entities from the content received by the application layer business, the intent recognition module is used to find the corresponding intent based on the recognized entity, the service recall module is used to find the service corresponding to the intent, the service sorting module is used to sort the services, the service notification module is used to pass the sorted service list to the various businesses at the application layer, and the intent self-learning module is used to learn new intent categories. The specific functions of each module in the multimodal recommendation service can be referred to the specific introduction of the method embodiment below.

[0263] As shown in FIG3B , the cloud device side may include: a decision cloud platform and an intelligent service platform (ISP).

[0264] The decision cloud platform can be a physical server, a cloud server, a server cluster, etc. The decision cloud platform can be provided uniformly by the device manufacturer of the electronic device.

[0265] The decision cloud platform is used to define the following:

[0266] 1. Global Intent System

[0267] The global intent system includes information on multiple types of intent.

[0268] An intent refers to a user's desired goal. Intents can be categorized into various types, including but not limited to unlocking a password, navigation, map search, taxi hailing, shopping, and ordering takeout. An intent category can also be understood as a less specific intent.

[0269] Each type of intent may include one or more of the following information: intent identifier, intent level (intentlevel), intent tag (tag), intent slot information (slotInfoList). Among them, the intent identifier indicates the intent category and may include one or more of the following: intent identification number (identity, ID), intent encoding. The intent level indicates the confidence of the intent. The intent slot information may include one or more of the following: slot name, data type filled in the slot, Chinese name of the data filled in the slot, indication information on whether the slot is necessary, etc. The slot can be regarded as a parameter, and the data type may include string type, number, etc.

[0270] What the decision cloud platform does is to define the information included in multiple types of intents, such as intent identification and intent slot information.

[0271] The following shows an example of information defined by the decision cloud platform:

[0272] "video.WATCH":{},

[0273] "HAIL_TAXI":{"intentLevel":5,"tag":"location","slotInfoList":[{"slotName":"end_name",

[0274] "slotType":"String","srcKey":"location"}]},

[0275] "ADDRESS_SEARCH":{},

[0276] "RECOGNIZE_CODE":{}

[0277] In the above example, the Decision Cloud Platform defines four intent types: video.WATCH (watch a video), HAIL_TAXI (hailing a taxi), ADDRESS_SEARCH (search for an address), and RECOGNIZE_CODE (identification code). The second intent, hailing a taxi, has a level of 5 and a tag of "address." It contains one slot named "destination," a string type, and an entity type of "address."

[0278] Refer to Table 1, which shows the information included in the taxi-hailing intent defined by the decision cloud platform.

[0279] Table 1

[0280] Different types of intent contain different information. In other words, the Decision Cloud Platform standardizes the expression of intent and constructs a database containing information about various intents, essentially building an intent framework.

[0281] If a new intent category needs to be added, the decision cloud platform only needs to define the information included in the new intent category so that the electronic device can recognize the new intent category.

[0282] By filling in specific values ​​in the slots of the intent information, a concrete intent can be obtained. Refer to Table 2, which shows the information contained in a concrete taxi-hailing intent.

[0283] Table 2

[0284] The concrete taxi-hailing intent can be "take a taxi from Shenzhen Honor Technology Park to Shenzhen Airport".

[0285] Reified intents can be categorized as valid or invalid, depending on whether the required slots are filled with values. If the required slots are filled with values, the intent is considered valid; however, whether or not the non-essential slots are filled with values ​​does not affect the validity of the intent.

[0286] 2. Mapping relationship between entity categories and intent categories

[0287] Entity information may include entity category and entity value.

[0288] Entity information can be implemented in the following format: "entity category": ["entity value"]. The following are some examples of entities:

[0289] "code":["Password code"]

[0290] "location": ["Address"]

[0291] "video":["Movie Name"]

[0292] "time":["date"]

[0293] "commodity":["commodity name"]

[0294] "delicacy":["delicacy name"]

[0295] "multiLocation":["Address 1","Address 2"]

[0296] The entity value in the above entity is replaced by the value type, and the actual entity value needs to be filled with a concrete numerical value.

[0297] The mapping relationship between entity categories and intent categories can be pre-defined by the decision cloud platform.

[0298] For example, the mapping relationship may include: if the entity class of the entity and the label in the intent information are the same, then the entity and the intent type correspond. That is, an entity can be mapped to the intent class whose intent label is the type of the entity. For example, the entity "location": ["address"] corresponds to the taxi-hailing intent shown in Table 1.

[0299] This mapping relationship can be configured manually or learned and continuously improved by machines.

[0300] An entity can be mapped to one intent category or to multiple intent categories.

[0301] In some implementations, the decision cloud platform can also predefine mappings between modalities and intent categories. For example, text modality can be mapped to search intent and note intent; video modality can be mapped to playback intent, etc.

[0302] In some implementations, the decision cloud platform may also predefine mapping relationships from entities and modalities to intents, which is equivalent to integrating the above two mapping relationships.

[0303] 3. Service sorting rules in the cold start phase

[0304] The cold start phase refers to the period before the electronic device has learned user preferences. Service ranking rules during the cold start phase can be set based on group preferences. For example, a group's preference order for services might be: ride-hailing, favorites, shopping, and so on.

[0305] The above information configured by the decision cloud platform can be sent to electronic devices. For example, the decision cloud platform can share the information of multiple types of configured intents and the mapping relationship between entity categories and mapping categories with the intent recognition module, and share the service sorting rules in the cold start phase with the service sorting module.

[0306] The Smart Service Platform is open to third-party service providers, allowing them to define or edit information about the services they provide. The information provided by these third parties and obtained by the Smart Service Platform can be referred to as the Global Service System.

[0307] Information about a service defined by a third party may include one or more of the following: service ID, service name, service icon, service type, identifier of the intent categories supported by the service, service slot name, application package name, component name, uniform resource locator (URL) to redirect to the service, and parameter names. The service type refers to the form in which the service is provided, and may include, for example, a widget, deep link, Android share intent, non-Android mini-program / light app / Hypertext Markup Language (HTML) webpage, etc. The intents supported by the service refer to the intent categories that the service can satisfy. This item can be selected by the third party from the intent categories defined by the decision cloud platform, and there can be one or more items. The service slot name indicates the slot used to obtain parameters from the intent when starting the service. This item can be selected by the third party from the slot names of the intent categories supported by the service defined by the decision cloud platform, and there can be one or more items. The application package name refers to the package name of the application to which the service belongs. The component name refers to the name of the component to which the service belongs. The parameter name refers to the parameter used to pass the value of the slot represented by the slot name when the service is executed. Not all of the above items are required, and some of them can be deleted based on actual needs.

[0308] If new services need to be connected to electronic devices, the smart service platform can flexibly expand these new services. Service providers can register these new services in the service management framework according to the standard format required by the smart service platform, and electronic devices can subsequently activate these new services. In this way, service providers do not need to adapt the service code; they only need to register the service information in the standard format to enable service access.

[0309] Refer to Table 3, which shows information about a service defined by a third party through the smart service platform.

[0310] Table 3

[0311] It can be seen that the service described in Table 3 supports the intent category coded as "com.hihonor.intent.navigation".

[0312] For example, if map application A provides a taxi service, in the information of the taxi service, you can specify that the data filled in the slot named "end_name" (for example, "Shenzhen Airport") be transferred to the "keyword" parameter of map application A, so that it is convenient to share data when the service is started later. If map application B also provides a taxi service, then in the information of the taxi service, you can specify that the data filled in the slot named "end_name" (for example, "Shenzhen Airport") be transferred to the "dname" parameter of map application B, so that it is convenient to share data when the service is started later. It can be seen that different services can have different implementation methods when transferring data. It can be seen that the service information provided by different third parties can be different.

[0313] The following exemplifies partial information of the taxi services provided by map application A and map application B respectively.

[0314] app: Map application A

[0315] Application package name:com.autonavi.minimap

[0316] URL: Deeplink:amap: / / navigation

[0317] Parameter: keyword

[0318] app: Map application B

[0319] Application package name:com.autonavi.minimap

[0320] URL: Deeplink:baidu: / / navigation

[0321] Parameter: dname

[0322] After obtaining the service information defined by each third party, the smart service platform can share this service information with the service management framework of the electronic device.

[0323] As mentioned above, the smart service platform standardizes the access methods of services and generates a database containing various service information based on the services accessed by each party. This is equivalent to mapping the differentiated execution interfaces and parameters of different services (including system services and third-party services) to the business side through the custom rules provided in the embodiments of this application, so that the business side can run various services through a unified standard interface.

[0324] The following will explain the role of each module in the above framework in conjunction with the specific process of the recommendation service.

[0325] As shown in FIG3B , the process of the recommendation service provided by the embodiment of the present application may include the following steps:

[0326] Step S1. The first service subscribes to supported intent categories and / or services.

[0327] The first service referred to here refers to a service with the function of recommending services, such as a smart flow service in an electronic device, a service that recommends services by copying, a service that recommends services by dragging and dropping, etc.

[0328] The first service can initiate a subscription to the service subscription module to subscribe to the intent category range and / or service range required by the first service. The intent category and / or service subscribed by the first service can be one or more. After the first service is started, the electronic device can identify intent categories within the intent category range subscribed by the first service, and can also recommend services within the service range subscribed by the first service based on the identified intent.

[0329] Different first services can subscribe to different intent categories and / or service scopes to meet the personalized needs of different service providers. A first service can dynamically subscribe to intent scopes and service scopes based on its own needs, enabling the first service to dynamically expand the intent categories and / or services it supports. The specific intent category and service scope subscribed to by the first service can be determined by the developer of the electronic device.

[0330] The following table shows an example of the service scope and intent category scope of a first service subscription:

[0331] List of subscribed services:

[0332] ["Taxi service","Navigation service",

[0333] "Search Service", "Global Collection Service"]

[0334] List of subscribed intent categories:

[0335] ["social.SHARE",

[0336] "email.SEND",

[0337] "office_software.PRINT",

[0338] "note.TAKE",

[0339] "HAIL_TAXI",

[0340] "favorites.ADD",

[0341] "schedule.CREATE",

[0342] "video.WATCH",]

[0343] The above service list contains the codes of the subscribed services, indicating that the subscribed services include the following: taxi service, navigation service, search service, and global favorites service.

[0344] The preceding intent category list contains the codes for the subscribed intent categories, indicating that the subscribed intent categories include the following: social sharing intent, email sharing intent, printing intent, note-taking intent, taxi-hailing intent, collection intent, trip creation intent, and video watching intent.

[0345] After a first service subscribes to an intent category and service, if the electronic device activates the first service, the multi-mode recommendation service in Figure 3B will pre-load the required resources, such as information about the subscribed intent category, based on the intent categories and services subscribed to by the first service. The multi-mode recommendation service can also notify the first service to pre-load static resources such as icons for the subscribed services. This can improve the efficiency of service recommendations.

[0346] The purpose of step S1 is to subscribe the first service to the supported intent categories and / or services, so that the electronic device can find the corresponding intent category from the intent categories subscribed to by the first service in the subsequent step S4, and to facilitate the electronic device to query the corresponding service from the service scope subscribed to by the first service in the subsequent step S5. Therefore, S1 can be executed before S4 and S5, and this application does not specifically limit its execution time.

[0347] Step S1 is an optional step and may not be performed in some implementations. For example, each service may be subscribed to all intent categories and / or all services supported by the electronic device by default, without requiring each service to actively subscribe.

[0348] Step S2: The electronic device receives first content input by the user for the first service.

[0349] The electronic device may receive the first content input by the user through the portal provided by the first service.

[0350] S2 can be executed when an electronic device starts the "smart circulation" service.

[0351] For example, referring to Figure 3A, after the user long presses image 071, the electronic device generates image 081. When the user drags image 081 to a distance less than a certain threshold from the right or left edge of the screen, S1 can be triggered. In this case, image 071 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0352] For another example, referring to the subsequent Figure 4, after the user long presses the text 111 in the user interface 11, the electronic device generates text 121. When the user drags the text 121 to a distance less than a certain threshold from the right or left edge of the screen, S1 can be triggered. In this case, the text 111 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0353] For another example, referring to the subsequent Figure 6, the user long presses the screenshot image 171 in the user interface 17 and then drags the screenshot image 171. When the screenshot image 171 is dragged to a distance less than a certain threshold from the right edge or the left edge of the screen, S1 can be triggered. In this case, the screenshot image 171 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0354] For another example, referring to the subsequent Figure 7, the user can click the identification control 203 in the user interface 20 to trigger the electronic device to identify the text information in the local screenshot box 202. After that, the user can long press the identified text information and drag the text information. When the text information is dragged to a distance from the right or left edge of the screen less than a certain threshold, S1 can be triggered. In this case, the text information is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0355] For another example, referring to the subsequent Figure 8, the user can long press and drag the image captured by the partial screenshot box 202. When the image is dragged to a distance less than a certain threshold from the right or left edge of the screen, S1 can be triggered. In this case, the image captured by the partial screenshot box 202 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0356] For another example, referring to the subsequent Figures 22C-22D and 22F-22G, the user can long press the target text 111 to trigger the electronic device to generate the corresponding control 112. After the user drags the control 112 to the right edge of the screen or the left edge of the screen, S1 can be triggered. In this case, the target text 111 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0357] For another example, referring to the subsequent Figures 25A-25F, the user can long press the image 411 generated by the screenshot and drag the image 411 to the right edge of the display screen or the left edge of the screen to trigger S1. In this case, the image 411 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0358] For another example, referring to the subsequent Figures 26A-26I, the user can long press the display screen to trigger the electronic device to display a floating control 512. The user can then drag the control 512 to the door opening hot zone on the left or right side of the screen to trigger S1. In this case, the image corresponding to the control 512 is the first content received by the electronic device through the entrance of the "Smart Flow" service.

[0359] S2 can also be executed when the electronic device initiates other services other than "Smart Flow". For example, after receiving an operation in which the user copies text, the electronic device can execute S2. The text copied by the user is the first content received by the electronic device through the entrance of the copy service. For another example, after receiving an operation in which the user drags content (such as text, images, etc.), the electronic device can execute S2. The content dragged by the user is the first content received by the electronic device through the entrance of the drag service.

[0360] The first content input by the user mentioned here is the same concept as the data to be transmitted and the information to be transmitted mentioned above and below. The first content input by the user can be unimodal or multimodal. Modality can refer to the category of content, which can include text, pictures, videos, documents, etc.

[0361] Step S3. The electronic device calls the multimodal content understanding module in the multimodal recommendation service to convert the received content into a first entity.

[0362] The multimodal content understanding module can call entity recognition capabilities to convert the received content into a first entity. For different types of input content, the electronic device can call different entity recognition capabilities to extract the features of the content and convert it into entities. For example, for text, natural language processing (NLP) can be used to extract its features; for pictures, computer vision (CV), optical character recognition (OCR) and other capabilities can be used to extract its features. Entity recognition capabilities may include local capabilities of electronic devices, as well as recognition capabilities on the cloud side. Entity recognition capabilities can be provided based on a multi-channel recognition framework of electronic devices. If there is a new entity recognition capability, the entity recognition capability of the electronic device can be expanded through a plug-in so that the electronic device can use the new entity recognition capability to identify entities.

[0363] The entities of this application can be represented in a standard format, that is, including entity categories and entity values. Step S3 is equivalent to the electronic device obtaining the entity category and value based on the received content. For the definition of entity categories and entity values, please refer to the relevant description above. In the embodiment of this application, the entity category can be selected within the range of entity categories supported by the electronic device. That is, the entities converted by the electronic device all belong to the entity category within the preset range.

[0364] There may be one or more first entities converted from the electronic device, and the categories of the multiple first entities may be different.

[0365] If there is a new modality, the multimodal content understanding module of the multimodal recommendation service in Figure 3B can add a plug-in that defines the information of the new modality. In this way, the new modality can be quickly expanded through the plug-in method, allowing the electronic device to analyze the content of the new modality.

[0366] Step S4. The electronic device calls the intent recognition module in the multimodal recommendation service to identify the first intent corresponding to the first entity.

[0367] The first intention is the intention of embodiment.

[0368] Specifically, the electronic device can first find the intent category corresponding to the first entity in the intent category defined by the decision cloud platform based on the mapping relationship between entity categories and intent categories, which can be called the first intent category. The mapping relationship between entities and intents can refer to the previous introduction. Then, the electronic device can fill in the slot of the first intent category with a specific value to obtain the concrete first intent. The value filled in the slot can be the value of the first entity, or it can be obtained by the electronic device through other methods such as positioning and learning.

[0369] The first intent may be defined as a valid intent, that is, the electronic device recognizes a valid intent corresponding to the first entity. The definition of a valid intent may be referred to above.

[0370] For example, assuming that the first entity is: "multiLocation":["Shenzhen Glory Technology Park","Shenzhen Airport"], the first intent category corresponding to the first entity can be as shown in Table 2 above, and the obtained concrete first intent can be as shown in Table 2.

[0371] In some implementations, if step S1 is executed, the electronic device can find the intent category corresponding to the first entity in the intent category of the first service subscription, and further find the concrete first intent.

[0372] In some embodiments, the electronic device can also find a corresponding first intent category based on the modality of the first content. The correspondence between modalities and intent categories is usually set according to the habits of group users. For example, the text modality can correspond to copy intent, search intent, and translation intent; the image modality can correspond to the sharing to social application intent; the video module can correspond to the play intent, etc.

[0373] The number of first intent categories corresponding to the first entity may be one or more. The number of first intents corresponding to the first entity may be one or more.

[0374] In steps S2-S4 above, the electronic device first obtains the input content, then converts the input content into entities, and then identifies the intent corresponding to the entities. During this process, the user input content may include multiple modalities, and the entities converted from the input content may also include multiple entities.

[0375] The important steps in the process of identifying multimodal content include the following:

[0376] Load intent metadata and business rules (also known as information fusion rules). Intent metadata refers to information in the global intent system, such as the encoding and slots of various intents. Business rules may include mappings between entity categories and / or modalities and intent categories, content in the global service system, etc. The loaded intent metadata and business rules will be used in subsequent steps. For details, please refer to the relevant descriptions of S3-S4 in Figure 3B above.

[0377] Determine the modality type and load various entity recognition capability models. Entity recognition capability models may include but are not limited to NLP models and CV models.

[0378] NLP text recognition and CV image recognition: This step actually involves serially identifying entities in content of different modalities.

[0379] The above process has the following shortcomings:

[0380] 1. After receiving multimodal input, electronic devices serially load the required resources. This leads to time-consuming entity and intent recognition, a lengthy service recommendation process, and a poor user experience.

[0381] 2. For different modalities, entity recognition and intent recognition are separate processes, resulting in low efficiency in recommendation services when electronic devices receive multimodal content.

[0382] 3. For multimodal content, if the entity recognition time process of one of the modes causes anomalies in the business currently started by the electronic device (such as smart flow service).

[0383] This application provides several performance optimization methods to solve the above problems respectively.

[0384] The performance optimization method provided in this application may be used to replace S3-S4 in FIG. 3B , and the method may include:

[0385] S11. Resource preloading.

[0386] This step can be performed after S2 in Figure 3B, that is, after the electronic device receives the first input content, it can perform resource preloading. For example, in the smart flow service, after the electronic device generates a floating element, if the electronic device receives an operation of the user starting to drag the floating element, the electronic device can start performing resource preloading.

[0387] The preloaded resources here may include, but are not limited to, intent metadata, business rules, and various entity recognition capability models. For the definitions of these resources, please refer to the previous section.

[0388] Electronic devices can load various resources in parallel.

[0389] Taking loading intent metadata as an example, the electronic device can first determine whether the latest version of intent metadata exists locally. If not, it requests the latest version of intent metadata from the cloud side (such as the decision cloud platform) and then loads the intent metadata into the cache.

[0390] Taking loading business rules as an example, the electronic device can first determine whether the latest version of the business rules exists locally. If not, it can request the latest version of the business rules from the cloud side (such as the decision cloud platform, smart service platform, etc.), and then load the business rules into the cache.

[0391] Taking loading an NLP model as an example, the electronic device can first determine whether the NLP model is initialized. If not, it notifies the artificial intelligence plug-in (such as AIplugin) to load the NLP model, and then the artificial intelligence plug-in can load the NLP model.

[0392] Taking loading the CV model as an example, the electronic device can first determine whether the CV model is initialized. If not, it notifies the artificial intelligence plug-in (such as AIplugin) to load the CV model, and then the artificial intelligence plug-in can load the CV model.

[0393] By preloading relevant resources, the latency for electronic devices to recognize intent can be shortened, improving the user experience. Experiments have shown that preloading resources can reduce latency by an average of 20% to 30%.

[0394] S12. Feature extraction.

[0395] This step corresponds to S3-S4 in FIG. 3B , that is, the purpose of S12 is to identify the entity corresponding to the user input content and to identify the intent corresponding to the entity.

[0396] For multimodal content input by the user, the entity recognition capability corresponding to each modality is concurrently called for each modality to extract features of the modal content. The entity recognition capability corresponding to each modality can be defined according to general rules. For example, the entity recognition capability corresponding to the text modality may include location information extraction, schedule information extraction, film and television entity extraction, product entity extraction, and password entity extraction. The entity recognition capability corresponding to the image modality may include image recognition capability and OCR recognition capability. Electronic devices can launch multiple threads, each thread is used to call a different entity recognition capability.

[0397] Specifically, if the user inputs content that includes both text and images, the text semantics and image information are extracted simultaneously. For text semantics extraction, the electronic device can launch multiple threads to extract different features from the text; for image information extraction, the electronic device can also launch multiple threads to extract different features from the image.

[0398] After extracting features from each modality, these features can be converted into entities, for example, entity categories and entity values ​​in a standard format. The electronic device can then identify the intent corresponding to each entity. Multiple modalities can be run simultaneously, and the above process can be repeated for each modality to determine the corresponding intent. After identifying multiple intents, they can be fused to obtain the final intent.

[0399] By extracting features concurrently from multiple channels, we can effectively reduce the overall business congestion caused by anomalies in the extraction of single modal information and improve business stability.

[0400] In some implementations, when an electronic device recognizes a concrete intent, it can terminate any threads currently in progress for identifying the entity. This shortens the latency of identifying the intent and prevents the overall service from being blocked by a single thread running too long. For example, an electronic device receives content in both text and image modes and uses two threads to extract features and identify intent from the text and image modes, respectively. When the thread corresponding to the text content recognizes a specific intent from the text content, it can interrupt the other thread to improve the overall service response efficiency.

[0401] Step S5. The electronic device calls the service recall module in the multi-mode recommendation service, and queries the service corresponding to the first intention based on the service information provided by the service management framework.

[0402] Specifically, the electronic device can search for the service information provided by the service management framework and query the service that matches the identifier of the first intent category. If the identifier of the intent category supported by the service is the same as the identifier of the first intent category, it means that the service can satisfy the intent belonging to the first intent category, that is, it can satisfy the first intent. For example, if the first intent is as shown in Table 2, then the encoding of the intent category supported by the service represented by Table 3 is consistent with the intent encoding of Table 2, then the service represented by Table 3 is considered to match the intent represented by Table 2.

[0403] In some implementations, if step S1 is executed, that is, the first service in the electronic device subscribes to a service scope, then when the electronic device executes step S5, it can query the service corresponding to the first intent in the service scope subscribed by the first service.

[0404] The number of services corresponding to the first intent can be one or more, which is not limited here.

[0405] Step S6. The electronic device calls the service ranking module in the multi-mode recommendation service to rank the multiple services found.

[0406] If multiple services are found in step S5, step S6 may be executed. The electronic device may use various rules to sort the multiple services, such as prioritizing the most recently used services, prioritizing the most frequently used services, prioritizing services with a high confidence level (i.e., a high level) corresponding to the intent category, and the user's application preference order, etc., which are not limited here.

[0407] Step S6 is an optional step. In some implementations, it is not necessary to sort the multiple services.

[0408] Step S7: The electronic device calls the service notification module and returns the acquired service information to the first service.

[0409] The electronic device can return information about multiple services to the first service in the form of a list. If there is a sort, the electronic device can return the sorted service information to the first service so that the first service can know the sort. For information about the service, please refer to the relevant introduction above. In addition, the electronic device will first determine the slot name of the service, then find the value of the same slot name in the first intent, and synchronously return the value to the first service. This makes it convenient for the electronic device to quickly start the first service. Please refer to the subsequent steps for details.

[0410] For example, if the first intent is as shown in Table 2 and the service is as shown in Table 3, the electronic device can return the value of the slot named "end_name" in the first intent (ie, "Shenzhen Airport") to the first business along with the service information.

[0411] Step S8. The electronic device displays the icon of the service obtained in step S7 on the user interface provided by the first service.

[0412] If the ranking information is obtained in step S7, the electronic device can display the icons of the obtained services in order from top to bottom according to the ranking. Here, the content displayed by the electronic device can refer to the various UI embodiments described above and below.

[0413] For example, the electronic device may display the icon of the service obtained by S7 in the icon area 092 of the user interface 09 of Figure 3A, or display the icon of the service obtained by S7 in the icon area 132 of the user interface 13 of Figure 4, or display the icon of the service obtained by S7 in the chart area 182 of the user interface 18 of Figure 6, or display the icon of the service obtained by S7 in the chart area 222 of the user interface 22 of Figure 7, or display the icon of the service obtained by S7 in the chart area 232 of the user interface 23 of Figure 8, or display the icon of the service obtained by S7 in the menu bar 116 of Figures 22H-22K and 22M, etc.

[0414] Step S9: The electronic device receives a user operation on the icon of the first service, and in response to the user operation, the first service shares the first content with the first service.

[0415] For example, referring to Figure 3A, after the electronic device receives the user operation of dragging image 081 onto icon 0923 and releasing it, it transfers image 071 to the application service corresponding to icon 0923. For another example, referring to Figure 8, after the electronic device receives the operation of dragging image 233 onto the icon in icon area 232 and releasing it, it transfers image 233 to the application service corresponding to the icon. For another example, referring to Figure 13, after the electronic device receives the operation of dragging image 372 onto the icon in icon area 372 and releasing it, it transfers image 373 to the application service corresponding to the icon. For another example, referring to Figure 17, after the electronic device receives the operation of dragging image 441 onto the "Global Favorites" icon and releasing it, it transfers image 441 to the "Global Favorites". For another example, referring to Figure 19, after the electronic device receives the user operation of dragging text 521 onto the "Map Navigation" icon 522 and releasing it, it transfers text 521 to "Map Navigation". For another example, referring to Figure 21 , after receiving a user operation of dragging and dropping image thumbnail collection 612 onto the "Friends WeChat" application service icon, the electronic device shares all images contained in image thumbnail collection 612 with WeChat friends. For another example, referring to Figure 22M , the electronic device 100 may detect a user operation of dragging and dropping control 112 onto a note application icon, and write target text 111 into the note application.

[0416] Referring to step S7, the first service of the electronic device can start the service according to the information of the service, for example, find the application package name of the service to start the application to which the service belongs, and then jump to the service according to the jump URL of the service, and then pass the obtained slot value of the service to the parameters specified by the service, so that the first content can be shared to the service.

[0417] For example, if the user selects the taxi service of map application A, the electronic device will pass the address entered by the user, "Building xx, No. xx, xx Road, xx District, xx City", to the taxi service through the parameter "keyword". After the electronic device starts the taxi service, it will display the taxi page with the destination filled in with the address.

[0418] In the process shown in Figure 3B, the modules called in sequence are the first service entry, the multimodal content understanding module, the intent recognition module, the service recall module, the service ranking module, and the service notification module. After step S2, the electronic device transmits the user input content from the first service to each of the above modules. In step S7, the service notification module returns the obtained service information to the first service.

[0419] The first business transmits request parameters to the multi-mode content understanding module. The request parameters may include caller information (such as the name of the first business, the source application of the first business), input content (such as the type of content, the content itself, the type and path of the file where the content is located), and service screening rules (such as service type, installation status of the type of service, and available status of the application to which the service belongs). Among them, the service screening rules can be used for the service recall module to query the service in step S5. The queried service must meet the screening rules. For example, the type of the queried service must be consistent with the service type indicated by the screening rules, the installation status of the application to which the queried service belongs must be consistent with the installation status indicated by the screening rules, and the available status of the application to which the queried service belongs must be consistent with the available status indicated by the screening rules. The service screening rules can be specified by the first business that initiates the request.

[0420] The service information returned by the service notification module can be found in the relevant introduction of step S7.

[0421] Through the above introduction, we can know the functions of each module on the electronic device side in the framework shown in FIG3B .

[0422] FIG3C exemplarily shows another partial structure of the framework for recommending services.

[0423] As shown in Figure 3C, the framework may also include a smart cloud, which can be located on the cloud device side. The smart cloud is used to define capability exposure policies and compile capability usage reports. Capability exposure policies indicate the recommended strategy for exposing services in electronic devices, while capability usage reports provide statistics on the frequency of each capability used in electronic devices, such as the frequency of use of each service and entity recognition capability.

[0424] The multi-mode recommendation service for electronic devices may also include a capability exposure management module, a capability call management module, an authentication / flow control / verification module, a query module, a subscription module, a service distribution module, a subscription status management module, and a trigger notification module. These modules are used to assist in the use of the various modules of the multi-mode recommendation service in the framework shown in Figure 3B.

[0425] The multiple modules in the multi-mode recommendation service shown in Figure 3C can be located in the upper layer or on the same side of each module in the multi-mode recommendation service in Figure 3B, and are used to encapsulate each module in the multi-mode recommendation service in Figure 3B into a unified interface for calling by the business side.

[0426] The framework shown in Figure 3B includes an intent self-learning module, which actively learns new intent categories and synchronizes these learned new intent categories with the decision cloud platform. The following describes the functions of this module.

[0427] As can be seen from step S4 of FIG. 3B , the electronic device can determine the intent based on the modality and entity category of the input content.

[0428] The mapping relationship between modalities and / or entity categories and intents is usually pre-established based on manual experience, which has the following problems:

[0429] 1. The intents obtained through manually formulated mapping rules are not in order, making it difficult to quickly find the services users need.

[0430] 2. Manually formulated mapping rules may be incomplete and may fail to find the services required by users;

[0431] 3. The user's true intention may be outside the predefined intent categories of the decision cloud platform. Manually formulated mapping rules cannot discover the user's new intention and cannot find the services the user needs.

[0432] This application provides several methods for self-learning of intent in the process of recommending services on electronic devices, which can respectively solve the above-mentioned problems.

[0433] FIG3D exemplarily shows the overall process of the method. As shown in FIG3D , the process may include:

[0434] S21. Intent category recall.

[0435] The purpose of this step is to identify the corresponding intent category from the user input. The identification methods can include the following:

[0436] (1) Identify the associated intent category based on the content.

[0437] Electronic devices can understand the semantics of content and identify associated intent categories based on that semantics. For example, they can identify the entity corresponding to the content and, based on the entity category, find intent categories labeled with that entity category. For example, if the input content is the address "Building xx, No. xx, xx Road, xx District, xx City," the semantic understanding can identify associated navigation intents and taxi-hailing intents.

[0438] (2) Identify the corresponding general intent category based on the modality of the input content.

[0439] Electronic devices can also identify intent categories based on the input modality. For example, if the input modality is text, search intent and note-taking intent can be identified. If the input modality is video, playback intent can be identified. The mapping between modality and intent category can be manually determined based on experience.

[0440] (3)Category of catch-all intention.

[0441] Fallback intent categories can be considered frequently used intent categories and can be predefined. For example, for text input, predefined intent categories can include taxi-hailing intent, navigation intent, and takeout ordering intent. If the first two recognition methods have already identified taxi-hailing intent and navigation intent for the text input, then the takeout ordering intent is considered the fallback intent category.

[0442] S22. Ranking of intent categories.

[0443] This step can merge and sort the multiple intent categories identified in the previous step.

[0444] The sorting rules may be preset rules, for example, the electronic device may pre-store the order of the defined intent categories. The sorting rules may also be determined according to the frequency of use by the user, for example, the electronic device may put the intent categories with higher user use frequency at the front. For another example, the sorting rules may be to sort in sequence according to the above three identification methods, for example, the intent category identified by the (1) identification method is ranked highest, the intent category identified by the (2) identification method is ranked in the middle, and the intent category identified by the (3) identification method is ranked last. For another example, the sorting rule may be that the higher the level of the intent category, the higher the ranking.

[0445] In some implementations, the electronic device can rank intent categories based on a click-through rate prediction model. The click-through rate prediction model takes one or more of the following features as input: time features and features of the user input content, and outputs a ranking order for the intent categories defined in the decision cloud platform. This ranking order can reflect the estimated click-through rate order of the intent categories, influenced by the time of the user input content and the content itself.

[0446] The click-through rate prediction model can be trained based on multiple sets of historical behavior data of users. Each set of historical behavior data is analyzed from multiple samples. A sample refers to the information of whether the user clicks (i.e., chooses to select) a service after the electronic device recommends the service to the user. In the sample, a label can be used to indicate whether the user has clicked the intention. It should be noted here that the user clicked on the icon of the service recommended by the electronic device. The service corresponds to a supported intent category, so it can also be considered that the user clicked on the intent category.

[0447] A set of historical behavior data may include the following items:

[0448] 1. Time feature: used to indicate whether the time when the user inputs the content is a weekday, whether it is a holiday, and one or more of the specific time period of the input content.

[0449] 2. Content features: used to indicate one or more of the source app, length, entity category, modality, etc. of the user input content.

[0450] 3. Click-through rate feature: This represents the click-through rate of each intent category by a group when using the recommendation service, while meeting the aforementioned time and content characteristics. This feature can be collected from the Internet. It also represents the click-through rate of each intent category by an individual (e.g., the current user of an electronic device) when using the recommendation service, while meeting the aforementioned time and content characteristics. The above group click-through rate and individual click-through rate can be combined.

[0451] During the training process, the electronic device can use time features and content features as input and click-through rate features as standard output to train the click-through rate prediction model. After training with multiple sets of historical behavior data, a usable click-through rate prediction model can be obtained.

[0452] The training process can also be performed by the cloud side. For example, each electronic device can upload historical behavior data to the cloud decision platform. After the cloud decision platform trains the click-through rate prediction model, the click-through rate prediction model is sent to each electronic device.

[0453] The trained click-through rate prediction model can then proceed to step S22. Specifically, the electronic device can extract the time features and content features of the current user input content and input these two features into the click-through rate prediction model. The click-through rate prediction model can then output the order of the defined intent categories. The electronic device can then use this order to obtain the order of the intent categories identified in S21.

[0454] Step S22 can be the answer to the first question mentioned above. After the electronic device sorts the intent categories, subsequent steps can recommend services to the user according to the sorted intent categories, so that the recommended services that are closer to the top are more likely to meet the user's actual intent, allowing the user to find the services they need more conveniently and quickly.

[0455] S23. Service sorting.

[0456] The electronic device may first refer to step S5 in FIG3B to query the services corresponding to the intent, and then refer to step S6 to sort the multiple services found. The difference is that in the process shown in FIG3D , the electronic device may sort the services according to the sorting of the intent categories in S22.

[0457] For example, the electronic device may obtain the services corresponding to each intent category in S22, roughly sort these services according to the order of intent categories, and then further sort the services in each intent category according to the strategy in S6.

[0458] S24. The electronic device displays the sorted service icons.

[0459] S25. The electronic device obtains the user's behavior data after a service is recommended and the user does not click on the icon of the recommended service.

[0460] After the electronic device recommends a service, if the user does not click the icon of the recommended service, it means that the service recommended by the electronic device cannot meet the actual needs of the user. Therefore, the electronic device needs to improve the preceding steps of recommending the service.

[0461] The reasons why the services recommended by electronic devices cannot meet the actual needs of users and the corresponding solutions may include the following two:

[0462] 1. The predefined mapping relationships of the decision cloud platform do not include the mapping relationship between the entity category and / or modality of the user input content and a certain intent category. In this case, even if the decision cloud platform defines the intent category, the electronic device cannot recognize the intent category from the current input content and cannot recommend the services required by the user.

[0463] To solve this problem, electronic devices learn the mapping relationship between entity categories and / or modalities and defined intent categories based on the user's historical behavior data as a supplementary recall of the mapping relationship.

[0464] The user's historical data includes multiple groups, and each group of behavioral data may include: the user inputs a certain category of entities (such as the first entity category) and / or the content of a certain modality (such as the first modality), and the user manually opens the service within a period of time after the electronic device recommends the service, without clicking the icon of the recommended service. The period of time can be set according to actual needs, for example, it can be 5 minutes. The above period of time can be calculated from the time when the electronic device turns off the first service, for example, it can be calculated from the time when the recommended service is stopped.

[0465] When the services recommended by the electronic device cannot meet the needs of the user, the user may manually open the services he needs, so the above historical data can be used to supplement the mapping relationship between entity categories and / or modalities to defined intent categories. Specifically, the electronic device can count the K services in the user's historical data that are manually opened by the user and are supported by the service management framework and opened manually by the user with the highest frequency, and confirm that there is a mapping relationship between the intent categories corresponding to these services and the entity categories and / or modalities of the user input content, and the mapping relationship can be recalled as supplementary information. K can be set as needed and is not limited here. When counting services, it is not necessary to count services supported by the service management framework, and statistics can also be counted across the entire service range.

[0466] In a specific example, the electronic device first counts the historical data after the user inputs the video, that is, the sequence of behaviors of the user manually opening the service after the user inputs the video to the electronic device and does not click on the icon of the recommended service. The entity category and modality of the video are both video. Afterwards, the user manually opens services such as Meituan Group Buying, Taobao Paili Tao, Baidu Search, Meituan Xiuxiu, and JD Photo Buying. Then, the electronic device filters out the services supported by the service management framework from the various services manually opened by the user, which may include Meituan Group Buying, Baidu Search, Taobao Paili Tao, and JD Photo Buying. The filtered services are then sorted by the frequency of opening, and the K services with the highest opening frequency in the above process are found. It is determined that there is a mapping relationship between the video entity category and / or video modality and the intent category supported by the K services, and the mapping relationship is added to the local mapping relationship of the electronic device.

[0467] In some implementations, the electronic device may also upload the mapping relationship learned locally to the decision cloud platform, so that the decision cloud platform can count the mapping relationships learned by multiple electronic devices to make the mapping relationship more complete.

[0468] The first solution mentioned above addresses the situation where the mapping relationship from entity categories and / or modalities to intent categories is incomplete. It proposes a mapping relationship mining method to mine the mapping relationship from subsequent behaviors that were not clicked after the recommendation, thereby supplementing the recall of the mapping relationship.

[0469] 2. The intent categories predefined by the decision cloud platform do not include the intent category to which the user's actual intent belongs. Therefore, electronic devices cannot recognize this intent category and cannot recommend the services required by the user.

[0470] To solve this problem, electronic devices can learn new intent categories that are not defined by the decision cloud platform based on the user's historical behavior data, and learn the mapping relationship between entity categories and / or modalities to the new intent categories as a supplement to the intent categories and the mapping relationship.

[0471] For the user's historical data, please refer to the relevant description in point 1 above.

[0472] Specifically, the electronic device can first filter out the services that are not supported by the service management framework opened manually by the user and support input content (such as supporting the storage of text, pictures, files, etc.) from the user's historical data, and then filter out the N services that are most frequently opened manually by the user from these services. Then, the electronic device can define a new intent category based on the N services as a supplement to the intent category. The N services can meet the intentions belonging to the new intent category. In addition, the electronic device can also recall the new intent category as a supplement to the above mapping relationship. N can be set as needed and is not limited here. When counting services, it is not necessary to count services that are not supported by the service management framework, and statistics can also be counted across the entire service range.

[0473] In a specific example, an electronic device first collects historical data after a user inputs a video, specifically the sequence of actions taken by the user to manually open services after inputting the video, without clicking on the recommended service icon. The entity category and modality of the video are both video. Subsequently, the user manually opens services such as Meituan Ticketing, Tao Piaopiao, Douban Movie Reviews, Wenxin Yiyan, and WeChat Reading. The electronic device then filters out services that are not supported by the service management framework but support input content, such as Meituan Ticketing, Tao Piaopiao, Douban Movie Reviews, and Wenxin Yiyan. The filtered services are then sorted by opening frequency to identify the N most frequently opened services, which may include, for example, Meituan Ticketing and Tao Piaopiao. The electronic device then defines a new intent category based on these N services, such as an intent category for purchasing movie tickets. The electronic device can then add the intent category for purchasing movie tickets to the predefined intent categories. The electronic device can also determine a mapping between the video entity category and / or the video modality and the intent category for purchasing movie tickets, and add this mapping to the electronic device's local mapping. In some embodiments, the smart service platform can also obtain information about the above N services defined by a third party, so as to facilitate the subsequent sharing of information about these services with the service management framework of the electronic device, so that the electronic device can find these services based on the supplemented new intent categories.

[0474] In some embodiments, the electronic device can also upload new intent categories and mapping relationships learned locally to the decision cloud platform, so that the decision cloud platform can count the intent category mapping relationships learned by multiple electronic devices to make the mapping relationships more complete.

[0475] The second solution mentioned above, aimed at the situation where the user's actual intention exceeds the scope of the intent categories defined by the decision cloud platform, proposes a new intent category mining method, which mines potential undefined new intent categories from subsequent behaviors that are not clicked after the recommendation, and defines the mapping relationship between the new intent category and the entity category and / or modality, thereby supplementing the recall of intent categories and mapping relationships.

[0476] FIG4 is a schematic diagram of the interface change process of the electronic device when the user transfers text across applications based on the “cross-application transfer interface”.

[0477] As shown in (A) of FIG4 , the user interface 11 may be an application interface of an application “browser” in an electronic device, or an application The user interface 11 may include text 111, text 112, image 113, and text option list 114, wherein:

[0478] Text 111 is the text that the user wants to transfer to other applications, so this part of the text has been selected by the user and is in the dark area; correspondingly, text 112 is the unselected text. It should be understood that, unlike the independence of pictures, text itself has continuity. For a whole paragraph of text, the user may often only need a part of the text in the whole paragraph when sharing the text. Therefore, although not shown in Figure 4, when selecting text in the user interface 11, the user can perform a long press operation on any text information in the user interface 11, and after the first cursor 111A and the last cursor 111B appear, complete the selection of the text by adjusting the position of the first cursor 111A and the last cursor 111B. As shown in (A) in Figure 4, the user has selected text 111 by adjusting the position of the first cursor 111A and the last cursor 111B.

[0479] The text option list 114 includes a copy option, a select all option, a search option, a translation option, and a share option. Any of the options can be used to respond to a user's operation, such as a click operation, so that the electronic device performs the operation corresponding to the option.

[0480] Like the operation of dragging the picture, the electronic device can also respond to the user's operation on the text 111, such as the long press and drag operation shown in (A) of FIG4, and display the user interface 12 shown in (B) of FIG4.

[0481] As shown in (B) in FIG4 , in the user interface 12, the original text 111 is lifted up by the user's long press operation. If the user does not let go, the text 121 can move with the user's fingertip in the user interface 12, and the text 121 can be displayed on top of the user interface 12 (i.e., the text 121 can cover any element in the user interface 08). It should be understood that the text content contained in the text 111 is the same as the text content contained in the text 121. Optionally, the text 121 can also be the text obtained after the electronic device shrinks the text 111, and after the text 111 is dragged up to form the text 121, the electronic device can still display the text 111 in the user interface 12 and perform a virtual or blurred processing on the text 111, i.e., the clarity of the text 111 in the user interface 12 can be lower than the clarity of the image text 111 in the user interface 11, to prompt the user that the text 111 has been selected and dragged up.

[0482] The electronic device may display a user interface 13 as shown in FIG4(C) in response to a user operation on the text 121, such as the drag operation shown in FIG4(B). As shown in FIG4(C), the user interface 13 is the cross-application transmission interface mentioned in the above description, which may include a sub-interface 131, an icon area 132, and text 133. Among them:

[0483] The specific styles and functions of the sub-interface 131 and the icon area 132 can be referred to the aforementioned description of the user interface 09 in FIG. 3A , which will not be repeated here.

[0484] It should be noted that the text content contained in text 133 is the same as the text content contained in text 111 and text 121. However, as can be seen from user interfaces 12 and 13, text 133 is the text obtained by the electronic device shrinking text 121; that is, after the "cross-application transfer interface" is opened, the electronic device also shrinks the text dragged by the user to a certain extent. In this way, text 133 does not cause a large area of ​​​​blocking the icons and text in icon area 132, making it easier for the user to select.

[0485] Similarly, in the user interface 13, the user places the dragged information element (i.e., text 133) on the corresponding icon in the icon area 132 and lets go. The electronic device can transmit the information element to the application service corresponding to the icon in response to the user operation, and display the application interface or feedback information corresponding to the application service icon.

[0486] Of course, when the electronic device is in landscape mode, the user can still transfer elements on the current interface across applications through the "cross-application transfer interface". The specific process and interface style can refer to the aforementioned descriptions of Figures 2 and 3A and user interface 14 and user interface 15 shown in Figure 5, and no further details will be given here.

[0487] It should be understood that most smart devices currently provide users with a quick screenshot function. For example, in some scenarios, users can double-click the screen or swipe down with three fingers to take a full-screen screenshot of the current interface of the electronic device. In addition, after the user takes a screenshot, some smart devices also provide users with a recognition function in this scenario to identify the user's screenshot (for example, extract text from the image or translate text, etc.). Therefore, with respect to the screenshot function provided by the electronic device, the following describes the process of using the cross-application transmission interface to transmit the captured image or text in the captured image across applications in the screenshot scenario in conjunction with Figures 6 and 7.

[0488] FIG6 is a schematic diagram of the interface change process of the electronic device when the user transmits the captured image across applications based on the cross-application transmission interface.

[0489] As shown in (A) of FIG6 , the user interface 16 may be an application interface of a “browser” application in an electronic device, or an application interface of a browser. In response to a user's quick screenshot operation on user interface 16, such as a double-click operation with a knuckle as shown in FIG6 (A), the electronic device can quickly capture a full-screen image of user interface 16 and display user interface 17 as shown in FIG6 (B).

[0490] As shown in FIG6(B), user interface 17 may include a screenshot image 171. Screenshot image 171 is an image obtained by the electronic device in response to a user taking a knuckle screenshot of user interface 16. After the screenshot is taken, screenshot image 171 may be displayed in user interface 17 for two to three seconds, allowing the user to directly click on screenshot image 171 to edit, save, or perform other operations on it. If the user does not perform any operations on screenshot image 171, the electronic device may automatically save it to the photo album.

[0491] In the embodiment of the present application, during the period when the electronic device displays the screenshot image 171 in the user interface 17, the user can long-press the screenshot image 171 and then drag it, or directly drag it to the side of the electronic device screen, and then transfer it across applications through the "cross-application transfer interface". As shown in Figure 6 (B), the electronic device can respond to the user's operation of dragging the screenshot image 171 to the side by displaying the user interface 18 shown in Figure 6 (C).

[0492] As shown in (C) of FIG6 , the user interface 18 is the cross-application transmission interface. The user interface 18 may include a sub-interface 181, an icon area 182, and an image 183, wherein:

[0493] The specific content and functions of the sub-interface 181 and the icon area 182 can be referred to the above description and will not be repeated here.

[0494] Similarly, in order to avoid excessive occlusion of the icon area 182 , when the screenshot image 171 approaches the icon area, the electronic device reduces the original screenshot image 171 to obtain image 183 .

[0495] Similarly, in the user interface 18, the user places the image 183 on the corresponding icon in the icon area 182 and lets go. The electronic device can transmit the information element to the application service corresponding to the icon in response to the user operation, and display the application interface or feedback information corresponding to the application service icon.

[0496] If the user releases the image 183 before placing it on the corresponding icon in the icon area 182 , the electronic device may redisplay the user interface 17 in response to the operation.

[0497] In an embodiment of the present application, user interface 16 can be called the "fifth interface", user interface 17 can be called the "sixth interface", screenshot image 171 or image 183 can both be called the "first image", user interface 18 can be called the "seventh interface", and the operation of the user placing image 183 on the corresponding icon in the icon area 182 and releasing it can be called the "ninth operation".

[0498] FIG7 is a schematic diagram of the interface change process of the electronic device when the user transmits the text in the captured image across applications based on the cross-application transmission interface.

[0499] As shown in (A) of FIG7 , the user interface 19 can be an application interface of an application “browser” in an electronic device, or an application In response to a user's quick screenshot operation on the user interface 19, such as the knuckle frame selection operation shown in FIG7 (A) (i.e., using a knuckle to draw an area on the screen), the electronic device can quickly capture the image selected by the user using the knuckle frame and display the user interface 20 shown in FIG7 (B).

[0500] As shown in FIG7 (B), the user interface 20 may include a thumbnail 201, a partial screenshot box 202, an identification control 203, a sharing control 205, and a shape option box 204.

[0501] The thumbnail 201 may be a thumbnail of the aforementioned user interface 19 .

[0502] The partial screenshot frame 202 is the screenshot area determined by the electronic device in response to the user's finger joint selection screenshot operation on the user interface 19. The user can adjust the edges of the partial screenshot frame 202 by dragging the edges of the partial screenshot frame, and can also press and hold the center area of ​​the partial screenshot frame 202 to adjust the specific position of the partial screenshot frame in the thumbnail 201. It should be understood that for the electronic device, the partial screenshot frame 202 is only used to select the image in the thumbnail 211, and the electronic device cannot obtain the information (such as text information) contained in the image in the partial screenshot frame 202.

[0503] The recognition control 203 is used to recognize text information contained in the image in the partial screenshot frame 202 .

[0504] The sharing control 205 is used to share the image in the partial screenshot frame 202 .

[0505] The shape option box 204 includes a free graphic option, a rectangle option, an ellipse option, and a heart option. The electronic device can adjust the specific shape of the partial screenshot box 202 to the shape corresponding to the option in response to a user's touch operation on any option, such as a click operation.

[0506] As shown in (B) of FIG7 , the electronic device can recognize the text information contained in the image in the partial screenshot frame 202 in response to the user clicking the recognition control 203. After the text information is recognized, the text information contained in the image in the partial screenshot frame 202 can be selected by the first cursor and the last cursor boxes. For details, please refer to the content displayed in the text box 212 in the user interface 21 in (C) of FIG7 . Afterwards, the user can long-press the recognized text information and drag it to the side of the electronic device screen to transfer the recognized text across applications through the "cross-application transfer interface."

[0507] As shown in FIG. 7 (C), the user interface 21 may include a thumbnail 211, a partial screenshot frame 202, text 2121, and text information 213, wherein:

[0508] The thumbnail 211 may be a thumbnail of the aforementioned user interface 19 .

[0509] The partial screenshot frame 202 is the partial screenshot frame 202 mentioned above. The text information in the image framed by the partial screenshot frame 202 has been recognized and extracted by the electronic device, and this part of the text is the text 2121 shown in (C) in FIG7 .

[0510] Text 213 is the text that is dragged up after the user performs a long press operation (not shown in FIG. 7 ) on text information 211. As shown in FIG. 7 (C), after the user drags text 213 up, the user can drag text information 213 to the side of the electronic device screen. In response to the user operation, the electronic device displays user interface 22 as shown in FIG. 7 (D).

[0511] As shown in (D) of FIG7 , the user interface 22 is the cross-application transmission interface. The user interface 22 may include a sub-interface 221, an icon area 222, and text 223, wherein:

[0512] The specific contents and functions of the sub-interface 221 and the icon area 222 can be referred to the above description and will not be repeated here.

[0513] Similarly, in order to avoid excessive obstruction in the icon area 222 , when the text 223 is close to the icon, the electronic device shrinks the text 213 to obtain the text 223 .

[0514] Similarly, in the user interface 22, the user places text 223 on the corresponding icon in the icon area 222 and lets go. The electronic device can transmit the text 223 to the application service corresponding to the icon in response to the user operation, and display the application interface or feedback information corresponding to the application service icon.

[0515] In an embodiment of the present application, the user interface 19 can be called the "eighth interface", the user interface 20 can be called the "ninth interface", the text 2121, text 223, and text 213 can all be called "first text information", the recognition control 203 can be called the "text recognition control", the user interface 21 can be called the "tenth interface", and the operation of the user dragging the text information 213 to the side of the electronic device screen can be called the "tenth operation", the user interface 22 can be called the "eleventh interface", and the operation of the user placing the text 223 on the corresponding icon in the icon area 222 and letting go can be called the "eleventh operation".

[0516] Of course, the image captured by the user through the partial screenshot function can also be transmitted through the cross-application transmission interface. Figure 8 shows the process of the user sharing the image captured by the partial screenshot function through the cross-application transmission interface.

[0517] The user interface 20 shown in (A) of FIG8 is the user interface 20 shown in (B) of FIG7 above. In combination with FIG8 and the above description, it can be seen that the user interface 20 may include a thumbnail 201, a partial screenshot frame 202, an identification control 203, a sharing control 205, and a shape option box 204. Among them, the partial screenshot frame 202 is the screenshot area determined by the electronic device in response to the user's finger joint frame selection screenshot operation on the user interface 19. It should be understood that although the partial screenshot frame 202 contains text, since the user does not click on the identification control 203 at this time, the partial screenshot frame 202 captures an image containing text. Similarly, as shown in (A) of FIG8, the user can long press and drag the image captured by the partial screenshot frame 202, and drag it to the side of the electronic device screen, and transmit the image captured by the partial screenshot frame 202 across applications through the "cross-application transmission interface".

[0518] As shown in (B) of FIG8 , the user interface 23 is the cross-application transmission interface. The user interface 23 may include a sub-interface 231, an icon area 232, and an image 233, wherein:

[0519] The specific contents and functions of the sub-interface 231 and the icon area 232 can be referred to the above description and will not be repeated here.

[0520] Similarly, to avoid excessive obstruction of icon area 232, when the image in partial screenshot frame 202 is dragged and brought close to the icon, the electronic device can reduce the image in partial screenshot frame 202 to obtain image 233. It should be noted that when the image in partial screenshot frame 202 is shared using the "cross-application transfer interface", the shape of partial screenshot frame 202 is retained, that is, the shape of image 233 is the same as the shape displayed by partial screenshot frame 202.

[0521] Similarly, in the user interface 23, the user places the image 233 on the corresponding icon in the icon area 232 and lets go. The electronic device can transmit the image 233 to the application service corresponding to the icon in response to the user operation, and display the application interface or feedback information corresponding to the application service icon.

[0522] In an embodiment of the present application, the user interface 20 can be called the "sixth interface", the user interface 23 can be called the "seventh interface", and the operation of the user placing the image 233 on the corresponding icon in the icon area 232 and releasing it can be called the "ninth operation".

[0523] It should be understood that in the daily use of electronic devices, users often drag and drop some elements. Therefore, how to determine the corresponding trigger rules for the cross-application transfer interface based on user operations and guide users to trigger the electronic device to display the "cross-application transfer interface" based on their operating habits is the key to further improving the efficiency of user interaction with electronic devices. Therefore, the trigger rules of the cross-application transfer interface will be explained below with reference to Figures 9-11.

[0524] FIG9 is a schematic diagram provided in an embodiment of the present application for illustrating triggering rules for a cross-application transmission interface.

[0525] As shown in (A) of FIG9 , the user interface 24 may be an application interface of a “browser” application in an electronic device, or an application interface of a browser. The application interface may also be the application interface of other applications, which is not limited in this application.

[0526] Next, we will use image 241 of user interface 24 as an example. As shown in FIG9(A), combined with the above description, it can be seen that the user can long-press image 241 to drag image 241 upward. In response to the user dragging image 241 upward, the electronic device can display user interface 25 as shown in FIG9(B).

[0527] As shown in FIG. 9B , the user interface 25 may include an image 241, an image 251, a left trigger area 252L, and a right trigger area 252R. In the embodiment of the present application, the left trigger area 252L and the right trigger area 252R may be collectively referred to as a “first trigger area”, where:

[0528] Image 251 is a thumbnail formed after the user drags up image 241. If the user does not let go, image 251 can move in the user interface 08 along with the user's fingertips. For details, please refer to the above-mentioned relevant instructions, which will not be repeated here.

[0529] The left trigger area 252L is a rectangular area bounded by the left edge of the electronic screen, and the right trigger area 252R is a rectangular area bounded by the right edge of the electronic screen. Specifically, the length of the left trigger area 252L can be d1 and the width can be W; the length of the right trigger area 252R can be d1 and the width can be W. Wherein, d1 can be set to 25% of the width of the electronic device screen, as shown in FIG9 (A), where the screen width is L, and W can be the height of the electronic device screen, as shown in FIG9 (A), where the screen height is h. It should be noted that in the user interface 25, the left trigger area 252L and the right trigger area 252R are displayed as dark transparent areas only to facilitate the user's understanding of the specific location and area of ​​these two areas. In actual application scenarios, the specific size and location of these two areas may not be displayed in the user interface, that is, the user cannot directly observe the specific outlines of these two areas in the user interface 25. The size and location of these two areas may be stored in the electronic device only in a form that is perceptible to the electronic device (e.g., program code).

[0530] When the user drags the image 251 to the left trigger area 252L or the right trigger area 252R, the electronic device may display a special effect at the edge of the screen. Optionally, after the user drags the image 251 to the left trigger area 252L or the right trigger area 252R and the image remains in the left trigger area 252L or the right trigger area 252R for a period of time T, the electronic device may display a special effect at the edge of the screen to prompt the user to drag the image further toward the side of the screen. Specifically, the above-mentioned T may be set to 600 milliseconds.

[0531] It should be understood that “dragging the image 251 into the left trigger area 252L or the right trigger area 252R” mentioned here means that after the user drags the image 251, the contact point between the user's finger and the screen (i.e., the anchor point) enters the left trigger area 252L or the right trigger area 252R, and does not mean that the image 251 completely enters the left trigger area 252L or the right trigger area 252R.

[0532] As shown in FIG9(B), when the user drags image 241 into right trigger area 252R, the electronic device may respond to this operation by displaying user interface 26 as shown in FIG9(C). User interface 26 may include image 241, image 251, and special effects area 261. Special effects area 261 may display user-visible special effects such as luminous effects and shadow effects, or other user-perceivable special effects, which are not limited in this application.

[0533] In combination with the above description, it can be seen that the above-mentioned special effects area 261 can prompt the user to further drag the image closer to the side of the screen to trigger the "cross-application transfer interface" interface to transfer the element to other applications. Therefore, in an embodiment of the present application, after the element is dragged up, a door opening area is also provided in the user interface. When the user drags the dragged element further to the door opening area, the electronic device can respond to the operation and display the cross-application transfer interface. As shown in (C) in Figure 9, when the user drags the image 251 to the right until it enters the door opening area, the electronic device can respond to the operation and display the user interface 27 shown in (D) in Figure 9.

[0534] As shown in the user interface 27 shown in (D) of FIG9 , the user interface 27 may include an image 241, an image 251, a left door opening area 271L, and a right door opening area 271R. In the embodiment of the present application, the left door opening area 271L and the right door opening area 271R may be collectively referred to as a “second triggering area”, wherein:

[0535] The left door area 271L is a rectangular area bounded by the left border of the electronic screen, and the right door area 271R is a rectangular area bounded by the right border of the electronic screen. Specifically, the length of the left door area 271L is d2, and the width is W; and the length of the right door area 271R is also d2, and the width is W, wherein d2 can be set to 36 pixel units, and W can be the height of the electronic device screen. It should be noted that in the user interface 27, the left door area 271L and the right door area 271R are displayed as dark transparent areas only to facilitate readers to understand the specific positions and areas of the two areas. In actual application scenarios, the specific sizes and positions of the two areas may not be displayed in the user interface, that is, the user cannot directly observe the specific outlines of the two areas in the user interface 27. The sizes and positions of the two areas can only be stored in the electronic device in a form that is perceptible to the electronic device (such as program code).

[0536] When the user drags the image 251 to the left door area 271L or the right door area 271R, the electronic device can immediately respond to the operation by displaying the cross-application transmission interface, that is, the user interface 28 shown in (E) of Figure 9. As shown in (E) of Figure 9, the user interface 28 includes a sub-interface 281, an icon area 282, and an image 283. Among them:

[0537] The elements displayed in the sub-interface 281, such as text, pictures, etc., are the same as the elements displayed in the user interface 24 (except for the status bar). The specific content and functions of the sub-interface 281 can be referred to the above related descriptions and will not be repeated here.

[0538] The electronic device can display one or more application service icons recommended based on the element selected by the user (i.e., image 241 in the embodiment of the present application) in the icon area 282. In addition, the icon area 282 can be a rectangular area bounded by the right border of the electronic screen. Specifically, the length and width of the icon area 282 can be d3 and W, respectively, where d3 is greater than the aforementioned d2. Specifically, d3 can be set to 120 pixel units, and W can be the height of the electronic device screen (i.e., h in the aforementioned description). Similarly, in the user interface 28, the icon area 282 is displayed as a dark transparent area only to facilitate the user to understand the specific position and area of ​​the area. In actual application scenarios, the specific size and position of the area may not be displayed in the user interface, that is, the user cannot directly observe the specific outline of the area in the user interface 28. The size and position of the area can be stored in the electronic device only in a form that is perceptible to the electronic device (such as program code).

[0539] The image content displayed by image 283 is the same as that displayed by image 251 and image 241, except for the difference in area size. It should be noted that compared with image 51 in user interface 25 to user interface 27, image 283 in user interface 28 is smaller in area. This is because when the electronic device displays user interface 28, image 251 in the original user interface 27 is reduced to prevent the image from being too large from blocking the icons and text in icon area 282 and interfering with the user's selection. In other words, image 241 can be reduced twice during the entire cross-application transmission process. The first time occurs when the user drags it up, specifically refer to image 251 and image 241 in (B) of Figure 9, and the second time occurs when the electronic device starts to display the "cross-application transmission interface", specifically refer to image 251 shown in (D) of Figure 9 and image 283 shown in (E) of Figure 9.

[0540] In the user interface 28, the user places the dragged information element (i.e., image 283) on the corresponding icon in the icon area 282 and lets go. The electronic device can transmit the information element to the application service corresponding to the icon in response to the user operation, and display the application interface or feedback information corresponding to the application service icon. This will be explained in more detail later and will not be repeated here.

[0541] In addition, in order to guide users to accurately place the dragged elements into the application services they need, in the user interface 28, when the user drags the image 283 close to an icon in the icon area 282, or when the distance between the image 283 and an icon in the icon area 282 is less than a certain threshold, the icon approached by the image 283 can display certain special effects, for example, the icon and the text corresponding to the icon can be adsorbed close to the image 283 and can be enlarged.

[0542] As shown in (E) of FIG9 , in user interface 28, after the user drags image 283 toward the “Friends WeChat” icon, the electronic device may display user interface 29 as shown in (F) of FIG9 . It can be seen from icon area 282 of user interface 29 that when image 283 approaches “Friends WeChat” icon 284, “Friends WeChat” icon 284 and its corresponding text 285 are further to the left than other icons and text, i.e., closer to image 283 than other icons and text; in addition, “Friends WeChat” icon 284 is also enlarged to remind the user that if they let go of the image 283 (or image 241) here, it will be transferred to the application (i.e., WeChat) or application service (WeChat-Friend Chat) corresponding to the “Friends WeChat” icon.

[0543] In addition, when the cross-application transmission interface is opened, the user can also exit the cross-application transmission interface through corresponding operations. Please refer to Figures 10 and 11 for details.

[0544] FIG10 shows the process of a user exiting the cross-application transmission interface through a door closing area.

[0545] As shown in (A) of FIG10 , the user interface 31 may be the cross-application transfer interface displayed by the electronic device after the user drags an image from the history interface to the side. The user interface 31 includes a sub-interface 311, an icon area 312, a door closing area 313, and an image 314, wherein:

[0546] The sub-interface 311 is an interface obtained by transforming the original interface where the image 314 is located;

[0547] Icon area 312 is an area for displaying application service icons recommended by one or more electronic devices based on the element selected by the user (i.e., image 314 in this embodiment of the application). Its specific style and function can be referred to the relevant description of Figure 9 above, and will not be repeated here.

[0548] The door closing area 313 can be a rectangular area bounded by the left border of the electronic screen. Specifically, the length and width of the door closing area 313 can be d4 and W respectively. Specifically, d4 can be set to half of the width of the electronic device screen (i.e., L in the above description), and W can be the height of the electronic device screen (i.e., h in the above description). Similarly, in the user interface 31, the icon area 312 and the door closing area 313 are displayed as dark transparent areas only to facilitate the user to understand the specific position and area of ​​the area. In actual application scenarios, the specific size and position of these two areas may not be displayed in the user interface, that is, the user cannot directly observe the specific outlines of the two areas in the user interface 31. The size and position of the two areas can be stored in the electronic device only in a form that is perceptible to the electronic device (such as program code).

[0549] As shown in (A) in Figure 10, after the user drags the image 314 to the door closing area 313, if the user still holds down the image 314, the electronic device can close the "cross-application transmission interface" and display the user interface 32 as shown in (B) in Figure 10.

[0550] As shown in FIG. 10(B), the user interface 32 may include images 321 and 322, wherein:

[0551] Image 321 is the thumbnail image immediately after the user drags up image 322. Unlike image 314, image 321 is the thumbnail image obtained by the electronic device reducing image 322 once, while image 314 is the thumbnail image obtained by the electronic device reducing image 321 twice after the cross-application transfer interface is opened.

[0552] In the case that the user does not release the image 321 , the user may drag the image 321 to the side of the screen to trigger the display of the cross-application transmission interface again.

[0553] FIG11 shows the process of the user exiting the cross-application transmission interface by directly releasing the finger.

[0554] As shown in (A) of FIG11 , the user interface 33 may be the cross-application transfer interface displayed by the electronic device after the user drags an image from the history interface to the side. The user interface 33 includes a sub-interface 331, an icon area 332, a door closing area 333, and an image 334, wherein:

[0555] The specific functions of the sub-interface 331 and the door closing area 333 can be referred to the aforementioned related descriptions and will not be repeated here.

[0556] It is easy to understand that when the total length of d4 and d3 is less than the width L of the electronic screen, there is still a portion of the user interface 33 that does not belong to the icon area 332 or the door closing area 333. In the user interface 33, this portion of the area can be a rectangular area 335 with a length of d5 and a width of h.

[0557] As shown in (A) of FIG11 , after the user drags the image 334 into the rectangular area 335 , if the user lets go directly at this time, the electronic device can close the “cross-application transfer interface” and display the user interface 34 as shown in (B) of FIG11 .

[0558] As shown in (B) of Figure 11 , user interface 34 is the original interface where image 314 is located. It may include image 341. Unlike user interface 32 described above, since the user has released the image, the thumbnail resulting from dragging image 341 is no longer displayed in user interface 34. If the user later wishes to transfer image 341 to another app through the "cross-app transfer interface," the user must again long-press image 341 to drag it to the corresponding thumbnail, and then drag the thumbnail to the side of the screen to trigger the cross-app transfer interface again.

[0559] In subsequent embodiments, the specific operation of the user triggering the "cross-application transmission interface" and closing the "cross-application transmission interface" and the corresponding interface change process of the electronic device can refer to the relevant description of the previous embodiment and will not be repeated later.

[0560] It should be understood that in the daily use of electronic devices, users often drag and drop some elements. For example, after a user uses the "smart split screen" function to open two application interfaces, or opens two application interfaces in the form of small windows or "picture-in-picture", the user will often transfer information across applications in this mode; and in the embodiment of the present application, in the split-screen state and the small window state, the user can directly complete the cross-application transmission of information through the split-screen function, or use the "cross-application transmission interface" provided by this application to complete the cross-application transmission of information. Combined with the aforementioned description of the triggering rules of the "cross-application transmission interface", the following describes different ways for electronic devices to transfer information across applications in the split-screen state and the small window state. Please refer to Figures 12-15 for details.

[0561] FIG12 shows the process of a user completing cross-application information transmission through the split-screen function of an electronic device in the split-screen state.

[0562] As shown in (A) of Figure 12, the user interface 35 includes a first display area 351 and a second display area 352. The first display area 351 is used to display the application interface of the "Notes" application, and the second display area 352 is used to display the application interface of the "Album" application. In this embodiment of the application, the user interface 35 can be referred to as the "first interface", the first display area 351 can be referred to as the "second sub-interface", and the second display area 352 can be referred to as the "first sub-interface".

[0563] At this time, the user long-presses the image 3521 in the second display area 352 and drags it up, obtaining a thumbnail 3522 of the image 3521. In an embodiment of the present application, the image 3521 or the thumbnail 3522 may be referred to as "first information". In addition, in combination with the foregoing description, it can be seen that after the image 3521 is dragged up, there may be two pairs of determination areas in the user interface 35, namely, area 353L and area 353R in the user interface 35, and area 354L and area 354R selected in the dotted box in the user interface 35. Among them, area 353L and area 353R are the left trigger area and the right trigger area, respectively; area 354L and area 354R are the left door opening area and the right door opening area, respectively. In an embodiment of the present application, area 353L and area 353R can be collectively referred to as the "first trigger area", and area 354L and area 354R can be collectively referred to as the "second trigger area". Combined with the above description, it can be seen that when the user drags the image 3522 to the left trigger area 354L or the right trigger area 354R and stays there for a period of time T, the electronic device can display special effects at the edge of the screen; when the user continues to drag the image 3522 to the left door area 354L or the right door area 354R, the electronic device will respond to the operation and display the corresponding cross-application transmission interface. However, if the user drags the image 3522 from the second display area 352 to the first display area 351 without letting the image 3522 touch the left door area 354L and the right door area 354R, the electronic device can still transfer the image 3522 (or image 3521) to the "Notes" application and display it in the first display area 351.

[0564] As shown in (A) of FIG12 , the user can drag the image 3522 to the first display area 351 in the direction indicated by the dotted arrow shown in (A) of FIG12 . In the embodiment of the present application, the operation of dragging the image 3522 to the first display area 351 can be referred to as the “first operation”; in this process, as long as the image 3522 does not touch the left door area 354L and the right door area 354R, the image 3522 can directly transfer the image 3522 (or image 3521) to the “Notes” application without opening the “cross-application transmission interface” and display it in the first display area 351. For details, please refer to the user interface 36 shown in (B) of FIG12 . It should be noted that even if the image 3522 briefly touches the left door area 354L or the right door area 354R during this process, the electronic device will only display the corresponding special effects on the side and will not directly trigger the “cross-application transmission interface”.

[0565] Also taking the user interface 35 in the above description as an example, FIG13 shows the process of a user completing cross-application transmission of information through the “cross-application transmission interface” of the electronic device in the split-screen state.

[0566] As shown in (A) of FIG13 , the content of the elements contained in the user interface 35 and the functions of each element can be referred to the aforementioned description of FIG12 and will not be repeated here. However, unlike the dragging method of the selected element (i.e., image 3522) shown in FIG12 , since the user now wants to transfer the image 3522 to another application through the “cross-application transfer interface”, this application may not be the “notes” application. Therefore, as shown in (A) and (B) of FIG13 , the user can drag the image 3522 to the right door opening area 354R in the direction indicated by the dotted arrow shown in (A) of FIG13 ; when the image 3522 enters the right door opening area 354R, the electronic device can immediately display the user interface 37 shown in (C) of FIG13 .

[0567] As shown in FIG. 13(C) , the user interface 37 may include a sub-interface 371 , an icon area 372 , and an image 373 .

[0568] in:

[0569] The specific content and functions of sub-interface 371 can be found in the aforementioned description and will not be further elaborated here. However, it should be noted that in the split-screen state, after the user triggers the "cross-application transfer interface" through an operation, the display screen in sub-interface 371 remains in split-screen mode and still includes two display areas, namely the first display area 3711 and the second display area 3712 shown in (C) of Figure 13.

[0570] The specific content and functions of the icon area 372 can be referred to the aforementioned related descriptions and will not be repeated here.

[0571] The image content displayed by image 373 is the same as the image content displayed by images 3521 and 3522, except for the difference in area size.

[0572] In the user interface 37, the user places the dragged information element, i.e., image 373, on the corresponding icon in the icon area 372 and lets go. The electronic device can then transmit the information element to the application service corresponding to the icon in response to the user operation and display the corresponding application interface. In the embodiment of the present application, the user interface 35 can be referred to as the "first interface", the image 3521 thumbnail 3522, or the image 373 can be referred to as the "first information", the operation of dragging the image 3522 to the right door opening area 354R can be referred to as the "second operation", the operation of placing the image 373 on the corresponding icon in the icon area 372 and letting go can be referred to as the "third operation", the user interface 37 can be referred to as the "second interface", and the icon displayed in the icon area 372 in the user interface 37 can be referred to as "at least one application service icon".

[0573] Similarly, when the electronic device is in a multi-window state with a small window, the user can also complete the process of cross-application transmission of information through the multi-window function or the "cross-application transmission interface". Figures 14 and 15 respectively show the process of the user completing the cross-application transmission of information through the split-screen function of the electronic device and the "cross-application transmission interface" in the small window state. In the small window state, the user interface displayed by the electronic device may include the application interfaces of two different applications, where the application interface of one application is displayed in full screen, and the application interface of the other application is displayed in a small window (floating window) on top. When information is transmitted across applications in the small window state, the user can directly drag and drop elements from one application interface to another application interface through the multi-window function. The specific operation process is similar to the operation process of cross-application transmission in the split-screen state in Figure 12. It can be understood that in the small window state, the method of transmitting information can be dragging from the small window (floating window) on top to the full-screen interface, or dragging from the full-screen interface to the small window (floating window) on top. In addition, in the small window state, users can also transfer through the "cross-application transfer interface". The specific operation process is similar to the operation process shown in Figure 13, so no further details will be given here. Please refer to Figures 14 and 15 for details, and combine them with the aforementioned relevant explanations of Figures 12 and 13 for understanding.

[0574] After using the "cross-application transfer interface" to transfer information across applications, the electronic device can also provide users with different prompts and display different interface effects based on the type of application service the information enters and the results of the information transfer. The following describes the possible interface styles that the electronic device may display after the user uses the "cross-application transfer interface" to transfer information across applications, using Figures 16-21.

[0575] ① After the transfer is completed, the snack bar is used to prompt the user, see Figure 16.

[0576] As shown in (A) of Figure 16 , the user interface 41 is a cross-application transmission interface for a user to transmit an image 411. The user is currently placing the dragged information element, i.e., the image 411, on the "note" icon on the right.

[0577] As the user releases the image, the electronic device may transmit the image 411 to the "Notes" application service in response to the user operation; if the transmission is successful, the electronic device may display the user interface 42 as shown in (B) in FIG. 16 .

[0578] As shown in (B) of FIG16 , a snack bar 421 may be displayed below the user interface 42. Specifically, the snack bar 421 may include prompt information 4211 and a jump control 4212, wherein:

[0579] The prompt information 4211 may include four characters “Transfer successful” to prompt the user that the electronic device has successfully transferred the image 411 to the “Notes” application service.

[0580] The jump control 4212 can respond to the user's touch operation to enable the electronic device to jump from the current interface to the interface of the application service "Notes".

[0581] In addition, if the user does not perform any operation on the snack bar 421, the snack bar 421 may automatically disappear after being displayed for a period of time (e.g., 2s-3s); or, when the snack bar 421 is still displayed in the user interface 42, the user may make the snack bar 421 disappear by going to other locations in the electronic user interface 42 except where the snack bar 421 is located.

[0582] As shown in FIG16(B), after the user clicks jump control 4212, the electronic device may respond to this operation by displaying user interface 43 as shown in FIG16(C). User interface 43 is the interface of the application service "Notes", and it can be seen from user interface 43 that image 411 has been transferred to "Notes" by the electronic device.

[0583] It should be understood that Figure 16 is merely an example of the user interface provided by the embodiment of the present application and should not constitute a limitation on the embodiment of the present application. For example, when the user transfers the dragged element to an application service in another immersive scenario, the electronic device can still use the snack bar in the interface to prompt the user.

[0584] ② After the transmission is completed, a capsule is generated to prompt the user, please refer to Figures 17 and 18.

[0585] As shown in (A) of Figure 17 , user interface 44 is a cross-application transfer interface for a user to transfer an image 441. The user may be placing the dragged information element, i.e., image 441, on the "global collection" icon on the right.

[0586] As the user releases the image, the electronic device can transfer the image 441 to the global collection (i.e., add the image 441 to the global collection folder) in response to the user operation; if the collection is successful, the electronic device can display the user interface 45 shown in (B) in Figure 16.

[0587] As shown in (B) in Figure 17, if the electronic device successfully collects the image 411 into the global favorites, a capsule 451 may be displayed on the right side of the user interface 45. Specifically, a "global favorites" icon 4511 may be displayed in the capsule 451. In addition, the user can change the specific position of the capsule 451 on the right side by pressing and holding it and dragging it. In addition, the user can also click on the capsule 451 as shown in (B) in Figure 17 to make the electronic device jump from the current interface to the interface of the application service "global favorites". The interface of the "global favorites" after the jump can be specifically referred to the user interface 43 shown in (B) in Figure 16 and the aforementioned related instructions, which will not be repeated here.

[0588] Optionally, capsule 451 may automatically disappear after being displayed on the side of the screen for a period of time. Alternatively, if the user does not actively close capsule 451, capsule 451 may remain displayed on the side of the screen, allowing the user to continue dragging other information elements contained in the current display interface of the electronic device and placing them in the "global collection". Please refer to Figure 18 for details.

[0589] As shown in (A) in Figure 18 , the user interface 45 is the user interface 45 shown in (B) in Figure 17 above. It is understandable that at this time the user has already collected the image 441 into the "global collection", and the capsule 451 is still displayed in the user interface 45. If the user wants to collect other elements in the interface into the "global collection" next, the user can drag the elements in the interface and move them to the capsule 451 and let go. As shown in (A) in Figure 18 , after the user selects the text 452, the user can long press the text 452 to drag it up and directly drag it to the capsule 451. The electronic device can respond to the user operation and display the user interface 46 shown in (B) in Figure 18 .

[0590] As shown in (B) of FIG18 , in user interface 46, after text 452 is dragged onto capsule 451, the electronic device may enlarge capsule 451 to facilitate user operation, i.e., capsule 451 in user interface 46 may be larger than capsule 451 in user interface 45, and the electronic device may reduce the original text 452 to text 461 in user interface 46. At this time, if the user releases text 461, the electronic device may, in response to the user operation, transfer text 461 (or text 452) to the global collection (i.e., add text 461 to the global collection). If the collection is successful, the electronic device may display user interface 47 as shown in (C) of FIG18 .

[0591] As shown in FIG18(C), capsule 451 is still displayed in user interface 47. The user can click capsule 451 as shown in FIG18(C) to open the "Global Favorites" interface of the application service. As a possible embodiment, after the user clicks capsule 451, the electronic device can display the "Global Favorites" interface of the application service in the form of a small window on the interface. For details, please refer to user interface 48 shown in FIG18(D).

[0592] As shown in (D) of FIG18 , the user interface 48 may include a main interface 481 and a sub-interface 482. The main interface 481 may be the application interface of the "browser" application in the electronic device, and the sub-interface 482 may be the small window interface of the application service "global collection." As can be seen from the user interface 48, the sub-interface 482 may include an image 4821, text 4822, and a function expansion control 4823, among which:

[0593] Image 4821 corresponds to image 441 in the above description;

[0594] The text 4822 corresponds to the text 452 or the text 461 in the above description;

[0595] The function expansion control 4823 can respond to a user's touch operation, such as a click operation as shown in (D) in FIG. 18 , to cause the electronic device to display a user interface 49 as shown in (E) in FIG. 18 .

[0596] As shown in (E) of FIG. 18 , in the user interface 49 , the sub-interface 482 displays an expand control 4824 , a minimize control 4825 , and a close control 4826 , wherein:

[0597] The minimization control 4825 can respond to a user operation to cause the electronic device to collapse the sub-interface 482 and display it in the form of a capsule on the side of the screen.

[0598] The close control 4826 can respond to user operation to cause the electronic device to close the sub-interface 482.

[0599] In response to a user operation, the expansion control 4824 can cause the electronic device to display the "Global Favorites" interface of the application service in full screen. As shown in FIG18(E), the electronic device can display the user interface 50 shown in FIG18(F) in response to the user clicking the expansion control 4824. As can be seen from the user interface 50, the electronic device has now displayed the "Global Favorites" interface of the application service in full screen, and the text image 441 and text 452 previously favorited by the user are also displayed accordingly in the user interface 50.

[0600] ③ After the transfer is complete, display the application window in the original interface or directly display the new application interface in full screen, see Figure 19.

[0601] As shown in FIG. 19(A), the user interface 51 is The user can long-press the text 511, drag the text 511 containing the location information, and drag it to the side of the electronic screen; the electronic device can respond to this operation and display the user interface 52 shown in (B) in FIG19 .

[0602] As shown in (B) of Figure 19, the user interface 52 is the cross-application transmission interface. In combination with the above description, it can be seen that the user can drag the text 521 obtained by shrinking the text 511 to any icon in the icon bar on the right side of the user interface 52 and then let go, and the text 521 (or text 511) can be transferred to the application service corresponding to the icon. As shown in (B) of Figure 19, when the user drags the text 521 to the "Map Navigation" icon 522 and then lets go, the electronic device can respond to the user operation and display the user interface 53 shown in (C) of Figure 19.

[0603] As shown in (C) of FIG19 , the user interface 53 may include a main interface 531 and a sub-interface 532. The main interface 531 may be an application interface in an electronic device. The application interface, sub-interface 532 is the application The user can adjust the specific position of the sub-interface 532 in the user interface 53 by pressing and dragging it. As can be seen from the user interface 53, the sub-interface 532 may include an expansion control 5321, a minimize control 5322, and a close control 5323, wherein:

[0604] The minimization control 5322 can respond to user operations to cause the electronic device to collapse the sub-interface 532 and display it in the form of a capsule on the side of the screen.

[0605] The close control 5323 can respond to user operation to cause the electronic device to close the sub-interface 532.

[0606] The expansion control 5321 can respond to user operations to enable the electronic device to display the application in full screen. As shown in (C) of FIG19 , the electronic device can respond to the user's click operation on the expansion control 4824 to display the user interface 54. It can be seen from the user interface 54 that the electronic device has now displayed the application in full screen. interface.

[0607] Optionally, after the user places the text 521 on the "Map Navigation" icon 522 and lets go, in response to the user operation, the electronic device may not display the user interface 53 as shown in (C) in Figure 19, but directly display the user interface 54 as shown in (D) in Figure 19, that is, directly display the new application interface in full screen.

[0608] ④ After the cross-device transfer is completed, the snack bar is displayed in the original interface to prompt the user, and the other devices are controlled to use small application windows or directly display the new application interface in full screen, see Figure 20.

[0609] It should be noted in advance that in the embodiments of the present application, after the electronic device establishes a communication connection with other smart devices, the electronic device can also transmit information across devices based on the "cross-application transmission interface" provided by the present application.

[0610] Figure 20 exemplifies the specific process of an electronic device using the "cross-application transmission interface" to send text information containing location information in WeChat to the vehicle-mounted terminal when a communication connection is established with the vehicle-mounted terminal, and using the vehicle-mounted terminal to navigate to the location information.

[0611] As shown in Figure 20, the electronic device 1900 has established a communication connection with the vehicle terminal via Bluetooth, Wi-Fi, etc. Among them, the electronic device 1900 can be an electronic device that supports the "cross-application transmission interface" interaction mode provided in the embodiment of the present application.

[0612] As shown in FIG. 20(A), the user interface 55 is The user can long-press the text 551 to drag the text 551 containing the location information to the side of the electronic device screen; the electronic device 1900 can respond to this operation by displaying the user interface 56 shown in (B) of FIG. 20 .

[0613] As shown in FIG20(B), the specific functions of the elements included in user interface 56 can be referred to in the aforementioned description of user interface 52 in FIG19. Text 561 corresponds to text 521, and the "Driving Navigation" icon 562 corresponds to the "Map Navigation" icon 522. "Driving Navigation" is a vehicle navigation service provided by electronic device 1900. After electronic device 1900 establishes a communication connection with another vehicle terminal, such as vehicle terminal 1901, the user can control the vehicle terminal for navigation using the "Driving Navigation" service provided by electronic device 1900. As shown in FIG20(B), if the user places text 561 on "Driving Navigation" icon 562 and then releases it, electronic device 1900 responds to the user's action by displaying user interface 57 as shown in FIG20(C), and transmits text 561 to vehicle terminal 1901, causing vehicle terminal 1901 to display user interface 58 as shown in FIG20(D).

[0614] As shown in (C) of FIG20 , a snack bar 571 may be displayed below the user interface 57. Specifically, the snack bar 571 may be used to display a prompt message “Successfully opened, please go to the associated device to view”, prompting the user that the electronic device 1900 has successfully sent the text 561 containing the location information to the vehicle-mounted terminal 1901, and that the vehicle-mounted terminal 1901 has opened a navigation application based on the received text 561 and is navigating to the location information.

[0615] As shown in (D) of FIG20 , the user interface 58 is the user interface displayed after the vehicle terminal 1901 receives the text transmitted by the electronic device 1900 based on the “cross-application transmission interface”. Specifically, the user interface 58 can be a navigation application installed in the vehicle terminal 1901 (e.g. The user interface 58 may also display an information bar 582, which may be used to prompt the user of the source device of the location information and prompt the user to confirm whether to navigate to the location corresponding to the location information.

[0616] ⑤ If the cross-device information transmission fails, a toast feedback prompt will be displayed on the original interface to prompt the user. Please refer to Figure 21.

[0617] It should be understood that for the electronic device provided by this application, not every cross-application transmission of information using the "cross-application transmission interface" will be successful. For example, when the size of an information element exceeds the upper limit specified by the target application service, or the format of the information element does not comply with the regulations of the target application service, the information element cannot be transmitted to the target application service through the "cross-application transmission interface". In this case, the electronic device can display toast feedback in the interface to prompt the user that the information has not been successfully transmitted.

[0618] As shown in (A) of FIG21 , user interface 60 is an application interface for the "Album" application, which displays 11 images, images 601 to 611. Among them, images 601 to 610 are images that have been selected by the user. The user can long press any of the 10 images (e.g., image 610 shown in (A) of FIG21 ) and then drag it to the side to cause the electronic device to display user interface 61 as shown in (B) of FIG21 .

[0619] As shown in (B) of FIG21, the user interface 61 is the cross-application transfer interface. In the user interface 61, the image thumbnail set 612 is the thumbnail set formed after the user drags and pulls up the image 601 to the image 610 in the user interface 60. The number "10" displayed in the upper left corner indicates that the user has selected a total of 10 images. If the transfer is successful, the electronic device can display The contact interface of the application is used to facilitate the user to share all the images contained in the image thumbnail set 612 with WeChat friends. However, since WeChat has a limit on the number of images that a user can send at a time (here it is assumed that the number does not exceed 9), and the image thumbnail set 612 contains a total of 10 images, which has exceeded the number limit specified by WeChat, the transmission process of images 601-610 through the "cross-application transmission interface" will not be successful. Therefore, after the user can press and hold the image thumbnail set 612 and drag it to the application service icon of "Friends WeChat" and then release it, in response to this operation, the electronic device can display the user interface 62 shown in (C) in Figure 21.

[0620] As shown in (C) of Figure 21, toast feedback 621 may be displayed below the user interface 62. Specifically, toast feedback 621 may display a prompt message "File has exceeded the transfer limit" to inform the user that the number of files being transferred exceeds the limit of the target application service, and therefore the transfer process has failed. Unlike the snack bar 421 mentioned in the previous description, toast feedback 621 generally does not display controls for jumping to other interfaces.

[0621] In addition, if the user does not perform any operation on the toast feedback 621, the toast feedback 621 may automatically disappear after being displayed for a period of time (for example, 2s-3s); or, when the toast feedback 621 is still displayed in the user interface 62, the user may make the toast feedback 621 disappear by clicking other locations in the electronic user interface 62 except the location where the toast feedback 621 is located.

[0622] Based on the foregoing description, in the method provided by this application, after obtaining the data to be transmitted, the electronic device generates a corresponding floating element. When the electronic device detects that a user has dragged the floating element into a preset hot zone, it can trigger a smart transfer service, displaying one or more icons of applications that can be used to receive the data to be transmitted. Subsequently, the electronic device can detect the operation of continuously dragging the floating element to the area where any application icon is located and then releasing it, and then send the data to be transmitted to the corresponding application.

[0623] This method provides users with a convenient, simple, fast, smooth and intuitive data sharing solution, which facilitates users to share data when needed and can improve the user's terminal usage experience. The information transmission method provided in the embodiment of the present application can also be called a data sharing method.

[0624] The dragging operation mentioned in the method can be performed by the user's finger, or by an information input device such as a stylus pen or a mouse.

[0625] The method provided in the embodiment of the present application will be introduced again below mainly from the perspective of hot zones.

[0626] Let me introduce a few concepts first, as follows:

[0627] Data to be transmitted

[0628] The data to be transmitted is the information mentioned in the above embodiment. The data to be transmitted can be stored locally in the electronic device, for example, in the storage path corresponding to some applications (such as a gallery) in the electronic device, or can also be stored in a cloud device.

[0629] The data to be transmitted can include various types. Here are some examples:

[0630] (1) Text. Text can be text in a web page displayed on an electronic device, text in a browser, text in a social software, text recognized in an image (such as an image stored in the electronic device's gallery, a screenshot, etc.), text converted from speech, etc.

[0631] (2) Documents. Documents may include text documents, table documents, code documents, etc.

[0632] (3) Pictures. Pictures may include static pictures or dynamic pictures. Pictures may also include full-screen screenshots, partial screenshots (i.e., irregularly shaped screenshots), edited screenshots, etc.

[0633] (4) Video. Videos may include videos shot by electronic devices, videos downloaded from the Internet, screen recordings, etc.

[0634] (5)Hyperlink.

[0635] Floating elements

[0636] The electronic device can display the data to be transmitted, and after the user inputs a user operation (such as a long press operation) for the data to be transmitted, the electronic device can generate a corresponding floating element.

[0637] A floating element is an element displayed on the display screen, suspended above other interface elements on the display screen, and can be dragged by the user to any position on the display screen. A floating element can be implemented as a thumbnail or preview of the corresponding data to be transmitted, or of course, it can be implemented in other forms such as a name, a label, or part or all of the content of the data to be transmitted, etc. This application does not limit this.

[0638] Hot Zone

[0639] A hotspot refers to an area on the display of an electronic device that is used to trigger a specific function.

[0640] The data sharing method provided in this application may involve multiple hot zones for triggering different functions during execution, which will be described in detail later. The size (i.e., dimensions), position, shape, etc. of each hot zone in this application can be set according to user needs and usage experience.

[0641] Each hot zone provided in this application is an area that the electronic device processes itself and does not need to be notified to the user. In other words, the electronic device does not need to notify the user of each hot zone on the display screen.

[0642] In some implementations, when an electronic device uses a function corresponding to a hot zone for the first time, the electronic device may provide a usage guide to prompt the user the location of each hot zone and the specific function that can be triggered by the hot zone.

[0643] In some other embodiments, the electronic device may also prompt the user with the hot zone. For example, each hot zone may be highlighted with a shadow, color, etc. to let the user know.

[0644] Smart circulation service

[0645] The Smart Flow Service, referred to above as "Smart Flow," is a system service or system function provided by the electronic device in this application, which can support the electronic device to conveniently transfer data from one application to another. Specifically, the Smart Flow Service supports the electronic device to display one or more icons of applications that can be used to receive the above-mentioned data to be transferred on the display screen, and after detecting the operation of continuously dragging a floating element to the area where any application icon is located and releasing it, the data to be transferred is sent to the corresponding application.

[0646] The smart transfer service is just a term used in this embodiment. Its meaning has been recorded in this embodiment, and its name does not constitute any limitation to this embodiment. In addition, in some other embodiments of this application, the smart transfer service can also be called other terms.

[0647] The following describes the data sharing method using the exemplary UI provided by this application.

[0648] 22A-22N are a set of user interfaces for implementing data sharing on the electronic device 100 .

[0649] Generate floating elements

[0650] Figure 22A is a user interface provided by an embodiment of the present application.

[0651] The application that provides the user interface can be a system application or a third-party application. In the embodiment of the present application, a system application refers to an application directly provided by the operating system of the electronic device 100 for implementing a specific function. A third-party application refers to an application provided by other developers and obtained by the electronic device 100 through the Internet for implementing a specific function.

[0652] In some embodiments, the above-mentioned user interface can also be the main interface (homepage), negative one screen, lock screen interface, etc. of the electronic device 100, and this application does not limit this.

[0653] During the process of displaying the user interface shown in Figure 22A, the electronic device 100 may detect a long press operation on the display screen. It is not limited to a long press operation, but may also be a double-click operation, a sliding operation along a preset specific trajectory, and other user operations, which are not limited in this application. In response to the above operation, the electronic device 100 may obtain the text in the area where the long press operation is located in the user interface, and display the following content on this part of the text: a text selection box, a drag bar at the start and end positions of the text selection box, and a toolbar above the text selection box. When the electronic device 100 initially detects a long press operation, the text selection box may be displayed only on a small part of the text (for example, 1-2 words in the area where the touch point of the long press operation is located). The drag bar at the start position and the drag bar at the end position of the text selection box can both be dragged by the user to adjust the size of the text selection box, that is, to adjust the selected text. The toolbar may include multiple controls for processing the selected text, such as a cut control, a copy control, a paste control, a share control, and the like.

[0654] Then, the electronic device 100 can detect the user operation of adjusting the selected text (such as dragging the position of the drag bar of the text selection box starting position or ending position to adjust the area or size of the text selection box). The electronic device 100 can determine the target text that the user wants to select based on the user operation of selecting text. Referring to Figure 22B, for example, after the user adjusts the text selection box to the size shown in Figure 22B, the electronic device 100 can determine the target text 111 based on the user operation.

[0655] 22C-22D , after determining the target text 111, the electronic device 100 may detect a long press operation on the target text 111. In response to this operation, the electronic device 100 may display a hovering control 112 corresponding to the target text 111. The control 112 may be located at the location of the touch point on the display screen when the user inputs the long press operation.

[0656] In some embodiments, in response to a long press operation on target text 111, the electronic device can generate a floating control 112 using the touch point of the long press operation on the display screen as an anchor point. Using the touch point as the anchor point means that a certain position point of the control 112 (such as the upper middle position, the midpoint of the upper edge, the center point, the midpoint of the right edge, or other point) is at the anchor point. Figure 22D illustrates this by taking a certain position on the right side of the control 112 as an example.

[0657] Optionally, the target text 111 may be displayed in the control 112. In some embodiments, when the target text 111 contains a large amount of content, the control 112 may display a portion of the target text 111, such as the first sentence of the target text 111 or a summary of the target text 111. In some embodiments, the control 112 may also be implemented as a preview or thumbnail of the target text 111.

[0658] The floating control 112 generated in FIG. 22D is the floating element in this application.

[0659] The suspended control 112 can adjust its display position in real time according to the user's dragging operation on the control 112 .

[0660] In some embodiments, when the user drags the control 112, if the touch point enters the edge of the display screen, part of the content of the control 112 may exceed the screen and not be displayed, or the electronic device may reduce the size of the control 112 to avoid incomplete display.

[0661] 22D , after the electronic device generates a suspended control 112, the text selection box below it continues to display to keep the corresponding text selected, but the drag bars at the start and end positions of the text selection box and the toolbar above the text selection box are no longer displayed. In some embodiments, when the electronic device generates a suspended control 112, it can also output a vibration signal to prompt the user that the suspended control 112 can be dragged.

[0662] Light band guide hot zone

[0663] The display screen has a corresponding light guide hotspot. The light guide hotspot is used to trigger the light guide effect in the direction indicated by the drag. The direction is the drag direction that triggers the smart flow service.

[0664] Referring to Figure 22D , after the electronic device 100 generates the hovering control 112, it also generates a light guide hotspot. As shown in Figure 22D , the light guide hotspot may include an area 113A bounded by the dotted line L1 on the left side of the display and the display's top edge E1, left edge E3, and bottom edge E2; and an area 113B bounded by the dotted line L2 on the right side of the display and the display's top edge E1, right edge E4, and bottom edge E2. Both areas 113A and 113B have the same height as the display, and their widths may be 25% of the display's width x.

[0665] 22D-22E , the electronic device 100 can detect a user operation in which the user drags the control 112 to slide toward the right side of the display screen. After detecting that the touch point of the user's finger in the display screen enters the light band guide hot zone 113B on the right side of the display screen, the electronic device 100 can display the light band 114 as shown in FIG22F on the right edge of the display screen. Alternatively, after detecting that the touch point of the user's finger in the display screen enters the light band guide hot zone 113B on the right side of the display screen and stays in the light band guide hot zone 113B for a period of time (such as 600ms), the electronic device 100 can display the light band. The long press operation for generating a suspended control 112 and the operation of dragging the control 112 to slide can be continuous and uninterrupted operations, or two operations with an interval in between.

[0666] The light strip 114 may be a strip-shaped interface element with attributes such as color, shadow, gradient, graphic fill (such as grid fill, diagonal fill), etc. In some embodiments, the light strip 114 may also present dynamic effects, such as changing color, flashing display, etc.

[0667] The light strip 114 on the right edge of the display screen can be used to prompt the following information: 1. The current control 112 has been dragged to the right side of the display screen; 2. Continuing to drag to the right is the dragging direction that triggers the smart flow service, that is, if the user continues to drag the control 112 to the right, the smart flow service can be triggered.

[0668] Referring to the control 112 shown in Figures 22D-22E, while dragging the control 112, the electronic device 100 can reduce the size of the control 112 to reduce obstruction of other content on the display screen.

[0669] In other embodiments, if the electronic device 100 detects that the user has dragged the control 112 in FIG. 22D to the light bar guidance hotspot 113A on the left side of the display screen, the electronic device 100 may display a light bar at the left edge of the display screen. This light bar may be used to indicate the following: 1. The control 112 has been dragged to the left side of the display screen; 2. Continuing to drag to the left is the direction in which the smart flow service will be triggered, i.e., if the user continues to drag the control 112 to the left, the smart flow service will be triggered.

[0670] Equivalently, dragging the control 112 to the left side of the display screen or to the light strip guidance hot zone on the right side can trigger the display of the light strip.

[0671] Door opening hot zone

[0672] The display screen has corresponding door opening hot zones, which are used to trigger smart flow services.

[0673] Referring to Figure 22G, after the electronic device 100 generates the floating control 112, the electronic device 100 generates the door opening hot zone. As shown in Figure 22G, the door opening hot zone may include an area 115A enclosed by the dotted line L3 on the left side of the display screen and the upper edge E1, left edge E3, and lower edge E2 of the display screen, and an area 115B enclosed by the dotted line L4 on the right side of the display screen and the upper edge E1, right edge E4, and lower edge E2 of the display screen. The height of area 115A and area 115B are the same as the height of the display screen, and the width can be 40 (density-independent pixel, dp) of the width of the display screen. Among them, dp is an abstract unit based on screen density, 1dp*pixel density / 160=actual number of pixels. Therefore, the width of area 115A and area 115B is related to the pixel density of the display screen. The greater the pixel density, the greater its width.

[0674] 22F-22G , the electronic device 100 can detect the user operation of the user continuing to drag the control 112 to the right side of the display screen. After detecting that the touch point of the user's finger in the display screen enters the door opening hot zone 115B on the right side of the display screen, the electronic device 100 can stop displaying the light strip and trigger the smart flow service. Alternatively, after detecting that the touch point of the user's finger in the display screen enters the door opening hot zone 115B on the right side of the display screen and stays in the door opening hot zone 115B for a period of time (such as 600ms), the electronic device 100 can stop displaying the light strip and trigger the smart flow service. The duration can be set as needed and is not limited here.

[0675] The smart transfer service refers to the electronic device 100 matching an application that can receive data to be transferred, providing the user with a service that quickly sends the data to be transferred to other target applications. The data corresponding to the floating element that the user acts on when the smart transfer service is triggered is the data to be transferred. For example, in the scenario shown in Figures 22F-22H where the drag control 112 is slid right into the right side door opening hot zone 115B to trigger the smart transfer service, the target text 111 corresponding to the control 112 is the data to be transferred.

[0676] Referring to Figure 22H, after detecting that the touch point of the user's finger in the display screen enters the door opening hot zone 115B on the right side of the display screen, or after detecting that the touch point of the user's finger in the display screen enters the door opening hot zone 115B on the right side of the display screen and stays in the door opening hot zone 115B for a period of time (such as 600ms), the electronic device 100 may display a menu bar 116 including icons of one or more candidate applications on the right side of the display screen.

[0677] The menu bar 116 may include a favorites icon, a search icon, a share icon, a note icon, a text message icon, and the like. Among them, the favorites application corresponding to the favorites icon may be a system application, which provides a global system favorites function. The note application corresponding to the note icon may be a system application or a third-party application, which may provide in-application recording, collection, and other functions. The candidate application is an application that supports receiving data to be transmitted, which is screened out by the electronic device 100 from all installed applications. The electronic device 100 may determine the candidate application based on the information type, semantic information, and the like of the data to be transmitted. The candidate applications may include global applications that support receiving any type of data (such as favorites applications, search applications, sharing applications, email applications, etc.), and may also include applications that only support receiving some types of data (such as picture editing applications that only support receiving pictures, navigation applications that only support receiving text, etc.). Different data to be transmitted may correspond to different menu bars.

[0678] In some embodiments, when the electronic device 100 displays the menu bar 116, the control 112 may be further scaled to avoid obstructing the menu bar 116. For example, the electronic device may reduce the control 112 so that the height of the control 112 is consistent with the height of the candidate application icon in the menu bar 116.

[0679] 22H , the electronic device 100 displays a menu bar 116 containing icons of candidate applications on the right side of the display screen. Simultaneously, the electronic device 100 may adjust the original foreground user interface to a right perspective style, thereby visually forming an "opening door" animation effect with the menu bar 116. The foreground user interface may refer to an interface consisting of other content displayed on the display screen excluding the status bar, or may refer to all content displayed on the display screen including the status bar.

[0680] If the foreground user interface originally displayed by the electronic device 100 includes dynamic effects, such as videos or animated images, the electronic device can capture a screenshot of the user interface and then adjust the screenshot to a perspective style. In other words, the perspective style interface can be a screenshot processed by the operating system of the electronic device. In other embodiments, the electronic device can also adjust the original user interface to a perspective style and continue to play dynamic effects in the perspective style user interface, such as continuing to play videos or animated images.

[0681] In other embodiments, if the electronic device 100 detects that the user drags the control 112 in Figures 22D-22F to the door opening hot zone 115A on the left side of the display screen, the electronic device 100 can display a menu bar containing icons of candidate applications on the left side of the display screen, and adjust the original foreground user interface to a left perspective style, forming an "opening the door" animation effect with the menu bar.

[0682] This is equivalent to dragging the control 112 to the left edge of the display screen or to the door opening hot zone on the right, which can trigger the "door opening" animation and also trigger the smart flow service.

[0683] The width of the door opening hot zone can be narrower than the width of the light band guidance hot zone, so that when the electronic device 100 drags the control 112, it will first enter the light band guidance hot zone and trigger the guidance light band effect, and then enter the door opening hot zone to trigger the smart flow service.

[0684] Hover to zoom in on hot spots

[0685] The display has a corresponding hover zoom hotspot. The hover zoom hotspot is used to trigger the proximity hover effect.

[0686] Referring to Figure 22H , after electronic device 100 displays menu bar 116, it generates a hover zoom hotspot. As shown in Figure 22H , the hover zoom hotspot may include an area 117 enclosed by dashed line L5 on the right side of the display screen and the top edge E1, right edge E4, and bottom edge E2 of the display screen. Area 117 has a height equal to the display screen height and a width of 120dp of the display screen width.

[0687] The proximity hover effect means that after the user drags the control 112 and the touch point on the display screen enters the hover magnification hot zone, the icon of the candidate application closest to the touch point in the menu bar 116 is enlarged in the basic size, and the application icons adjacent to and before the icon of the candidate application are also enlarged accordingly, but the degree of enlargement is less than that of the icon of the candidate application.

[0688] Among them, the basic size can be set as needed and is not limited here. The icon of the candidate application closest to the touch point can be enlarged to a first ratio of the basic size (such as 150% or other ratios), and its enlarged state can be called a hover state. The adjacent application icons before and after the icon of the candidate application closest to the touch point can be enlarged to a second ratio of the basic size (such as 120% or other ratios), and its enlarged state can be called a nearby hover state. The first ratio is greater than the second ratio, and the first ratio and the second ratio can be preset.

[0689] Specifically, the electronic device first detects whether the touch point is located in the hover magnification area. If so, it determines the icon of the candidate application that is closest to the touch point. For example, the electronic device 100 first determines the position of the icons of each candidate application of the basic size in the display screen, and then draws a straight line in the horizontal direction parallel to the display screen with the touch point. The icon of the candidate application that coincides with the straight line is the icon of the candidate application that is closest. For another example, the electronic device 100 first determines the position of the icons of each candidate application of the basic size in the display screen, and then calculates the distance between the touch point and the center point of each candidate application icon. The icon of the candidate application with the smallest distance is the icon of the candidate application that is closest.

[0690] Referring to Figure 22H, the touch point of the user's finger on the display screen enters the hover magnification hot zone 117, and the touch point is closest to the note icon in the menu bar 116, so the note icon is magnified, and the sharing icons and information icons adjacent to it are secondarily magnified, and the corresponding text at the bottom of the icon can also be enlarged accordingly.

[0691] Referring to Figure 22I , as the user continues to drag control 112, the touch point also moves to the position shown in Figure 22I . In Figure 22I , the user's finger and the touch point on the display are closest to the search icon in menu bar 116 , so the share icon is enlarged, and the adjacent favorites and share icons are secondary enlarged, and the corresponding text at the bottom of the icon is also enlarged accordingly.

[0692] As can be seen from Figures 22H and 22I , in the proximity hover effect, the closer the user's touch point and the display are to the candidate application icon in menu bar 116, the larger the corresponding candidate application icon can be. This can create the effect of the candidate application icon in menu bar 116 being attracted by the user's finger, and can also draw the user's attention to the corresponding candidate application icon by enlarging the size of the application icon.

[0693] In some embodiments, the electronic device 100 may first display the candidate application icons at a base size, and then magnify a portion of the candidate application icons based on the position of the touch point in the hover magnification hotspot, presenting a dynamic change process. In other embodiments, the processing speed of the electronic device 100 is fast enough that it is not necessary to first display the candidate application icons at a base size, and the magnified portion of the candidate application icons may be directly displayed based on the position of the touch point in the hover magnification hotspot.

[0694] In other embodiments, if the electronic device 100 displays a menu bar containing icons of candidate applications on the left side of the display screen and adjusts the original foreground user interface to a left perspective style, the electronic device 100 generates a hover zoom hotspot on the left side of the display screen. The hover zoom hotspot on the left side of the display screen can be used to trigger a hover animation effect of the menu bar on the left side of the display screen.

[0695] Equivalently, both the left and right sides of the display screen in the electronic device 100 may include hover hotspots, and both may trigger hover animations. The hover hotspot is always located in the direction of the menu bar in the display screen.

[0696] In addition to the proximity hover effect triggered by the hover hot zone described above, in some embodiments, after the user drags the control 112 and the touch point on the display screen enters the hover magnification hot zone, the electronic device can also perform other effects on the icon of the candidate application closest to the touch point in the menu bar 116, such as increasing the display brightness of the icon of the candidate application, marking the icon of the candidate application with a striking color, etc.

[0697] Closed hot zone

[0698] The electronic device 100 displays a menu bar containing icons of candidate applications on the side of the display screen. After the original foreground user interface is adjusted to a perspective style, that is, after the door opening effect is triggered, the display screen corresponds to the door closing hot zone. The door closing hot zone is used to close the smart flow service.

[0699] Referring to Figure 22J , electronic device 100 displays a menu bar 116 containing candidate application icons on the right side of the display screen. After adjusting the original foreground user interface to a right-perspective style, electronic device 100 generates a closing hotspot 118 bounded by a dotted line L6 on the left side of the display screen and the top edge E1, left edge E3, and bottom edge E2 of the display screen. The height of closing hotspot 118 is the same as the height of the display screen, and its width can be half the width x of the display screen.

[0700] 22J-22K , the electronic device 100 may detect a user operation in which the user continues to drag the control 112 toward the left side of the display screen.

[0701] As shown in Figures 22I-22J , the electronic device detects that the user has dragged control 112 from hover zoom hotspot 117 to a portion of the center of the display screen, which is outside of hover zoom hotspot 117 and door closing hotspot 118. Therefore, electronic device 100 stops displaying the proximity hover animation. That is, referring to Figure 22J , the icons in menu bar 116 are all the same size and are the base size.

[0702] As shown in FIG22K , upon detecting that the touch point of the user's finger on the display screen enters the door-closing hot zone 118 on the left side of the display screen, the electronic device 100 may shut down the smart flow service. Alternatively, upon detecting that the touch point of the user's finger on the display screen enters the door-closing hot zone 118 on the left side of the display screen and remains in the door-closing hot zone 118 for a certain period of time (e.g., 600ms), the electronic device 100 may shut down the smart flow service. This period of time can be set as needed and is not limited here.

[0703] Referring to Figure 22L , disabling the Smart Flow service means that electronic device 100 stops displaying menu bar 116 on the right side of the display screen and adjusts the original foreground user interface (excluding the status bar) from a right-perspective style to a non-perspective style. Combined with Figures 22K and 22L , the style adjustment of the foreground user interface and the disappearance of menu bar 116 can vividly create a "door closing" animation effect.

[0704] In other embodiments, if the electronic device 100 displays a menu bar containing icons of candidate applications on the left side of the display screen and adjusts the original foreground user interface to a left perspective style, the electronic device 100 generates a door-closing hot zone on the right side of the display screen. The door-closing hot zone on the right side of the display screen can be used to trigger a "door-closing" animation effect of the menu bar on the left side of the display screen.

[0705] Equivalently, both the left and right sides of the display screen in the electronic device 100 may include a door closing hot zone, and both may trigger the "door closing" animation effect. The door closing hot zone is always located in the opposite direction of the menu bar in the display screen.

[0706] In the present application, during the process of dragging the control 112 shown in FIG. 22E to FIG. 22J , the user's finger and the display screen may be in continuous contact, that is, the dragging operation may be continuous and uninterrupted.

[0707] After the electronic device 100 detects that the user's finger and the touch point of the display screen have entered the door opening hot zone and the smart flow service has been started (i.e., the menu bar 116 is displayed), if the user's finger leaves the display screen, that is, if the electronic device does not detect the touch point, the electronic device 100 will turn off the smart flow service (i.e., stop displaying the menu bar 116). For example, if the finger touch point shown in Figure 22H disappears, the electronic device will also stop displaying the menu bar 116 in Figure 22H. Equivalently, in this application, the user can turn off the smart flow service by closing the door hot zone or by stopping touching the display screen.

[0708] Transferring Data

[0709] Figures 22M-22N show the user interface for transmitting data through the smart flow service.

[0710] Referring to Figure 22M, after the electronic device 100 detects that the user drags the control 112 to the door opening hot zone 115B on the right and displays the menu bar 116, it can detect the user operation of the user continuing to drag the control 112 to the area where the icon of any candidate application is located in the menu bar 116 and then releasing it. Specifically, referring to Figure 22M, the electronic device 100 can detect the user operation of the user continuing to drag the control 112 to the note application icon and releasing it. Thus, the electronic device 100 can confirm that the note application is the target application. After detecting that the control 112 is dragged to the note application icon, the electronic device 100 can highlight the note application icon, such as increasing it, changing its color, increasing its brightness, and outputting a vibration signal, etc., to prompt the user that the note application is currently selected as the target application to avoid erroneous transmission.

[0711] In some embodiments, after the electronic device 100 detects the user operation of continuing to drag the control 112 to the area where the icon of any candidate application is located in the menu bar 116 and then releasing it, it can also display an animation of the control 112 being sucked into or stored in the icon of the corresponding candidate application to prompt the user that the electronic device will transfer the data to be transferred to the corresponding candidate application.

[0712] Subsequently, the electronic device 100 may write the data to be transmitted (ie, the target text 111 ) corresponding to the control 112 into the note application.

[0713] Referring to Figure 22N, after writing the target text 111 into the note application, the electronic device 100 can close the menu bar 116 of the smart transfer service and redisplay the original user interface. Optionally, the electronic device 100 can also display a prompt window 119. The prompt window 119 can prompt the user that the transfer is successful, that is, the data to be transferred selected by the user has been successfully written to the target application. Among them, the prompt window 119 can also include a control 120. The control 120 can be used to open the target application that receives the aforementioned data to be transferred, such as a note application, so that the user can continue to edit the data written by the smart transfer service in the target application.

[0714] The electronic device 100 shown in FIG. 22N is not limited to directly writing the data to be transmitted into the target application. In some other embodiments, after the electronic device 100 determines the target application, it can also display a window provided by the target application. One or more options can be displayed in the window. The user can set the usage of the data to be transmitted in the target application (such as printing parameters, sharing to contacts, etc.) according to the above one or more options. Then, the target application can process the data to be transmitted according to the usage set by the above one or more options.

[0715] Various hot spots in mobile phones

[0716] FIG. 23 shows examples of various hot zones when the mobile phone is in portrait mode.

[0717] In Figure 23:

[0718] a shows the door opening hot zone, which includes two areas on the left and right sides of the display, each occupying 40dp of the display width.

[0719] b shows the light band guide hot zone, including two areas on the left and right sides of the display screen, each occupying 1 / 4 of the width of the display screen.

[0720] c shows the hover zoom hotspot for “door open right”, which includes an area on the right side of the display, occupying 120dp of the display width.

[0721] d shows the door closing hot zone when the door is opened right, including an area on the left side of the display screen, which occupies 1 / 2 of the width of the display screen.

[0722] e shows the hover zoom hotspot for "door open left," which includes an area on the left side of the display that occupies 120dp of the display width.

[0723] f shows the door closing hot zone when the door is open to the left, including an area on the right side of the display screen, which occupies 1 / 2 of the width of the display screen.

[0724] Figures 22A-22N and 23 above all use a mobile phone as an example to illustrate various hotspots and functions of a mobile phone in portrait mode. In the embodiments of the present application, the hotspots of an electronic device of the same form factor may vary in different states. The states mentioned here may include portrait mode and landscape mode.

[0725] Refer to FIG24 , which exemplarily shows various hot zones when the mobile phone is in landscape mode.

[0726] Unlike the portrait orientation in Figures 22A-22N and Figure 2, the position of each edge relative to the user has changed in Figure 24 because the phone is in landscape orientation. Each edge in Figure 24 corresponds to the edges with the same reference numbers in Figures 22A-22N.

[0727] In Figure 24:

[0728] The area enclosed by the dotted line L3 and the display screen edges E1, E4, and E3, as well as the area enclosed by the dotted line L4 and the display screen edges E2, E4, and E3, are the door opening hot zones;

[0729] The area enclosed by the dotted line L1 and the display screen edges E1, E4, and E3, as well as the area enclosed by the dotted line L2 and the display screen edges E2, E4, and E3, are the light band guidance hot zones;

[0730] The area enclosed by the dotted line L5 and the display edges E2, E4, and E3 is the hover zoom hotspot for the "right-open door" effect.

[0731] The area enclosed by the dotted line L6 and the display screen edges E1, E4, and E3 is the door closing hot zone under the "right-open door" effect.

[0732] Of course, under the "left-open door" effect, the hover zoom hot zone and the door closing hot zone in Figure 24 are both in the opposite direction of the currently shown side.

[0733] This isn't limited to mobile phones; other electronic devices, such as foldable phones and tablets, can also have different hotspots in landscape and portrait modes. The locations of each hotspot on the display are similar to those described for mobile phones, so please refer to the previous section for details.

[0734] The height of each of the hot zones mentioned in this application can be measured as a percentage of the display screen height or as a percentage of the display screen height in dp. The same is true for the width, which is not limited here.

[0735] The names of the above-mentioned thermal zones mentioned in this application are only examples. The functions of each thermal zone have been recorded in the previous text, and their names do not constitute a limitation to this application.

[0736] The size (i.e., dimensions), position, shape, etc. of each hot zone mentioned in the above UI in the display screen are all examples. In specific implementation, the size (i.e., dimensions), position, shape, etc. of each hot zone in the electronic device can be set according to user needs and usage experience, and this application does not impose specific restrictions on this.

[0737] The gray fill effect used to illustrate each hotspot in the above UI embodiment is for illustrative purposes only. In the actual user interface presented to the user by the electronic device, the corresponding hotspot will not include the gray fill effect. In other words, the electronic device does not need to indicate the location of each hotspot to the user; the user can sense the presence of each hotspot while using the hotspot function.

[0738] Anti-accidental touch of door opening hot zone

[0739] 25A-25F illustrate a method of sharing screenshots.

[0740] 25A , while displaying the user interface shown in FIG25A , the electronic device 100 may detect a screenshot operation. The screenshot operation may be performed by a physical button on the electronic device 100, such as a power button or a volume button, or by a touch operation on the display screen of the electronic device 100, such as a three-finger swipe operation, etc., and this application does not impose any restrictions on this.

[0741] In response to the above screenshot operation, the electronic device 100 can save a frame of the user interface on the current display screen, i.e., a screenshot image, also known as a screenshot. The electronic device 100 can write the screenshot to the memory of the electronic device 100 for storage, so that the user can browse the screenshot at any time through the access operation.

[0742] In an embodiment of the present application, the electronic device 100 may further display a preview of the above screenshot on the display screen to prompt the user to obtain a frame of screenshot.

[0743] Figure 25B is a user interface of an electronic device displaying a screenshot preview image on a display screen. As shown in Figure 25B, the electronic device 100 may display image 411 in the lower left corner area of ​​the display screen. Image 411 may be a screenshot preview image of the user interface shown in Figure 25A. The user can preview the screenshot obtained by the aforementioned screenshot operation through the above-mentioned image 411. The above-mentioned lower left corner area of ​​the display screen may be referred to as the default screenshot display area. It is not limited to the lower left corner. The default screenshot display area may also be the upper right corner, lower right corner, upper left corner, etc. of the display screen, and this embodiment of the application does not impose any restrictions on this.

[0744] The electronic device 100 may keep displaying the image 411 for a period of time according to a preset timer. The period of time may be, for example, 3 seconds. After the timer expires, the electronic device 100 may no longer display the image 411.

[0745] In the embodiment of the present application, during the process of displaying the image 411 , the electronic device 100 may detect whether there is any user operation acting on the image 411 .

[0746] Referring to Figure 25C, the electronic device 100 can detect an operation of long pressing the display screen on the image 411 (hereinafter referred to as the long press operation), and the touch point of the user's finger is located in other areas outside the door opening hot zone 412A and the door opening hot zone 412B. In response to the above user operation, the electronic device 100 can switch the image 411 to a suspended state as shown in Figure 25D. The suspended image 411 can adjust the display position in real time following the motion trajectory of the user's touch operation. The suspended image 411 is the floating element of this application.

[0747] In some embodiments, after the electronic device 100 switches the image 411 to the suspended state, no prompt information may be output on the user interface, that is, it appears that there is no difference before and after the switch. In other embodiments, after the electronic device 100 switches the image 411 to the suspended state, some prompt information may be output on the user interface (for example, adding a black border to the image 411 as shown in FIG. 25D, or displaying other dynamic effects around the image 411), thereby prompting the user that the image 411 can currently be dragged by the user.

[0748] Referring to Figures 25D-25E, after the electronic device 100 generates the suspended image 411, it can detect the operation of continuously dragging the suspended image 411 to the right side of the display screen to the door opening hot zone 412B. In response to this operation, the electronic device 100 can start the smart flow service, that is, displaying a menu bar containing icons of multiple candidate applications on the right side of the display screen. Afterwards, the electronic device 100 can also detect the operation of continuing to drag the suspended image 411 into the area where the icon of a candidate application is located in the menu bar, and in response to this operation, send the original screenshot of the image 411 to the application corresponding to the icon of the candidate application.

[0749] After the user triggers the electronic device 100 to generate a floating element, there may be multiple intentions, such as dragging the floating element to the door opening hot zone to trigger the smart flow service, or dragging the floating element to another location on the display screen (such as dragging the preview image of a screenshot, dragging the location of the text in a memo), etc. The door opening hot zone and other locations on the display screen may overlap, so when the user drags the floating element, the electronic device 100 needs to accurately identify the user's intention and then accurately respond to the user's needs to avoid misoperation.

[0750] FIG25G shows a solution for preventing accidental touches by reducing the door opening hot zone.

[0751] The user interface shown in FIG. 25G may be displayed by the electronic device when a long press operation is detected on the image 411 in FIG. 25B .

[0752] As shown in Figure 25G, the door opening hot zone in the user interface includes: the area 412A enclosed by the dotted line L7, the dotted line L8, and the upper edge E1 and the left edge E3 of the display screen, and the area 412B enclosed by the dotted line L4 and the upper edge E1, the right edge E4, and the lower edge E2 of the display screen.

[0753] Compared to a conventional door opening hot zone before the accidental touch prevention solution is enabled (such as the door opening hot zone in Figure 22G or Figure 25G ), the door opening hot zone in Figure 25G has undergone one or more of the following adjustments:

[0754] 1. Because the default screenshot display area of ​​the electronic device in FIG25G is located in the lower left corner of the display screen, the width of the door opening hot zone 412A on the same side (i.e., the left side of the display screen) is shorter than the width of a conventional door opening hot zone on the same side. For example, the width of the door opening hot zone 412A in FIG25G may be 1 / 2 the width of the door opening hot zone 412A in FIG25C.

[0755] 2. Because the default screenshot display area of ​​the electronic device in FIG25G is located in the lower left corner of the display screen, the height of the door opening hot zone 412A on the same side (i.e., the left side of the display screen) can be from the top of the display screen to a position in the middle of the display screen rather than directly reaching the bottom of the display screen. For example, the height of the door opening hot zone 412A can be from the top of the display screen to the top of the image 411.

[0756] The door opening hot zone adjusted based on any one or more of the above is smaller than the conventional door opening hot zone. When the user operates (e.g., long presses) to bring up image 411, the probability of the touch point directly entering the door opening hot zone is reduced, thereby reducing the probability of the smart flow service being directly triggered after the image 411 is brought up.

[0757] In this way, the user can more easily and conveniently choose to trigger the smart flow service or simply move the position of image 411 on the display screen while dragging image 411. Specifically, the user can trigger the smart flow service by dragging image 411 to the left door opening hot zone 412A or the right door opening hot zone 412B in Figure 25G. The user can also move the position of image 411 on the display screen by dragging image 411 to an area outside the door opening hot zone in Figure 25G.

[0758] In other embodiments, if the default screenshot display area is located in another area of ​​the electronic device's display screen, the door opening hotspot after generating the floating element can be adjusted accordingly based on the default screenshot display area. For example, if the default screenshot display area is located in the upper right corner, the door opening hotspot on the right side of the display screen can be half the width of the regular right door opening hotspot, and its height can be from the top to the bottom of the screenshot preview image.

[0759] Figures 25H-25L show a solution in which the smart flow service is triggered only after entering the door opening hot zone twice, thereby preventing accidental touches.

[0760] 25H , the electronic device 100 displays an image 411 in the lower left corner area of ​​the display screen in response to the user's screenshot operation.

[0761] Afterwards, the electronic device 100 may detect a long press operation (hereinafter referred to as a long press operation) on the image 411. In response to the above user operation, the electronic device 100 may switch the image 411 to a suspended state as shown in FIG25I.

[0762] As shown in Figure 25H , when electronic device 100 detects a long press operation on image 411, the user's finger touch point on the display screen is located in the door opening hot zone on the left side of the display screen. At this time, although the touch point enters the door opening hot zone, the electronic device does not trigger the smart flow service, that is, the "door opening" animation effect is not triggered.

[0763] Referring to Figures 25I-25L , electronic device 100 can detect a user action in which a user drags image 411 outside the left door opening hot zone and then drags image 411 back into the left door opening hot zone. Only after detecting that the user's finger touch point on the display screen has entered the door opening hot zone a second time will the electronic device trigger the smart flow service, i.e., trigger the "door opening" animation.

[0764] Refer to Figure 25L, which shows the menu bar displayed after the electronic device triggers the smart flow service after the user's finger touches the touch point on the display screen for the second time and enters the door opening hot zone.

[0765] It can be seen that after generating a floating element, if the touch point of the user's finger on the display screen is located in the door opening hot zone for the first time, the smart flow service will not be triggered. The smart flow service will be triggered only when the touch point of the user's finger on the display screen moves out of the door opening hot zone and then enters the door opening hot zone for the second time. The first touch point located in the door opening hot zone here means that the touch point of the user operation used to make the element float on the display screen is located in the door opening hot zone. The door opening hot zones where the two touch points are located can be the door opening hot zone on the left and right sides of the display screen, respectively, or they can both be the door opening hot zones on the same side.

[0766] In this way, the user can more easily and conveniently choose to trigger the smart flow service or simply move the position of image 411 on the display screen while dragging image 411. Specifically, the user can trigger the smart flow service by dragging image 411 a second time to the door opening hot zone in Figure 25D; the user can move the position of image 411 on the display screen by directly dragging image 411 to an area outside the door opening hot zone in Figure 25D.

[0767] In other embodiments, if the electronic device detects a long press operation on image 411 and the touch point of the user's finger on the display screen is outside the door opening hot zone, then after detecting that the user drags image 411 to the door opening hot zone for the first time, the smart flow service can be triggered.

[0768] In the above embodiments, the operation of triggering the smart flow service may include, in addition to the touch point being in the door opening hot zone, staying in the door opening hot zone for a certain period of time (e.g., 600ms). This time period can be set as needed and is not limited here.

[0769] This is not limited to screenshot-type data to be transmitted. After a floating element is generated for other types of data to be transmitted, any of the above methods can be used to prevent accidental touches.

[0770] 25M-25R show a solution for preventing accidental touches that combines the above two methods.

[0771] In the anti-accidental-touch solution shown in Figures 25M-25R, the electronic device reduces the door opening hot zone, and triggers the smart flow service only after the touch point operated by the user enters the door opening hot zone twice.

[0772] FIG. 25M shows a user interface provided by the electronic device 100 .

[0773] Figure 25N shows a floating control 413 generated after the electronic device 100 detects the user operation in Figure 25M. The specific generation process of the control 413 can refer to the relevant description of Figures 22A to 22D above.

[0774] Referring to Figures 25N and 25O, there are two corresponding door opening hot zones in the display screen: area 412A enclosed by the dotted line L9 and the upper edge E1, left edge E3, and lower edge E2 of the display screen, and area 412B enclosed by the dotted line L4 and the upper edge E1, right edge E4, and lower edge E2 of the display screen.

[0775] Compared with the conventional door opening hot zone before the anti-mistouch scheme is enabled (such as the door opening hot zone in Figure 22G), the width of the door opening hot zone 414A on the left side of the display screen is shorter than the width of the conventional left door opening hot zone. For example, the width of the door opening hot zone 414A in Figure 25N can be 1 / 2 of the width of the door opening hot zone 412A in Figure 25C.

[0776] In some embodiments, the width of the door opening hot zone 414B on the right side of the display screen may also be shorter than the width of the conventional right door opening hot zone. For example, the width of the door opening hot zone 414B in Figure 25N may be 1 / 2 of the wi...

Claims

1. An information transmission method, characterized in that: The method comprises: Displaying a first interface, the first interface including a first sub-interface and a second sub-interface, the first sub-interface being used to display an application interface of a first application, the second sub-interface being used to display an application interface of a second application, the first sub-interface including first information, and the second sub-interface being used to transmit the first information to the second application under a first operation on the first information; In response to a second operation on the first information, displaying a second interface; the second interface includes the first information and at least one application service icon; In response to a third operation on the first information, the first information is transmitted to an application service corresponding to a target icon, where the target icon is any one of the at least one application service icon.

2. The method according to claim 1, characterized in that: In the first interface, The first sub-interface and the second sub-interface are displayed in a top-bottom split screen or a left-right split screen; Alternatively, the first sub-interface and the second sub-interface are displayed in a picture-in-picture format.

3. The method according to claim 1 or 2, characterized in that: The third operation is an operation of associating the first information with the target icon.

4. The method according to any one of claims 1 to 3, characterized in that: The third operation is an operation of moving the first information to the target icon.

5. The method according to claim 3 or 4, characterized in that: In the process of moving the first information to the target icon, the target icon is enlarged and / or the position of the target icon in the first interface is moved, and the direction of moving the target icon is opposite to the direction of moving the first information.

6. The method according to any one of claims 1 to 5, characterized in that: The at least one application service icon includes at least one first icon, and the at least one first icon is an application service icon recommended based on the first information.

7. The method according to any one of claims 1 to 6, characterized in that: The target icon includes a corresponding text description, and the text description is used to prompt the application or application service function corresponding to the target icon.

8. The method according to any one of claims 1 to 7, characterized in that: After transmitting the first information to the application service corresponding to the target icon, the method further includes: A third interface is displayed, wherein the third interface includes all the contents of the first interface, and the third interface also includes any one of a floating window and an information feedback bar, wherein the floating window is used to display the target icon or an application service interface corresponding to the target icon, and the information feedback bar is used to prompt the electronic device for the transmission result of the first information.

9. The method according to claim 8, characterized in that The third interface includes a floating window, the floating window is used to display the target icon, the third interface also includes second information, and after displaying the third interface, the method further includes: In response to a fourth operation on the second information, the second information is transmitted to the application service corresponding to the target icon, and the fourth operation is an operation of associating the second information with the floating window.

10. The method according to claim 8, characterized in that The third interface includes a floating window, the floating window is used to display the application service interface corresponding to the target icon, the floating window includes an expansion control, and after displaying the third interface, the method further includes: In response to a fifth operation on the expansion control, the application service interface displayed in the floating window is displayed in full screen.

11. The method according to claim 8, characterized in that The third interface includes an information feedback bar, the information feedback bar includes a jump control, and after displaying the third interface, the method further includes: In response to a sixth operation on the jump control, a fourth interface is displayed, where the fourth interface is an application service interface corresponding to the target icon.

12. The method according to any one of claims 1 to 11, characterized in that: The first interface further includes a first trigger area and a second trigger area, the first trigger area is larger than the second trigger area, the second operation includes a first sub-operation and a second sub-operation, the first sub-operation is an operation of moving the first information to the first trigger area, and the second sub-operation is an operation of moving the first information to the second trigger area, and the displaying of the second interface in response to the second operation on the first information in the first interface includes: In response to the first sub-operation, when the first information stays in the first trigger area for a first time period, a light guide effect is displayed in the first interface, where the light guide effect is used to indicate that the first information is moved to the second trigger area; In response to the second sub-operation, the second interface is displayed.

13. The method according to claim 12, characterized in that When the first sub-interface and the second sub-interface are displayed in a split-screen manner, the first trigger area exists in both the first interface and the second sub-interface, and the second trigger area exists in both the first interface and the second sub-interface.

14. The method according to any one of claims 6 to 13, characterized in that: The second interface also includes an icon area, and the at least one application service icon is displayed in the icon area. The at least one application service icon also includes at least one second icon, and the at least one second icon is a custom displayed application service icon. The at least one first icon is displayed above the at least one second icon.

15. The method according to claim 14, characterized in that The at least one application service icon includes at least one icon displayed in the icon area and at least one icon not displayed in the icon area, the target icon is not displayed in the icon area, the second interface further includes a third trigger area, and before responding to the second operation on the first information, the method further includes: In response to a seventh operation on the first information, the at least one icon displayed in the icon area is updated until the target icon is displayed in the icon area, and the seventh operation is an operation of moving the first information to the third trigger area.

16. The method according to any one of claims 1 to 15, characterized in that: An area of ​​the first information in the second interface is smaller than an area of ​​the first information in the first interface.

17. A method for information transmission, characterized in that: include: In response to any one of a global screenshot operation, a partial screenshot operation, and a long screenshot operation on the fifth interface, displaying a sixth interface, wherein the sixth interface includes a first image obtained by taking a screenshot of the fifth interface; In response to an eighth operation on the first image in the six interfaces, displaying a seventh interface; the seventh interface includes the first image and at least one application service icon; In response to a ninth operation on the first image in the seventh interface, the first image is transmitted to an application service corresponding to a target icon, where the target icon is any one of the at least one application service icon.

18. The method according to claim 17, characterized in that The first image is an image obtained by performing a global screenshot operation on the fifth interface, and in response to an eighth operation on the first image in the six interfaces, displaying the seventh interface includes: In response to a click operation on the first image in the sixth interface, an editing interface is displayed, the editing interface including the first image and editing options, the editing options being used to edit the first image; In response to the editing operation of the first image through the editing option, after the image content of the first image in the editing interface is updated, in response to the moving operation of the first image in the editing interface, the seventh interface is displayed, and the eighth operation includes the click operation, the editing operation and the moving operation.

19. A method for information transmission, characterized in that: include: In response to a partial screenshot operation on the eighth interface, a ninth interface is displayed, wherein the ninth interface includes a second image obtained by performing a partial screenshot operation on the eighth interface and a text recognition control, wherein the text recognition control is used to recognize text contained in the second image; In response to a click operation on the text recognition control, a tenth interface is displayed, wherein the tenth interface includes first text information obtained by recognizing the second image; In response to the tenth operation on the first text information in the tenth interface, displaying an eleventh interface; the eleventh interface includes the first text information and at least one application service icon; In response to an eleventh operation on the first text information in the eleventh interface, the first text information is transmitted to an application service corresponding to a target icon, where the target icon is any one of the at least one application service icon.

20. An electronic device, characterized in that: The electronic device comprises: one or more processors, a memory and a display screen; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 19.

21. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the electronic device executes the method as described in any one of claims 1-19.

22. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on an electronic device, the electronic device is caused to execute the method as claimed in any one of claims 1 to 19.

23. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 19 is implemented.

24. A data sharing method, characterized in that: The method comprises: Displaying a first interface, wherein the first interface includes a first element; A drag operation on the first element is detected; When the drag operation is a first drag operation, the touch start point of the first drag operation is on the first element and located in a first area of ​​the display screen, the touch end point is located in a second area of ​​the display screen, a first menu bar is displayed, the first menu bar includes an icon of a first application and an icon of a second application, and the first area and the second area do not overlap; After detecting that the first element is continued to be dragged into the area where the icon of the first application is located and released, sharing the first data corresponding to the first element to the first application; When the drag operation is a second drag operation, the touch start point of the second drag operation is on the first element and located in the second area, the touch end point is located in the second area of ​​the display screen, and the track of the second drag operation is located in the second area, the first menu bar is not displayed; When the drag operation is the third drag operation, the touch start point of the third drag operation is on the first element and located in the second area, the touch end point is located in the second area, and the track of the third drag operation passes through the first area to display the first menu bar.

25. The method according to claim 24, characterized in that When the drag operation is a first drag operation, in response to the first drag operation, displaying the first menu bar; When the drag operation is a second drag operation, the first menu bar is not displayed in response to the second drag operation; When the drag operation is a third drag operation, the first menu bar is displayed in response to the third drag operation.

26. The method according to claim 24 or 25, characterized in that When the drag operation is a first drag operation, the touch end point of the first drag operation stays in the second area. Display the first menu bar after a first duration; When the drag operation is a second drag operation, after the touch end point of the second drag operation stays in the second area for the first time period, the first menu bar is not displayed; When the drag operation is the third drag operation, the first menu bar is displayed after the touch end point of the third drag operation stays in the second area for the first time period.

27. The method according to any one of claims 24 to 26, characterized in that: The first dragging operation and the operation of continuing to drag the first element into the area where the icon of the first application is located and releasing the element are continuous and uninterrupted.

28. The method according to any one of claims 24 to 27, characterized in that: Before displaying the first interface, the method further includes: Displaying a second interface, wherein the second interface includes text; Detecting an operation of selecting a first text among the texts, displaying a toolbar on the first text, the toolbar including a plurality of controls for processing the first text; The displaying of the first interface includes: Upon detecting a long press operation on the first text, stopping displaying the toolbar and displaying the first interface; The first element includes a preview image of the first text, and the first data includes the first text.

29. The method according to any one of claims 24 to 28, characterized in that: Before displaying the first interface, the method further includes: When the second interface is displayed, a screenshot operation is detected; The displaying of the first interface includes: In response to the screenshot operation, the first interface is displayed, the first element includes a preview image of the screenshot image of the second interface, and the first data includes the screenshot image.

30. The method according to any one of claims 24 to 29, characterized in that: Before displaying the first interface, the method further includes: displaying a second interface, the second interface including an icon of the first image and an icon of the second image; detecting an operation of selecting the icon of the first image; The displaying of the first interface includes: Detecting a long press operation on the icon of the first image, and displaying the first interface; The first element includes a preview image of an icon of the first image, and the first data includes the first image.

31. The method according to any one of claims 24 to 30, characterized in that: The first menu bar and the second area where the touch endpoint of the first drag operation or the third drag operation is located are on the same side of the display screen.

32. The method according to any one of claims 24 to 31, characterized in that: The method further comprises: When the dragging operation is the first dragging operation, or when the dragging operation is the third dragging operation, when the first menu bar is displayed, the first interface is also adjusted to a perspective style of a first side, where the first side is a side of the first menu bar in the display screen.

33. The method according to any one of claims 24 to 32, characterized in that: The first interface also includes a video being played, and the method further includes: When the dragging operation is the first dragging operation, or when the dragging operation is the third dragging operation, when the first menu bar is displayed, a screenshot image of the first interface is displayed, and the screenshot image of the first interface is displayed in a perspective style of a first side, where the first side is a side of the first menu bar in the display screen.

34. The method according to any one of claims 24 to 33, characterized in that: The second area includes: a rectangular area at the left edge of the display screen, and a rectangular area at the right edge of the display screen, and the height of the rectangular area is the same as the height of the display screen.

35. The method according to any one of claims 24 to 34, characterized in that: When the drag operation is a first drag operation, a track of the first drag operation passes through the third area, and the third area includes the second area; The method further comprises: In response to an operation of dragging the first element to the third area in the first drag operation, displaying a light strip, wherein the light strip and the first menu bar are located on the same side of the display screen; The displaying of the first menu bar comprises: In response to an operation of dragging the first element to the third area and then to the second area in the first drag operation, the light band is stopped from being displayed and a first menu bar is displayed.

36. The method according to claim 35, characterized in that The third area includes: a rectangular area at the left edge of the display screen, and a rectangular area at the right edge of the display screen. The height of the third area is the same as the height of the display screen.

37. The method according to any one of claims 24 to 36, characterized in that: The first menu bar further includes an icon of a third application and an icon of a fourth application, and the icon of the first application, the icon of the second application, the icon of the third application, and the icon of the fourth application are arranged in sequence from top to bottom; after displaying the first menu bar, the method further includes: A fourth drag operation of continuing to drag the first element within a fourth area is detected, the fourth area includes the second area where a touch endpoint of the first drag operation or the third drag operation is located, a touch start point of the fourth drag operation is the same as a touch endpoint of the first drag operation or the third drag operation, and the touch endpoint of the fourth drag operation is closest to an icon of the second application in the first menu bar; In response to the fourth drag operation, the icon of the fourth application is displayed as a basic size, the icon of the second application is displayed as a size that is enlarged by a first ratio on the basic size, and the icon of the first application and the icon of the second application are displayed as a basic size. The icons of the three applications are displayed as sizes that are enlarged by a second ratio based on the basic size; the first ratio is larger than the second ratio.

38. The method according to claim 37, characterized in that The fourth area includes: a rectangular area including the second area where the touch endpoint of the first drag operation or the third drag operation is located, and a height of the fourth area is the same as a height of the display screen.

39. The method according to claim 37 or 38, characterized in that After detecting a fourth drag operation of continuing to drag the first element within a fourth area, the method further includes: detecting a fifth drag operation of continuing to drag the first element to a fifth area, wherein the fifth area does not overlap with the fourth area; In response to the fifth drag operation, each icon in the first menu bar is displayed as the basic size.

40. The method according to any one of claims 24 to 39, characterized in that: When the drag operation is the first drag operation, or when the drag operation is the third drag operation, after displaying the first menu bar, the method further includes: detecting a sixth drag operation of continuing to drag the first element to a sixth area, the sixth area not overlapping the second area, and the sixth area being located on an opposite side of the first menu bar in the display screen; In response to the sixth drag operation, the first menu bar stops being displayed.

41. The method according to any one of claims 24 to 40, characterized in that: The method further comprises: Displaying a third interface, wherein the third interface includes the second element; A drag operation on the second element is detected; When the drag operation is the seventh drag operation, the touch start point of the seventh drag operation is on the second element and located in the seventh area of ​​the display screen, the touch end point is located in the eighth area of ​​the display screen, the second menu bar is displayed, and the seventh area and the eighth area do not overlap; When the drag operation is an eighth drag operation, the touch start point of the eighth drag operation is on the second element and located in the eighth area, the touch end point is located in the eighth area of ​​the display screen, and the track of the eighth drag operation passes through the seventh area, and the second menu bar is displayed; When the drag operation is not the seventh drag operation or the eighth drag operation, not displaying the second menu bar; The eighth region is a partial region of the second region.

42. The method according to claim 41, characterized in that The eighth area includes: a rectangular area at the left edge of the display screen, and a rectangular area at the right edge of the display screen; the height of the eighth area is the same as the height of the second area, and the width of the eighth area is half of the width of the second area.

43. The method according to claim 41 or 42, characterized in that The third interface is provided by a fifth application, and the fifth application includes a text editing application.

44. The method according to any one of claims 24 to 43, characterized in that The second area when the display screen is in a horizontal state is different from the second area when the display screen is in a vertical state.

45. The method according to any one of claims 24 to 44, characterized in that The first interface is provided by a sixth application, and the sixth application is different from the first application.

46. ​​An electronic device, characterized in that: include: A memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to implement the method according to any one of claims 24 to 45.

47. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 24 to 45 is implemented.

48. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 24 to 45.

49. A method for recommending services, characterized in that: The method is applied to an electronic device, and the method comprises: Get the first content; converting the first content into a first entity, wherein the first entity includes an entity category and an entity value; Identifying a corresponding first intent according to the first entity; determining a service that supports the first intent; An icon for the service is shown in the display.

50. The method according to claim 49, characterized in that Identifying the corresponding first intent according to the first entity specifically includes: Determine, according to a mapping relationship between entity categories and / or modalities and intent categories, an entity category of the first entity and / or a first intent category corresponding to the modality to which the first content belongs; A first intent is determined according to the first intent category and the entity value.

51. The method according to claim 49 or 50, characterized in that Identifying the corresponding first intent according to the first entity specifically includes: In the global intent system, find a first intent category whose label is the entity category of the first entity; Filling the entity value of the first entity into the slot indicated by the first intent category to obtain the first intent; The global intent system includes information of multiple types of intents, and the information of each type of intent includes the following items of the corresponding category: label, slot name.

52. The method according to claim 51, characterized in that The information of each type of intent also includes one or more of the following for the corresponding category: identification, level, category of the value filled in the slot, and necessity of the slot.

53. The method according to claim 51 or 52, characterized in that The global intent system is pre-stored in the electronic device, or downloaded from the network by the electronic device.

54. The method according to any one of claims 51 to 53, characterized in that The information of each type of intent also includes the identifier of the corresponding category, and determines the services that support the first intent, including: In the global service system, find a service whose identifier of a supported intent category is a first identifier, where the first identifier is an identifier of a category to which the first intent belongs; Determine a service whose identifier of the intent category supported in the global service system is the first identifier as a service supporting the first intent; The global service system includes information of multiple services, and the information of each service includes: an identifier of an intent category supported by the corresponding service.

55. The method according to claim 54, characterized in that The information of each service also includes one or more of the following items of the corresponding service: ID, name, icon, type, slot name, package name of the application to which it belongs, name of the component, URL, and parameter name for transmitting data.

56. The method according to claim 55, characterized in that The global service system is pre-stored in the electronic device, or downloaded from the network by the electronic device.

57. The method according to claim 55 or 56, characterized in that The information of each service also includes: a slot name, a parameter name for transmitting data, and the method further includes: receiving a user operation acting on an icon of a first service among the service icons; Passing the value of the first slot in the first intent to the first parameter of the first service, where the first slot is the slot indicated by the slot name of the information of the first service, and the first parameter is the parameter indicated by the parameter name of the information of the first service; The first service is started based on the first parameter.

58. The method according to any one of claims 49 to 57, characterized in that Acquiring the first content specifically includes: acquiring the first content through the entrance of the first service; Before acquiring the first content, the method further includes: Subscribe to an intent category supported by the first service, to which the first intent belongs.

59. The method according to any one of claims 49 to 57, characterized in that Acquiring the first content specifically includes: acquiring the first content through the entrance of the first service; Before acquiring the first content, the method further includes: Subscribe to a service supported by the first business, where the first service belongs to the services supported by the first business.

60. The method according to any one of claims 49 to 59, characterized in that After determining a service that supports the first intent and before displaying an icon of the service on a display screen, the method further includes: Multiple services supporting the first intent are sorted, and icons of the services displayed on the display screen are arranged in sequence according to the sorting of the multiple services.

61. The method according to claim 60, characterized in that Before sorting the multiple services supporting the first intent, the method further includes: The multiple first intentions corresponding to the first entity are sorted, and the sorting of the multiple services is obtained according to the sorting of the multiple first intentions.

62. The method according to claim 61, characterized in that Sorting the multiple first intentions corresponding to the first entity specifically includes: Using a click rate prediction model to rank multiple first intents corresponding to the first entity; in, The click rate prediction model is trained based on multiple sets of first historical behavior data, where a set of first historical behavior data includes: time features, content features, and click rate features; The time feature indicates one or more of the following: whether the time when the electronic device historically acquired the content was a weekday, a holiday, or a time period; The content feature indicates one or more of the following: a source application of the content acquired by the electronic device, a length of the content, an entity category of an entity corresponding to the content, and a modality of the content; The click rate feature indicates: after the electronic device obtains the content and recommends the service, the user's click rate on the recommended service.

63. The method according to any one of claims 49 to 62, characterized in that The method further comprises: Acquire multiple sets of second historical behavior data; Determining, based on the plurality of sets of second historical behavior data, that a first entity category and / or a first modality has a mapping relationship with a second intent category; The second set of historical behavior data includes: behavior data of the electronic device starting the service under the trigger of the user within a first time period after the electronic device displays the icon of the recommended service on the display screen according to the content corresponding to the first entity category and / or the first modality input by the user and does not receive a user operation acting on the icon of the service, and after the electronic device stops displaying the icon of the service; Among the multiple groups of second historical behavior data, K services with the highest startup frequency support the second intent category.

64. The method according to any one of claims 49 to 63, characterized in that The method further comprises: Acquire multiple sets of second historical behavior data; determining a third intent category based on the plurality of sets of second historical behavior data; The second set of historical behavior data includes: behavior data of the electronic device starting the service under the trigger of the user within a first time period after the electronic device displays the icon of the recommended service on the display screen according to the content corresponding to the first entity category and / or the first modality input by the user and does not receive a user operation acting on the icon of the service, and after the electronic device stops displaying the icon of the service; The third intent category is defined based on the N services with the highest startup frequencies in the multiple sets of second historical behavior data.

65. The method according to claim 64, characterized in that The method further comprises: Determine that the first entity category and / or the first modality have a mapping relationship with a third intent category.

66. The method of claim 50, wherein: After acquiring the first content, the method further includes: Loads mappings between entity categories and / or modalities and intent categories.

67. The method according to any one of claims 50 to 52, characterized in that After acquiring the first content, the method further includes: Load the global intent system.

68. The method according to any one of claims 53 to 56, characterized in that After acquiring the first content, the method further includes: The global service system is loaded.

69. The method according to any one of claims 49 to 68, characterized in that The first content includes content of a first modality and content of a second modality, and converting the first content into a first entity specifically includes: Starting a first thread and a second thread in parallel, wherein the first thread is used to extract features of content in the first modality, and the second thread is used to extract features of content in the second modality; A first entity is obtained according to characteristics of the content of the first modality and characteristics of the content of the second modality.

70. The method according to claim 69, characterized in that The first thread includes one or more threads, the second thread includes one or more threads, and one thread is used to call an entity recognition capability to recognize an entity.

71. An electronic device, characterized in that: include: A memory, a processor and a computer program stored on the memory, wherein the processor executes the computer program to implement the method according to any one of claims 49 to 70.

72. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 49-70 is implemented.

73. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 49 to 70.

Images