Interaction method, display method, electronic equipment and readable medium

Abstract

The invention relates to the technical field of electronics, and discloses an interaction method, a display method, electronic equipment and a readable medium. Dragging inside an application interface and between applications is realized through a dragging function, so that the application using the interaction method provided by the embodiment of the invention can realize the dragging function inside the application and can also realize the dragging function between the applications, after a user triggers a dragging event, the dragging intention (dragging type) of the user is firstly judged, and then the dragging intention (dragging type) of the user is judged. And then the dragging function corresponding to the determined dragging type is executed, so that the use experience of the user is improved.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to an interaction method, a display method, an electronic device and a readable medium. Background Art

[0002] The user can drag a display element in the application interface, such as an icon (hereinafter referred to as a drag object), from one location to another location in the application interface (hereinafter referred to as an internal drag function).

[0003] For example, Figure 1a As shown, the user interface 10a of the mobile phone 10 includes an icon 101 corresponding to "content A", an icon 102 corresponding to "content B", and an icon 103 corresponding to "content C". If a user long presses the icon 103, as the user drags the icon 103 to the preset exchange area of ​​the icon 102, Figure 1b As shown, the positions of icon 102 and icon 103 are exchanged, and the mobile phone 10 displays as shown in FIG. Figure 1c User interface 10c is shown.

[0004] Or, for example, if it is detected that the user is in Figure 1a The user interface shown in FIG. 1 long presses the icon 103, and as the user drags the icon 103 to the preset merge area of ​​the icon 102, as shown in FIG. Figure 1d As shown, the icon 102 and the icon 103 are merged, and the mobile phone 10 displays as follows Figure 1e User interface 10e is shown.

[0005] The user can also drag the icon in the application interface to another location outside the application (hereinafter referred to as the external drag function).

[0006] For example, Figure 2a As shown, the user interface 20a of the mobile phone 10 includes an application interface 201 of the photo album application and a main interface 202 of the mobile phone 10. The application interface 201 includes pictures 2011 and 2012, and the main interface 202 includes icons of multiple applications. If it is detected that the user has long pressed the picture 2012, such as Figure 2b As shown, a touch view 2013 corresponding to the image 2012 is generated. As the user drags the touch view 2013 to the preset receiving area of ​​the main interface 202, the main interface 202 receives the touch view 2013 and processes it into a desktop card 2021. The mobile phone 10 displays as shown in FIG. Figure 2c User interface 20c is shown.

[0007] or for example, Figure 2dAs shown, the user interface 20d of the mobile phone 10 includes an application interface 201 of the photo album application and an application interface 203 of the memo application. Among them, the application interface 201 includes pictures 2011 and 2012, and the application interface 203 includes the content of "Memo 1". If it is detected that the user has long pressed the picture 2011, such as Figure 2e As shown, a touch view 2014 corresponding to the image 2011 is generated. As the user drags the touch view 2014 to the preset receiving area of ​​the application interface 203, the application interface 203 receives the touch view 2014 and processes it into the image content 2031 of "Memo 1", and the mobile phone 10 displays as shown. Figure 2f The user interface 20f is shown to enable content sharing between applications.

[0008] Current electronic devices implement internal dragging and external dragging in different ways. When it is detected that a display element in an application has started to be dragged, since it is impossible to determine whether the user's dragging intention is to drag inside the application interface or to drag outside the application interface, the electronic device can only choose one of the internal dragging and external dragging functions to respond to the user's drag. If the internal dragging function is selected and the user's dragging intention is to drag outside the application interface, external dragging cannot be implemented. If the external dragging function is selected and the user's dragging intention is to drag inside the application interface, internal dragging cannot be implemented, affecting the user experience. Summary of the invention

[0009] The purpose of the present application is to provide an interaction method, a display method, an electronic device and a readable medium.

[0010] A first aspect of the present application provides an interaction method, which is applied to an electronic device, including: displaying a first application interface of a first application, the first application interface including a first display element located at a first position; detecting a user's dragging operation on the first display element, and obtaining position information of the first display element during the dragging process; determining that the first display element is dragged to a second position of the first application interface; and based on the second position, determining a display mode of the first display element.

[0011] In the embodiment of the present application, the first application interface may be a user interface of the first application, for example Figure 7c The user interface 702 of the favorite application is shown; the first display element may be a drag object, such as Figure 7c As shown in Figure 7021, the first position may be the initial position information of the dragged object; the dragging operation may be a long press operation; and the second position may be the position information of the dragged object during the dragging process. After detecting that the user has long pressed the dragged object of the first application, the position information of the dragged object is obtained in real time, and based on the position of the dragged object in the first application, the current drag event is determined, and the dragged object is controlled to execute the corresponding event.

[0012] It can be understood that the interaction method provided in the embodiment of the present application first determines the user's dragging intention (drag type) when the user triggers a drag event, and then executes the drag function corresponding to the determined drag type, thereby improving the user's experience.

[0013] In a possible implementation of the first aspect above, determining a display mode of the first display element based on the second position includes: corresponding to the end of the user's dragging operation at the second position and the second position being in a first receiving area of ​​the first application, displaying the first display element at the second position.

[0014] In an embodiment of the present application, the first receiving area may be a preset receiving area of ​​the first application. When it is detected that the contact view is located in the preset receiving area of ​​the first application (e.g., a blank area, etc.), and the system service of the electronic device determines that the current drag has ended, for example, it is detected that the user cancels the long press operation, then the first view of the dragged object is displayed in the preset receiving area.

[0015] In a possible implementation of the first aspect above, the first application interface also includes a second display element located at a third position, and based on the second position, a display mode of the first display element is determined, including: the third position corresponding to the second position and corresponding to the second display element satisfies an exchange condition, and the second display element is displayed at the first position, wherein the exchange condition includes: there is an overlapping area between the third position and the second position, and the area of ​​the overlapping area is greater than or equal to the first threshold and less than the second threshold.

[0016] In an embodiment of the present application, the second display element may be a second view, which is a view corresponding to other display elements (such as icons, pictures, text, etc.) in the first application except the dragged object; the third position may be the display position of the second view; the first threshold may be an exchange threshold; and the second threshold may be a merge threshold. After the second view of the first application determines that the event type is an exchange event, it exchanges the display position with the touch view. The exchange condition may be that there is an overlapping area between the display position of the touch view and the display position of the second view, and the area of ​​the overlapping area is greater than or equal to the exchange threshold (for example, 50% of the area of ​​the second view) and less than the merge threshold (for example, 80% of the area of ​​the second view).

[0017] In a possible implementation of the first aspect above, determining the display mode of the first display element based on the second position also includes: detecting that the user's drag operation ends in a first time period after the first display element is displayed at the second position, and displaying the first display element at the second position.

[0018] In an embodiment of the present application, if after the swap occurs, the contact view stays at the swapped position (the position of the second view) for a first period of time and then detects that the user cancels the drag operation, the dragging animation is to display the first view at the swapped position.

[0019] In a possible implementation of the first aspect above, it also includes: detecting that the user's drag operation has not ended in a first time period after the first display element is displayed at the second position, and setting the end position of the return animation of the first display element to the second position; detecting that the user's drag operation ends after the first display element moves to a fourth position, and displaying the return animation of the first display element, wherein the fourth position is located in an invalid area.

[0020] In the embodiment of the present application, the fourth position may be an invalid position; the invalid area may be an invalid position (a position that does not overlap with the second view and does not belong to the preset receiving area of ​​the first application). If the contact view moves to an invalid position (a position that does not overlap with the second view and does not belong to the preset receiving area of ​​the first application) after the exchange, the user cancels the drag operation, the dragging animation is a return animation, and the starting position of the return animation is the current invalid position, and the end position is the position after the exchange.

[0021] In a possible implementation of the first aspect above, the end position of the return animation of the first display element is set to the second position in the following manner: the view module of the application layer of the electronic device sends the position information of the second position to the drag controller of the frame layer of the electronic device; the drag controller sends the position information of the second position to the drag state of the frame layer, and controls the drag state to set the end position of the return animation of the first display element to the second position, and sends it to the display module of the frame layer; and after detecting that the first display element moves to the fourth position, the user's drag operation ends, and the return animation of the first display element is displayed, including: the display module displays the return animation of the first display element moving from the fourth position to the second position based on the position information of the second position and the position information of the fourth position.

[0022] In an embodiment of the present application, the view module of the application layer may be the view frame of the application layer; the framework layer may be the application framework layer; and the display module of the framework layer may be the display frame of the application framework layer. The process of generating the return animation of the drag by the electronic device may be: the view frame of the application layer sends the updated starting position (the position of the second view) to the drag controller of the application framework layer; the drag controller sends the position information of the contact view to the drag state of the application framework layer, and controls the drag state to set the end position of the return animation of the dragged object to the updated starting position, and sends it to the display frame of the application framework layer; the display frame displays the return animation of the first display element moving from the invalid position to the updated starting position based on the updated starting position and the invalid position.

[0023] In a possible implementation of the first aspect above, the first application interface includes a second display element, and based on the second position, determining the display mode of the first display element, also includes: corresponding to the end of the user's drag operation at the second position, and the third position corresponding to the second position and the second display element meets the merging condition, displaying the first display element and the second display element at the third position, wherein the merging condition includes that the third position overlaps with the second position, and the area of ​​the overlapping area is greater than or equal to the second threshold.

[0024] In an embodiment of the present application, after the second view of the first application determines that the event type is a merge event, it merges the display position with the touch view. The merging condition can be that there is an overlapping area between the display position of the touch view and the display position of the second view, and the area of ​​the overlapping area is greater than or equal to the merging threshold (for example, 80% of the area of ​​the second view).

[0025] In a possible implementation of the first aspect above, a user's dragging operation on the first display element is detected, and the position information of the first display element during the dragging process is obtained, including: a first application in an application layer of an electronic device detects the user's dragging operation on the first display element, generates a drag start event, and sends the drag start event to a view module in a framework layer of the electronic device; the view module of the framework layer generates a contact view corresponding to the first display element based on the drag start event, and controls the display module of the framework layer to display the contact view and set the first display element to be invisible; a drag controller of the framework layer obtains a drag move event during the dragging process, wherein the drag move event includes the position information of the contact view; the drag controller of the framework layer sends the position information to the drag state of the framework layer, and controls the drag state to send the position information to the display module of the framework layer; and when the drag operation is detected to be ended, the display module of the framework layer controls the contact view to be invisible.

[0026] In the embodiment of the present application, the view module of the framework layer may be the view framework of the application framework layer; setting the first display element invisible may be hiding the first view; controlling the touch view invisible may be hiding the touch view. When the first application detects a user's long press operation on the dragged object in the first application, the first application sends a drag start event corresponding to the first view to the view framework in the application framework layer; the view framework generates a touch view corresponding to the dragged object, that is, a new view corresponding to the touch point of the user's finger (for example, Figure 7d The drag controller receives the drag move event and sends the received drag move event to the drag state; the drag state sends the touch view and its position information to the display frame according to the touch coordinates in the received drag move event.

[0027] A second aspect of the present application provides a display method, which is applied to an electronic device, including: displaying a first application interface of a first application, the first application interface including a first display element located at a first position and a second display element located at a second position; detecting that a user drags the first display element to a third position, and displaying the second display element at the first position, wherein the third position satisfies an exchange condition; detecting that the user continues to drag the first display element to a fourth position and ends the dragging operation, and displaying a return animation of the first display element, wherein the end position of the first display element in the return animation is the second position.

[0028] In an embodiment of the present application, a user interface of a first application and a user interface of a second application are displayed, and the user interface of the first application includes a drag object and other display elements. It can be understood that the view corresponding to the drag object is the first view, and the view corresponding to other display elements is the second view. When a user's drag operation on an object is detected (such as long press to move), and the position information of the drag object and the position information of the second view meet the exchange condition, the display position of the drag object and the second view are exchanged. When it is detected that the user continues to drag the drag object to an invalid position and ends the drag, the return animation of the drag object is displayed.

[0029] In a possible implementation of the second aspect above, the exchange condition includes: there is an overlapping area between the third position and the second position, and the area of ​​the overlapping area is greater than or equal to the first threshold and less than the second threshold.

[0030] In an embodiment of the present application, the exchange condition may be that there is an overlapping area between the display position of the contact view and the display position of the second view, and the area of ​​the overlapping area is greater than or equal to the exchange threshold (for example, 50% of the area of ​​the second view) and less than the merge threshold (for example, 80% of the area of ​​the second view).

[0031] In a possible implementation of the second aspect above, it is detected that the user drags the first display element to a third position, and the second display element is displayed at the first position, including: the touch drive of the electronic device detects that the user drags the first display element to the third position, and sends the position information of the third position to the drag controller of the frame layer of the electronic device; the drag controller sends the position information of the third position to the drag state of the frame layer of the electronic device, and controls the drag state to send the position information to the view module of the first application of the application layer of the electronic device; after the view module of the first application determines that the third position meets the exchange condition based on the position information of the third position, it sends the position information of the second position to the drag controller, and controls the display module of the frame layer of the electronic device to display the second display element at the first position; the drag controller sends the position information of the second position to the drag state of the frame layer, and controls the drag state to set the end position of the return animation of the first display element to the second position, and sends it to the display module of the frame layer.

[0032] In an embodiment of the present application, the touch driver detects the user's dragging operation on the drag object, and sends the position information of the current drag object to the drag controller. The drag controller sends the position information of the current drag object to the drag state, and controls the drag state to send the position information to the second view of the first application. After the second view of the first application determines that the position information of the current drag object meets the exchange condition based on the position information of the current drag object, it sends the position information of the current drag object to the drag controller, and controls the display framework to display the second view at the position of the first view. The drag controller sends the position of the second view, that is, the initial position of the updated first view, to the drag state, and controls the drag state to set the end position of the return animation of the drag object to the position of the second view, and sends it to the display module of the framework layer.

[0033] In a possible implementation of the second aspect above, it is detected that the user continues to drag the first display element to the fourth position and ends the dragging operation, and the return animation of the first display element is displayed, including: the touch driver detects that the user drags the first display element to the fourth position and ends the dragging operation, and sends the position information of the fourth position and the end drag information to the drag controller; the drag controller sends the position information of the fourth position to the drag state, and controls the drag state to send the position information of the fourth position to the display module; the display module displays the return animation of the first display element moving from the fourth position to the second position based on the position information of the second position and the position information of the fourth position.

[0034] In the embodiment of the present application, after the touch driver detects that the user drags the drag object to an invalid position and cancels the drag, the invalid position and the end of the drag information are sent to the drag controller. The drag controller sends the position information of the invalid position to the drag state, and controls the drag state to send the position information of the invalid position to the display module; the display module displays the return animation of the drag object moving from the invalid position to the initial position of the updated first view based on the initial position and the invalid position of the updated first view.

[0035] The third aspect of the present application provides an electronic device, characterized in that it includes: a display module for displaying a first application interface of a first application, the first application interface including a first display element located at a first position; a touch drive for detecting a user's dragging operation on the first display element, obtaining position information of the first display element during the dragging process, and sending the position information to a drag controller and the first application; and a drag controller for determining a display mode of the first display element based on the first display element being dragged by the user to a second position of the first application interface.

[0036] In the embodiment of the present application, the display module may be a display frame.

[0037] In a possible implementation of the third aspect above, it also includes: a first application, used to generate a drag start event, and send the drag start event to the view module of the frame layer of the electronic device; a view module, used to generate a contact view corresponding to the first display element based on the drag start event, and control the display module of the frame layer to display the contact view and set the first display element invisible; a drag state, used to receive the position information sent by the drag controller, and send the position information to the display module of the frame layer; and the drag controller is also used to obtain a drag move event during the dragging process, wherein the drag move event includes the position information of the contact view, and sends the position information to the drag state of the frame layer, and controls the drag state to send the position information to the display module of the frame layer; the display module is also used to display the return animation of the first display element based on the position information.

[0038] In the embodiment of the present application, the view module may be a view framework.

[0039] A fourth aspect of the present application provides a readable medium on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes any one of the methods of the first and second aspects above.

[0040] The fifth aspect of the present application provides an electronic device, comprising: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the processors of the electronic device, for executing any one of the methods of the first and second aspects above. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figures 1a to 1e According to an embodiment of the present application, a schematic diagram of a user interface change of an internal drag function is shown;

[0043] Figures 2a to 2f A schematic diagram showing changes in a user interface of an external drag function according to an embodiment of the present application is shown;

[0044] Figure 3 According to an embodiment of the present application, a schematic flow chart of a method for implementing internal dragging is shown;

[0045] Figure 4 According to an embodiment of the present application, a schematic diagram of a software architecture of an electronic device corresponding to an internal drag function is shown;

[0046] Figure 5 According to an embodiment of the present application, a schematic flow chart of a method for implementing external dragging is shown;

[0047] Figure 6 According to an embodiment of the present application, a schematic diagram of the software architecture of an electronic device corresponding to an external drag function is shown;

[0048] Figures 7a to 7h According to an embodiment of the present application, a schematic diagram of a user interface change process of dragging between applications and dragging within an application is shown;

[0049] Figure 8 According to an embodiment of the present application, a flow chart of an interaction method is shown;

[0050] Fig. 9 According to an embodiment of the present application, a schematic diagram of contact position information is shown;

[0051] Fig.10 A schematic diagram of an exchange event is shown according to an embodiment of the present application;

[0052] Fig.11 A schematic diagram of a merging event is shown according to an embodiment of the present application;

[0053] Fig.12 According to an embodiment of the present application, a schematic diagram of external dragging is shown;

[0054] Fig.13 According to an embodiment of the present application, a schematic diagram of the interaction process of a method of dragging within an application is shown;

[0055] Fig.14 According to an embodiment of the present application, a schematic diagram of a software architecture of an electronic device is shown;

[0056] Fig.15 According to an embodiment of the present application, a schematic diagram of the interaction process of various modules in the process of updating the initial display position of a contact point view is shown;

[0057] Fig.16 According to an embodiment of the present application, a structural schematic diagram of an electronic device is shown. DETAILED DESCRIPTION

[0058] Illustrative embodiments of the present application include, but are not limited to, an interaction method, a display method, an electronic device, and a readable medium.

[0059] As mentioned above, since current electronic devices implement internal drag functions and external drag functions differently, there may be a problem that the user's dragging cannot be implemented because the drag function selected by the electronic device is different from the user's dragging intention, affecting the user's experience.

[0060] In some embodiments, the internal drag function pre-sets the movement mode of the drag object. For example, the drag event corresponding to moving the drag object upward is to move the display position of the drag object upward by a preset distance. There is no need to obtain the position information of the drag object in real time to determine the drag event. Specifically, Figure 3 A flow chart of a method for implementing internal drag is shown. Figure 3 As shown, the process includes but is not limited to the following steps:

[0061] S301: Detecting a user's long press and drag operation on an object, and obtaining a moving direction and position of the touch point.

[0062] For example, reference Figure 1a , detect the user's long press operation of icon 103, and obtain the moving direction and position of the contact.

[0063] S302: Determine the type of the drag event based on the moving direction and position of the contact point.

[0064] For example, reference Figure 1b As the user drags icon 103 to the preset exchange area of ​​icon 102, the type of the drag event is determined to be an upward event, and one icon is moved upward.

[0065] S303: Execute a corresponding drag response based on the type of the drag event.

[0066] For example, reference Figure 1c , move the position of icon 103 up one icon, that is, exchange the positions of icon 102 and icon 103.

[0067] Combine the following Figure 4 The software architecture diagram shown in Figure 3 The implementation method of internal dragging shown is explained.

[0068] like Figure 4 As shown, the application layer includes a first application, wherein the first application includes a first view (a view corresponding to the dragged object) and a view control. The view control includes a motion effect auxiliary class and a motion effect auxiliary class callback.

[0069] In some embodiments, the motion assistance class obtains touch events reported by the touch driver (such as the movement direction and position information of the touch point), matches the preset movement mode and the type of the current drag event based on the received touch events, and generates a drag response for the first view, such as moving an icon up.

[0070] The animation auxiliary class callback returns the drag response generated by the animation auxiliary class to the first view.

[0071] The application framework (Framework) layer includes the view framework (View) and the display framework (Surface), etc.

[0072] The hardware abstraction layer (HAL) is used to provide the application framework layer with a common interface for calling drivers in the kernel layer and distribute input events sent by the kernel layer to the upper layer, namely the application framework layer.

[0073] The kernel layer includes display drivers and touch drivers. The kernel layer can send input events (such as drag and drop events) reported by the touch driver to the upper layer (i.e., the application framework layer) for processing through the hardware abstraction layer (HAL).

[0074] It is understandable that the above Figure 3 and Figure 4 The internal drag function shown pre-sets the movement mode of the drag object. It only needs to match the corresponding movement mode and drag event through information such as the movement direction and position information of the contact, and execute the response corresponding to the drag event. After matching the movement mode, there is no need to obtain the position information of the drag object in real time.

[0075] In some embodiments, the implementation of the external drag function relies on the system service of the electronic device to assist each application in dragging the content of one application to another application. After the system service receives the drag start event corresponding to the drag object, it generates a contact view, obtains the touch event (such as the location information of the contact view), and sends the received drag start event and touch event to the second application, so that the second application can determine whether to receive the content corresponding to the contact view.

[0076] Figure 5 A flow chart of a method for implementing external drag is shown. Figure 5 As shown, the process includes but is not limited to the following steps:

[0077] S501: Detecting a user long pressing operation on a drag object in a first application, and sending a drag start event to a system service of the electronic device.

[0078] For example, reference Figure 2d , detecting the user's long press operation on the picture 2011 of the album application, and sending a drag start event to the system service of the mobile phone 10.

[0079] S502: The system service of the electronic device generates a touch point view corresponding to the drag object, and sends the received drag start event to the second application.

[0080] For example, reference Figure 2e , the system service of the mobile phone 10 generates a touch view 2014 corresponding to the image 2011, and sends the received drag start event to the memo application.

[0081] S503: The drag listener of the second application detects that the drag object enters a preset receiving area of ​​the second application.

[0082] For example, reference Figure 2e , the drag listener of the memo application detects that the touch view 2014 enters the preset receiving area of ​​the memo application.

[0083] S504: Acquire the content of the dragged object, analyze and process it, and determine whether to receive the dragged object.

[0084] For example, reference Figure 2f , the memo application obtains the content of the picture 2011, analyzes and processes the content of the picture 2011, receives the picture 2011 and uses it as the picture content 2031 of "Memo 1".

[0085] Combine the following Figure 6 The software architecture diagram shown in Figure 5 The implementation method of internal dragging shown is explained.

[0086] like Figure 6As shown, the application layer includes a first application and a second application. The first application includes a first view (a view corresponding to the dragged object), and the second application includes a view system (a view system composed of a view corresponding to the window of the second application and views corresponding to each display element in the second application).

[0087] The application framework layer includes the view framework, display framework, and system services (SystemServer).

[0088] In some embodiments, when the user's finger long presses the drag object in the first application, the first application sends a drag start event (startDragAndDrop) to the view framework, the view framework receives the drag start event, generates a contact view, and sends the contact view to the display framework.

[0089] The system service includes a window manager (WindowManager), which includes a session control (Session) class, a drag controller (DragDropController), a drag state (DragState), and a drag input event receiver (DragInputEventReceiver).

[0090] In some embodiments, the view framework sends a task of processing drag (performDrag) to the drag controller by calling the session control class. The drag controller creates a drag state, and the drag state distributes a drag event (drag start event) to the second application of the application layer by calling the session control class.

[0091] In some embodiments, after receiving the drag event, the drag listener corresponding to the view system of the second application determines whether the touch view enters the preset receiving area of ​​the second application based on the positional relationship between the corresponding view in the second application and the touch view. If the judgment result is yes, the second application obtains the content of the dragged object, analyzes and processes it, and determines whether to receive the dragged object.

[0092] The hardware abstraction layer is used to provide the application framework layer with a common interface for calling drivers in the kernel layer, and distribute input events sent by the kernel layer to the upper layer, namely the application framework layer.

[0093] The kernel layer includes display drivers and touch drivers. The kernel layer can send input events (such as drag and drop events) reported by the touch driver to the upper layer for processing through the hardware abstraction layer.

[0094] It is understandable that the above Figure 5 and Figure 6The external drag function shown relies on the system service of the electronic device to assist in dragging the first view in the first application to the second application, and the second application determines whether the touch view enters the preset receiving area of ​​the second application by obtaining the position information of the touch view in real time.

[0095] It can be understood that the current electronic devices implement the internal drag function using the above Figure 3 and Figure 4 The corresponding method described above is used to realize the external drag function Figure 5 and Figure 6 In the corresponding described methods, the two drag functions have different implementation methods. There may be a problem that the dragging corresponding to the user's dragging intention cannot be implemented because the drag function pre-set by the electronic device for each application is different from the user's dragging intention, affecting the user's experience.

[0096] In view of this, an embodiment of the present application provides an interaction method, which implements dragging within an application interface and between applications through a drag function.

[0097] Specifically, after detecting that the user long presses the drag object (first view) of the first application, the position information of the drag object is obtained in real time, and based on the position relationship between the position information of the drag object and the first application window, it is determined whether to implement dragging within the application interface of the first application or between the application and the second application.

[0098] If it is determined that the drag is within the application interface of the first application, the position information of the dragged object is sent to other display elements (second view) in the first application. Based on the position overlap relationship between the other display elements in the first application and the dragged object, it is determined whether the current drag event is an exchange event, a merge event or an invalid event, and the dragged object is controlled to execute the corresponding event.

[0099] If it is determined to be a drag between a second application, the position information of the dragged object is sent to the display element in the second application (the view system of the second application), and based on the position information of the dragged object, it is determined whether the dragged object enters the preset receiving area of ​​the second application; if it enters the preset receiving area of ​​the second application, the content of the dragged object (such as type, carried data, etc.) is sent to the second application to determine whether the second application receives the dragged object.

[0100] It can be understood that the interaction method provided in the embodiment of the present application realizes dragging within the application interface and between applications through a drag function, so that the application using the interaction method provided in the embodiment of the present application can realize the drag function within the application as well as the drag function between applications. When the user triggers a drag event, the user's drag intention (drag type) is first determined, and then the drag function corresponding to the determined drag type is executed, thereby improving the user experience.

[0101] For example, Figures 7a to 7h According to an embodiment of the present application, a schematic diagram of a user interface change process of dragging between applications and dragging within an application is shown.

[0102] Specifically, Figure 7a The user interface 70a of the mobile phone 10 is shown, and the interface 70a includes a user interface 701 of a browser application and a user interface 702 of a favorites application. The user interface 701 includes pictures 7011 and text content, etc. The user interface 702 includes "Content A", "Content B" and "Content C" and the like.

[0103] like Figure 7a As shown, in response to the user's long press operation on the image 7011, a drag image 7012 corresponding to the image 7011 is generated, and the image 7011 is hidden, and the display is as shown in FIG. Figure 7b In the user interface 70b shown in FIG. 1 , as the user drags (long presses and moves) the image 7012 to a preset drag area of ​​the user interface 702 of the favorite application (e.g., an area overlapping with the window of the user interface 702 of the favorite application), the image 7011 is displayed in the user interface 702 of the favorite application, and the image 7011 is dragged between the browser application and the favorite application. The mobile phone 10 displays the following Figure 7c User interface 70c is shown.

[0104] like Figure 7c As shown, the interface 70c includes a user interface 701 of a browser application and a user interface 702 of a collection application. The user interface 701 includes a picture 7011 and text content, etc. The user interface 702 includes "Content A", "Content B", "Content C" and a picture 7021, etc. It can be understood that Figure 7a Compared with the user interface 70a of the mobile phone 10, the picture 7021 is a picture dragged from the browser application to the favorites application.

[0105] like Figure 7c As shown, in response to the user's long press operation on the image 7021, a drag image 7022 corresponding to the image 7021 is generated, and the image 7021 is hidden, and the display is as shown in FIG. Figure 7dIn the user interface 70d shown, as the user drags (long press and move) the picture 7022 to the preset exchange area of ​​the icon corresponding to "Content A" (for example, the area overlapping with the icon corresponding to "Content A", and the overlapping area is less than or equal to half of the area of ​​the icon corresponding to "Content A"), the display positions of the icon corresponding to "Content A" and the picture 7021 are exchanged, and the order of the icon 702 and the picture 7021 corresponding to "Content A" is realized, and the mobile phone 10 displays as follows: Figure 7e User interface 70e is shown.

[0106] like Figure 7e As shown, the interface 70e includes a user interface 701 of a browser application and a user interface 702 of a collection application. The user interface 701 includes a picture 7011 and text content, etc. The user interface 702 includes a picture 7021, "Content A", "Content B" and "Content C", etc. It can be understood that Figure 7c It shows that compared with the user interface 70c of the mobile phone 10, the display positions of the picture 7021 and the icon corresponding to "Content A" are swapped.

[0107] Figure 7f The user interface 70f of the mobile phone 10 is shown, and the interface 70f includes a user interface 702 of the collection application and a user interface 703 of the memo application. The user interface 702 includes a picture 7021, "Content A", "Content B" and "Content C", etc. The user interface 703 includes the content of "Memo 1".

[0108] like Figure 7f As shown, in response to the user's long press operation on the image 7021, a drag image 7022 corresponding to the image 7021 is generated, and the image 7021 is hidden, and the display is as shown in FIG. Figure 7g In the user interface 70g shown in FIG. 1 , as the user drags (long presses and moves) the image 7022 to a preset drag area (e.g., an area overlapping with the window of the user interface 702 of the memo application) of the user interface 703 of the memo application, the image 7021 is displayed in the user interface 703 of the memo application, and the image 7021 is dragged between the favorite application and the memo application. The mobile phone 10 displays the following: Figure 7h User interface 70h is shown.

[0109] like Figure 7h As shown, the interface 70h includes a user interface 702 of the collection application and a user interface 703 of the memo application. The user interface 702 includes the content of "Memo 1" and a picture 7031. It can be understood that Figure 7f Compared with the user interface 70 f of the mobile phone 10 , the picture 7031 is a picture dragged from the Favorites application to the Memo application.

[0110] It can be understood that the user interface involved in the embodiment of the present application is a graphical user interface (GUI) displayed on the display screen of an electronic device, and may include, for example, a main interface (also called a desktop), a negative-one-screen display interface, and display interfaces of various application programs (application, app).

[0111] It can be understood that the application involved in the embodiments of the present application is a computer program that can realize one or more specific functions. Generally, multiple applications can be installed in an electronic device. For example, a memo application, a camera application, a text message application, a browser application, a mailbox application, etc.

[0112] It is understood that the interaction method provided in the embodiment of the present application can be applied to any electronic device including a display screen, such as a mobile phone, a tablet computer, a laptop computer, a car computer, a personal digital assistant (PDA), etc. The exemplary embodiments of the electronic device include but are not limited to a device equipped with Or mobile terminal devices of other operating systems. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0113] In order to better understand the technical solutions of the embodiments of the present application, some technical solutions of the present application are introduced in detail below.

[0114] Figure 8 According to an embodiment of the present application, a flow chart of an interaction method is shown. Figure 8 As shown, the process includes but is not limited to the following steps:

[0115] S801: The first application detects that the user has long pressed the drag object, and sends a drag start event to the system service of the electronic device.

[0116] In some embodiments, when the first application detects a user's long press operation on a drag object in the first application, the first application sends a drag start event corresponding to a first view to a system service of the electronic device, where the first view is a view corresponding to the drag object.

[0117] It can be understood that the long press operation can be, for example, the user touching (pressing without lifting the hand) the position where the drag object is displayed on the electronic device display screen, and the touch time exceeds the touch threshold, such as 3 seconds.

[0118] It can be understood that the first application can be an application that comes with the electronic device when it leaves the factory, or it can be an application that the user installs during the use of the electronic device, and this application does not impose any restrictions on this.

[0119] S802: The system service of the electronic device generates a touch point view corresponding to the dragged object, and obtains position information of the touch point view.

[0120] In some embodiments, the system service of the electronic device receives the drag start event sent by the first application, generates a touch point view corresponding to the drag object, that is, a view corresponding to the touch point of the user's finger, and continuously obtains the drag move event as the user's finger moves.

[0121] It can be understood that the drag movement event includes the position information of the touch point view. Fig. 9 Schematic diagram of contact position information shown in FIG. Fig. 9 As shown, the upper left corner of the electronic device display screen 901 is taken as the coordinate origin, the x-axis is rightward and the y-axis is downward, and the position information (position coordinates) of the touch point M is (x1, y1).

[0122] In some embodiments, after the system service of the electronic device generates the contact view corresponding to the dragged object, the first view corresponding to the dragged object is hidden.

[0123] S803: The system service of the electronic device determines the drag type.

[0124] In some embodiments, if the determination result is internal drag, step S804 is executed, and the system service of the electronic device distributes the drag event to the second view of the first application.

[0125] In other embodiments, if the determination result is external drag, step S809 is executed, and the system service of the electronic device distributes the drag event to the second application.

[0126] S804: The system service of the electronic device distributes the drag event to the second view of the first application.

[0127] In some embodiments, after the system service of the electronic device determines that the drag type is an internal drag, the received drag event (drag move event) is sent to the second view of the first application.

[0128] It can be understood that the second view is a view corresponding to other display elements (such as icons, pictures, texts, etc.) in the first application except the dragged object.

[0129] S805: The second view determines the event type.

[0130] In some embodiments, if the determination result is a swap event, step S806 is executed, and the second view determines to swap display positions with the touch point view.

[0131] In some other embodiments, if the determination result is a merge event, step S807 is executed, and the second view determines the display position merged with the touch point view.

[0132] For example, the event type can be determined by the following logic:

[0133] IF falls in the hot zone of the exchange position:

[0134] IF is an internal drag scene:

[0135] IF there are no pending swap operations:

[0136] It is considered that a swap operation is required and will be executed after 300ms.

[0137] ELSE falls in the merge hotspot:

[0138] IF internal drag scene:

[0139] IF there is no pending merge operation:

[0140] It is considered necessary to perform a merge operation, which will be executed after 300ms.

[0141] It can be understood that the exchange position hot zone represents the position area that meets the exchange conditions, the merge position hot zone represents the position area that meets the merge conditions, 300ms represents the preset execution time, and in other embodiments, it can also be 200ms. This application does not limit the specific value of the execution time.

[0142] It can be understood that the exchange condition can be that the display position of the touch view and the display position of the second view have an overlapping area, and the area of ​​the overlapping area is greater than or equal to the exchange threshold (for example, 50% of the area of ​​the second view) and less than the merging threshold (for example, 80% of the area of ​​the second view). The merging condition can be that the display position of the touch view and the display position of the second view have an overlapping area, and the area of ​​the overlapping area is greater than or equal to the merging threshold.

[0143] S806: The second view determines to exchange display positions with the touch point view.

[0144] In some embodiments, after determining that the event type is a swap event, the second view of the first application exchanges display position with the touch view.

[0145] For example, reference Fig.10 Schematic diagram of the exchange event shown. Fig.10 As shown in (a) in FIG. 1 , when the position information of the icon corresponding to “Content 3” and the icon corresponding to “Content 2” meet the exchange condition, for example, there is an overlapping area between the icon corresponding to “Content 3” and the icon corresponding to “Content 2”, and the area of ​​the overlapping area is less than or equal to half of the area of ​​the icon corresponding to “Content 2”, the following is displayed: Fig.10In the interface shown in (b), the display positions of the icon corresponding to "Content 3" and the icon corresponding to "Content 2" are swapped.

[0146] S807: The second view is determined to be combined with the touch point view and displayed at a position.

[0147] In some embodiments, after the second view of the first application determines that the event type is a merge event, the second view of the first application merges the display position with the contact view.

[0148] For example, reference Fig.11 Schematic diagram of the merging event shown. Fig.11 As shown in (a) of FIG. 1 , when the position information of the icon corresponding to “Content 3” and the icon corresponding to “Content 2” meet the merging condition, for example, there is an overlapping area between the icon corresponding to “Content 3” and the icon corresponding to “Content 2”, and the area of ​​the overlapping area is less than or equal to 80% of the area of ​​the icon corresponding to “Content 2”, the following is displayed: Fig.11 In the interface shown in (b), the display positions of the icon corresponding to "Content 3" and the icon corresponding to "Content 2" are merged into "File 1".

[0149] S808: The second view updates the initial display position of the contact view to the system service.

[0150] In some embodiments, after the second view and the touch view trigger the exchange or merge of display positions, the initial display position of the touch view is updated to the system service so that when the user cancels the drag (cancels the long press operation), the initial position of the touch view corresponds to the exchanged or merged position, that is, the first view is displayed at the exchanged or merged position.

[0151] S809: The system service of the electronic device distributes the drag event to the second application.

[0152] In some embodiments, after the system service of the electronic device determines that the drag type is an external drag, the received drag event (drag move event) is sent to the second application.

[0153] S810: The drag listener of the second application detects that the touch view enters a preset receiving area of ​​the second application.

[0154] In some embodiments, the drag listener of the second application determines whether the touch view enters the preset receiving area of ​​the second application based on the received position information, so that further processing is performed when it is detected that the touch view enters the preset receiving area of ​​the second application.

[0155] For example, reference Fig.12 The schematic diagram of external drag is shown in Figure 1. Fig.12As shown, when the icon corresponding to "Content 3" in the first application enters the preset acceptance area (blank area) of the second application, it is further determined whether to accept "Content 3".

[0156] S811: Acquire the content of the dragged object, analyze and process it, and determine whether to receive the dragged object.

[0157] In some embodiments, when it is detected that the touch view enters the preset receiving area of ​​the second application, the second application obtains the content of the drag object corresponding to the touch view, analyzes and processes it, and determines whether to receive the drag object.

[0158] S812: The system service of the electronic device determines whether the dragging is completed.

[0159] In some embodiments, if the judgment result is yes, indicating that the current drag has ended, step S813 is executed, and the system service of the electronic device generates a drag animation.

[0160] In other embodiments, if the determination result is no, indicating that the current drag has not ended, step S803 is executed, and the system service of the electronic device determines the type of drag.

[0161] S813: The system service of the electronic device generates a dragging animation.

[0162] In some embodiments, if the drag type is an internal drag, the system service of the electronic device determines that the current drag has ended, for example, detects that the user cancels the long press operation, and generates and displays a drag animation according to the position information of the contact view.

[0163] It can be understood that if a merger or exchange occurs, in the generated dragging animation, the initial position of the dragged object is the position after the exchange or merger.

[0164] In some embodiments, if after the swap occurs, the contact view stays at the swapped position (the position of the second view) for a first period of time and then detects that the user cancels the drag operation, the dragging animation is to display the first view at the swapped position and hide the contact view.

[0165] In some embodiments, if after the swap occurs, the contact view moves to an invalid position (a position that does not overlap with the second view and does not belong to the preset receiving area of ​​the first application), the user's cancellation of the drag operation is detected, the dragging animation is a return animation, and the starting position of the return animation is the current invalid position, and the ending position is the position after the swap.

[0166] In some embodiments, if the drag type is an external drag, the system service of the electronic device determines that the current drag has ended, for example, it detects that the user cancels the long press operation, and redisplays the first view of the dragged object hidden in the first application. If the initial position has been updated, the display position of the first view is the updated position.

[0167] It can be understood that in other embodiments, the event type may also include a move event. When it is detected that the contact view is located in a preset receiving area of ​​the first application (such as a blank area, etc.), and the system service of the electronic device determines that the current drag has ended, for example, it detects that the user cancels the long press operation, then the first view of the dragged object is displayed in the preset receiving area.

[0168] It can be understood that in other embodiments, according to actual needs, the above Figure 8 The steps shown can be combined, deleted or replaced with other steps that are conducive to achieving the purpose of this application. For example, the above-mentioned step S810 and step S811 can be combined into one step, and this application does not limit this.

[0169] Fig.13 According to an embodiment of the present application, a schematic diagram of the interaction process of a method of dragging within an application is shown. Fig.13 As shown, the process includes:

[0170] S1301: User 11 long presses a first view of a first application 12 .

[0171] In some embodiments, the user 11 long presses the view corresponding to the dragged object in the first application 12 (for example, Figure 7c As shown in the picture 7021), a drag is initiated for the drag object, wherein the view corresponding to the drag object is the first view.

[0172] It can be understood that the first application can be an application that comes with the electronic device when it leaves the factory, or it can be an application that the user installs during the use of the electronic device, and this application does not impose any restrictions on this.

[0173] S1302 : The first application 12 initiates a drag event to the view framework 13 .

[0174] In some embodiments, when the first application 12 detects a long press operation of the user 11 on the drag object in the first application 12 , the first application 12 sends a drag start event corresponding to the first view to the view framework 13 in the application framework layer.

[0175] S1303: The view framework 13 sends an instruction to create a touch view to the display framework 16.

[0176] In some embodiments, the view framework 13 generates a touch point view corresponding to the dragged object, that is, a new view corresponding to the touch point of the user's finger (eg, Figure 7d In addition, the view framework 13 sends an instruction to create a touch view to the display framework 16, so that the display framework 16 sends an instruction to display the touch view to the display driver 22 to display the touch view corresponding to the dragged object.

[0177] It can be understood that, in some embodiments, the instruction for creating the touch view also includes an instruction for hiding the first view corresponding to the dragged object.

[0178] S1304: The display framework 16 sends an instruction to display the contact view to the display driver 22.

[0179] In some embodiments, the display framework 16 sends an instruction to display the contact view to the display driver 22, so as to display the contact view corresponding to the dragged object.

[0180] S1305: The view framework 13 initiates a task of processing the drag to the drag controller 14 .

[0181] In some embodiments, the view framework 13 calls the session control class to send a task (performDrag) for processing the drag to the drag controller 14 .

[0182] S1306: The drag controller 14 sends an instruction to create a drag state to the drag state 15.

[0183] In some embodiments, the drag controller 14 sends an instruction to create a drag state to the drag state 15 and creates the drag state.

[0184] S1307: The drag state 15 sends a drag start event to the root view 20.

[0185] S1308 : The touch driver 19 distributes the touch event to the input manager 18 .

[0186] It can be understood that the touch event (such as a drag and drop event) includes the position information of the touch point view. Fig. 9 Schematic diagram of contact position information shown in FIG. Fig. 9 As shown, the upper left corner of the electronic device display screen 901 is taken as the coordinate origin, the x-axis is rightward and the y-axis is downward, and the position information (position coordinates) of the touch point M is (x1, y1).

[0187] S1309 : The touch driver 19 distributes the touch event to the drag input event receiver 17 .

[0188] S1310 : The drag input event receiver 17 sends a touch event to the drag controller 14 .

[0189] In some embodiments, after receiving a touch event (such as a drag move event), the drag input event receiver 17 sends a drag start event and the received drag move event to the drag controller 14. It can be understood that the drag controller obtains the drag move event through the drag input event receiver 17 interface.

[0190] S1311 : The drag controller 14 updates the drag status to the drag state 15 .

[0191] In some embodiments, after receiving the drag movement event, the drag controller 14 sends the received drag movement event to the drag state 15 .

[0192] S1312: The drag state 15 updates the contact view to the display framework 16.

[0193] In some embodiments, the drag state 15 sends the touch view and its position information to the display framework 16 according to the touch coordinates in the received drag movement event, generates a new position of the touch view, and updates the touch view.

[0194] S1313 : The touch driver 19 updates the input window to the input manager 18 .

[0195] In some embodiments, as the user's finger moves, the touch view may enter the second application (an application other than the first application in the electronic device 100). The touch driver 19 updates the input window to the input manager 18, so that the input manager 18 updates the input window when the touch view enters the window of the second application.

[0196] S1314: The input manager 18 sends a drag start instruction to the drag controller 14.

[0197] In some embodiments, the input manager 18 sends a drag start instruction to the drag controller 14. If the first application accepts and processes the drag start event, the input manager 18 sends a drag event (drag move event) to the drag controller 14 when the subsequent contact view moves with the user's finger in the first application.

[0198] S1315: The drag controller 14 sends a drag event to the drag state 15.

[0199] S1316: The drag state 15 distributes the drag event to the root view 20.

[0200] S1317 : The root view 20 distributes the drag event to the view framework 13 .

[0201] S1318 : The view framework 13 sends a drag event to the second view 21 .

[0202] In some embodiments, after the view framework 13 receives the drag event from the input root view 20 , the view framework 13 distributes the drag event to the second view 21 according to the position information of the touch view.

[0203] It can be understood that the view framework 13 can send the drag event to the drag listener corresponding to the second view 21 (such as other views of the first application) so that the drag listener of the application layer can determine whether the touch view and the second view 21 are merged or exchanged based on the position relationship between the second view 21 and the touch view.

[0204] In some embodiments, if the input window updated in the above step S1313 is an application window corresponding to the second application, the drag event is sent to the second application (eg, a view system consisting of a root view in the second application and views corresponding to each display element).

[0205] S1319: The second view 21 determines that the drag type is internal drag.

[0206] In some embodiments, the drag listener (onDragListener) corresponding to the second view 21 determines that the drag type of the current drag event is an internal drag based on the received drag movement event, that is, based on the position information of the contact view.

[0207] In other embodiments, the drag listener corresponding to the second view 21 obtains the content of the drag object based on receiving the drag move event, and if the content of the drag object is data in the first application, determines that the drag type of the current drag event is internal drag.

[0208] S1320: The second view 21 determines to merge or exchange positions.

[0209] In some embodiments, the drag listener corresponding to the second view 21 determines the positional relationship (e.g., overlapping area) between the touch view and the second view 21 based on the received drag movement event, i.e., based on the position information of the touch view, and determines to merge or exchange the position of the first view corresponding to the touch view.

[0210] S1321 : The second view 21 updates the starting point of the contact view to the root view 20 .

[0211] In some embodiments, after the second view 21 is merged or swapped with the first view corresponding to the touch view, the new position (the position of the merged or swapped first view and second view 21 ) is sent to the root view 20 .

[0212] S1322 : The root view 20 sends the updated starting point of the touch view to the drag controller 14 .

[0213] S1323: The drag controller 14 updates the start point of the touch view to the drag state 15.

[0214] S1324: User 11 cancels the long press.

[0215] S1325: The touch driver 19 sends an instruction to end the dragging to the input manager 18.

[0216] In some embodiments, after the user 11 cancels the long press, the touch driver 19 sends an instruction to end the drag to the input manager 18 .

[0217] S1326: The input manager 18 sends an instruction to end the dragging to the drag controller 14.

[0218] In some embodiments, input manager 18 sends an instruction to end the drag to drag controller 14, so that drag controller 14 notifies root view 20 of the end of the drag through drag state 15, such as sending an end drag event.

[0219] S1327: The drag controller 14 sends an end-drag event to the drag state 15.

[0220] S1328: Drag state 15 sends an end drag event to the root view 20.

[0221] S1329: The root view 20 sends an instruction to restore the state to the first application 12.

[0222] In some embodiments, the instruction to restore the state may be, for example, an instruction to generate an animation or to display a hidden first view, hide a contact view, or the like.

[0223] S1330: The first application 12 sends an instruction to end the dragging to the view framework 13 .

[0224] S1331: The view framework 13 sends an instruction to end the dragging to the drag controller 14.

[0225] S1332: The drag controller 14 sends an instruction to end the drag to the drag state 15.

[0226] S1333: The drag state 15 sends an instruction to execute the animation to the display frame 16.

[0227] In some embodiments, the dragging state 15 continuously modifies the position information of the contact view within a period of time (i.e., the animation time) until the starting point of the contact view is changed to the updated starting point, and sends an instruction to execute the animation to the display framework 16 to update the animation to the display driver 22.

[0228] S1334: The display framework 16 sends an instruction to update the display to the display driver 22.

[0229] Combine the following Fig.14 The software architecture diagram shown illustrates the implementation method of dragging within the application in an embodiment of the present application.

[0230] like Fig.14 As shown, the software architecture of the electronic device 100 mainly includes four layers, including from bottom to top: a kernel layer, a hardware abstraction layer (HAL), an application framework layer, and an application layer.

[0231] The application layer includes various applications, and each application in the application layer can determine how the display interface is displayed. Fig.14 As shown, the application layer includes a first application, wherein the first application includes a first view (a view corresponding to the dragged object) and a second view (a view corresponding to other display elements in the first application except the dragged object).

[0232] The application framework layer includes the view framework (View), display framework (Surface) and system service (SystemServer).

[0233] In some embodiments, when the user's finger long presses the drag object in the first application, the first application sends a drag start event (startDragAndDrop) to the view framework, the view framework receives the drag start event, generates a contact view, and sends the contact view to the display framework.

[0234] System services include the WindowManager and the InputManager. The WindowManager includes the Session class, the DragDropController, the DragState, and the DragInputEventReceiver.

[0235] The session control class is a class in the window manager that is used to communicate between system services and applications.

[0236] In some embodiments, the view framework receives a drag start event sent by the first application, sends an instruction to create a touch view to the display framework, and sends a task to handle the drag to the drag controller by calling a session control class.

[0237] In some embodiments, after the drag controller receives the task of processing the drag, it sends instructions to create and update the drag state based on the touch events (drag move events) sent by the receiving input manager and the drag input receiver, so that the drag state is sent to the display framework, the position of the contact view is modified, and the drag event (drag move event) is distributed to the first application through the session control class.

[0238] In some embodiments, the drag state is sent to the display framework, the position of the touch view is modified, and the drag event (drag move event) is distributed to the first application through the session control class.

[0239] The kernel layer mainly includes drivers for input / output devices. Output devices may include display screens, speakers, etc. Input devices may include sensors (such as touch sensors), keyboards, microphones, etc. The input devices in the kernel layer can send input events (such as drag and drop events) reported by the touch sensor driver to the upper layer (i.e., application framework layer) for processing through the hardware abstraction layer (HAL). It can be understood that Fig.14 Only the touch sensor driver and the display driver are exemplarily shown in the figure, but the drivers included in the core layer are not limited to the touch sensor driver and the display driver.

[0240] The hardware abstraction layer is used to provide the application framework layer with a common interface for calling drivers in the kernel layer, and distribute input events sent by the kernel layer to the upper layer, namely the application framework layer.

[0241] It is to be understood that the software framework illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0242] Understandably, combined Fig.14 According to the software framework shown, the process of generating the return animation of dragging by the electronic device can be: the view framework of the application layer sends the updated starting position (the position of the second view) to the drag controller of the application framework layer; the drag controller sends the position information of the contact view to the drag state of the application framework layer, and controls the drag state to set the end position of the return animation of the dragged object to the updated starting position, and sends it to the display framework of the application framework layer; the display framework displays the return animation of the first display element moving from the invalid position to the updated starting position based on the updated starting position and the invalid position.

[0243] In order to better understand the technical solutions of the embodiments of the present application, Fig.15According to an embodiment of the present application, a schematic diagram of the interaction process of various modules in the process of updating the initial display position of the contact view is shown. Fig.15 As shown, the process includes:

[0244] S1: Views in the first application are merged or swapped.

[0245] In some embodiments, the first application detects that the contact view and the second view in the first application are merged or exchanged.

[0246] S2: The first application obtains the position of the view after the swap.

[0247] In some embodiments, after detecting that the views are merged or exchanged, the first application obtains position information of the contact view and the second view after the exchange or merger.

[0248] S3: The first application obtains view coordinates.

[0249] In some embodiments, the first application obtains the coordinates of the contact views after the exchange or merger.

[0250] S4: The system service's drag controller updates the recorded starting coordinates.

[0251] In some embodiments, the drag controller of the system service updates the recorded starting coordinates according to the coordinates of the contact views after the exchange or merger, that is, updates the starting position of the first view to the coordinates of the contact views after the exchange or merger.

[0252] S5: The first application detects that the dragging is stopped.

[0253] In some embodiments, the first application detects that the dragging is stopped, for example, the user cancels the long press operation.

[0254] S6: The drag controller of the system service starts to execute the animation.

[0255] In some embodiments, after the first application detects that the dragging is stopped, the drag controller of the system service generates an animation according to the position information of the contact view and the starting position of the first view, and starts to execute the animation.

[0256] S7: The view framework continuously updates the position of the touch view until the new starting point.

[0257] In some embodiments, the view framework continuously updates the position of the contact view and displays the contact view until the position of the contact view is at the starting position of the first view.

[0258] In other embodiments, the view frame displays the first view, and moves the first view from the position of the touch view when the dragging stops to the starting position of the first view.

[0259] It can be understood that the first view and the contact view are not displayed at the same time. If the contact view is displayed, the first view is controlled not to be displayed; if the first view is displayed, the contact view is controlled not to be displayed.

[0260] It can be understood that the software architecture of the electronic device 100 corresponding to the implementation method of the external drag of the application in the embodiment of the present application can refer to Figure 6 and Figure 6 This application will not go into details here.

[0261] To sum up, the interaction method provided in the embodiment of the present application realizes dragging within the application interface and between applications through a drag function, so that the application using the interaction method provided in the embodiment of the present application can realize the drag function within the application as well as the drag function between applications. When the user triggers a drag event, the user's drag intention (drag type) is first determined, and then the drag function corresponding to the determined drag type is executed, thereby improving the user experience.

[0262] The embodiment of the present application also provides a display method, including: displaying a user interface of a first application and a user interface of a second application, wherein the user interface of the first application includes a drag object and other display elements. It can be understood that the view corresponding to the drag object is the first view, and the view corresponding to the other display elements is the second view.

[0263] When a drag operation (eg, long press to move) of the object by the user is detected and the position information of the dragged object and the position information of the second view meet the exchange condition, the display positions of the dragged object and the second view are exchanged.

[0264] When it is detected that the user continues to drag the dragged object to an invalid position and ends the drag, the return animation of the dragged object is displayed.

[0265] In some embodiments, the exchange condition may be that there is an overlapping area between the display position of the contact view and the display position of the second view, and the area of ​​the overlapping area is greater than or equal to the exchange threshold (for example, 50% of the area of ​​the second view) and less than the merge threshold (for example, 80% of the area of ​​the second view).

[0266] It can be understood that the invalid position may be a position that does not overlap with the second view and does not belong to the preset receiving area of ​​the first application.

[0267] Combined with the above Fig.14The software framework of the electronic device 100 shown, the display method provided by the embodiment of the present application may specifically include: the touch driver detects the user's drag operation on the drag object, and sends the position information of the current drag object to the drag controller. The drag controller sends the position information of the current drag object to the drag state, and controls the drag state to send the position information to the second view of the first application. After the second view of the first application determines that the position information of the current drag object meets the exchange condition based on the position information of the current drag object, it sends the position information of the current drag object to the drag controller, and controls the display framework to display the second view at the position of the first view. The drag controller sends the position of the second view, that is, the initial position of the updated first view, to the drag state, and controls the drag state to set the end position of the return animation of the drag object to the position of the second view, and sends it to the display module of the framework layer. After the touch driver detects that the user drags the drag object to an invalid position and cancels the drag, the invalid position and the end drag information are sent to the drag controller. The drag controller sends the position information of the invalid position to the drag state, and controls the drag state to send the position information of the invalid position to the display module; the display module displays a return animation of the drag object moving from the invalid position to the initial position of the updated first view based on the initial position and the invalid position of the updated first view.

[0268] In order to better understand the technical solutions of the embodiments of the present application, the structure of the equipment involved in the present application is introduced below with reference to the accompanying drawings.

[0269] For example, Fig.16 According to an embodiment of the present application, a structural schematic diagram of an electronic device 100 is shown.

[0270] like Fig.16 As shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, an interface module 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0271] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0272] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0273] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0274] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0275] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0276] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor.

[0277] The display screen 194 is used to display images, videos, etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0278] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement a data storage function.

[0279] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, at least one application required for a function, etc. The data storage area may store data created during the use of the electronic device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor.

[0280] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0281] Program code may be applied to input instructions to perform the functions described herein and to generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0282] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0283] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, instructions may be distributed over a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including, but not limited to, floppy disks, optical disks, optical disks, read-only memories (CD-ROMs), magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), magnetic or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in electrical, optical, acoustic, or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0284] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0285] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0286] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0287] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. An interactive method, applied to an electronic device, characterized in that: include: Displaying a first application interface of a first application, wherein the first application interface includes a first display element located at a first position; Detecting a user dragging operation on the first display element, and acquiring position information of the first display element during the dragging process; Determining that the first display element is dragged to a second position of the first application interface; Based on the second position, a display mode of the first display element is determined.

2. The method according to claim 1, characterized in that The determining, based on the second position, a display mode of the first display element includes: In response to the user's dragging operation being ended at the second position and the second position being in the first receiving area of ​​the first application, the first display element is displayed at the second position.

3. The method according to claim 1, characterized in that The first application interface also includes a second display element located at a third position, and The determining, based on the second position, a display mode of the first display element includes: The third position corresponding to the second position and the second display element satisfies an exchange condition, and the second display element is displayed at the first position, wherein the exchange condition includes: there is an overlapping area between the third position and the second position, and the area of ​​the overlapping area is greater than or equal to a first threshold and less than a second threshold.

4. The method according to claim 3, characterized in that The determining, based on the second position, a display mode of the first display element further includes: It is detected that a first time period after the first display element is displayed at the second position, the dragging operation of the user ends, and the first display element is displayed at the second position.

5. The method according to claim 3, characterized in that: Also includes: detecting that the dragging operation of the user has not ended in a first time period after the first display element is displayed at the second position, and setting the end position of the return animation of the first display element to the second position; After detecting that the first display element moves to a fourth position, the user's dragging operation ends and a return animation of the first display element is displayed, wherein the fourth position is located in an invalid area.

6. The method according to claim 5, characterized in that The end position of the return animation of the first display element is set to the second position in the following manner: The view module of the application layer of the electronic device sends the position information of the second position to the drag controller of the framework layer of the electronic device; The drag controller sends the position information of the second position to the drag state of the framework layer, controls the drag state to set the end position of the return animation of the first display element to the second position, and sends it to the display module of the framework layer; and After detecting that the first display element moves to the fourth position, the dragging operation of the user ends, and displaying the return animation of the first display element includes: The display module displays a return animation of the first display element moving from the fourth position to the second position based on the position information of the second position and the position information of the fourth position.

7. The method according to claim 1, characterized in that The first application interface includes a second display element, and The determining, based on the second position, a display mode of the first display element further includes: Corresponding to the end of the user's dragging operation at the second position, and the second position and the third position corresponding to the second display element satisfying a merging condition, the first display element and the second display element are displayed at the third position, wherein the merging condition includes that the third position overlaps with the second position, and the area of ​​the overlapping area is greater than or equal to a second threshold.

8. The method according to claim 1, characterized in that The detecting a user's dragging operation on the first display element and acquiring position information of the first display element during the dragging process includes: The first application of the application layer of the electronic device detects a user dragging operation on the first display element, generates a drag start event, and sends the drag start event to the view module of the framework layer of the electronic device; The view module of the framework layer generates a touch view corresponding to the first display element based on the drag start event, and controls the display module of the framework layer to display the touch view and set the first display element to be invisible; The drag controller of the framework layer obtains a drag movement event during the dragging process, wherein the drag movement event includes position information of the contact view; The drag controller of the framework layer sends the position information to the drag state of the framework layer, and controls the drag state to send the position information to the display module of the framework layer; and When it is detected that the dragging operation ends, the display module of the framework layer controls the touch point view to be invisible.

9. A display method, applied to an electronic device, characterized in that: include: Displaying a first application interface of a first application, wherein the first application interface includes a first display element located at a first position and a second display element located at a second position; It is detected that the user drags the first display element to a third position, and the second display element is displayed at the first position, wherein the third position satisfies the exchange condition; It is detected that the user continues to drag the first display element to the fourth position and ends the dragging operation, and a return animation of the first display element is displayed, wherein the end position of the first display element in the return animation is the second position.

10. The method according to claim 9, characterized in that The exchange conditions include: There is an overlapping area between the third position and the second position, and the area of ​​the overlapping area is greater than or equal to a first threshold and less than a second threshold.

11. The method according to claim 9, characterized in that The detecting that the user drags the first display element to a third position and displaying the second display element at the first position includes: The touch driver of the electronic device detects that the user drags the first display element to a third position, and sends position information of the third position to the drag controller of the frame layer of the electronic device; The drag controller sends the position information of the third position to the drag state of the framework layer of the electronic device, and controls the drag state to send the position information to the view module of the first application of the application layer of the electronic device; After determining that the third position meets the exchange condition based on the position information of the third position, the view module of the first application sends the position information of the second position to the drag controller, and controls the display module of the frame layer of the electronic device to display the second display element at the first position; The drag controller sends the position information of the second position to the drag state of the framework layer, and controls the drag state to set the end position of the return animation of the first display element to the second position, and sends it to the display module of the framework layer.

12. The method according to claim 11, characterized in that The detecting that the user continues to drag the first display element to the fourth position and ends the dragging operation, and displaying the return animation of the first display element includes: The touch driver detects that the user drags the first display element to a fourth position and ends the drag operation, and sends position information of the fourth position and the drag end information to the drag controller; The drag controller sends the position information of the fourth position to the drag state, and controls the drag state to send the position information of the fourth position to the display module; The display module displays a return animation of the first display element moving from the fourth position to the second position based on the position information of the second position and the position information of the fourth position.

13. An electronic device, characterized in that: include: A display module, configured to display a first application interface of a first application, wherein the first application interface includes a first display element located at a first position; a touch driver, configured to detect a user's dragging operation on the first display element, obtain position information of the first display element during the dragging process, and send the position information to a drag controller and a first application; The drag controller is used to determine the display mode of the first display element based on the first display element being dragged by the user to the second position of the first application interface.

14. The electronic device according to claim 13, characterized in that: Also includes: A first application is used to generate a drag start event and send the drag start event to a view module of a framework layer of the electronic device; A view module, configured to generate a touch view corresponding to the first display element based on the drag start event, and control the display module of the framework layer to display the touch view and set the first display element to be invisible; The drag state is used to receive the position information sent by the drag controller and send the position information to the display module of the framework layer; and The drag controller is further used to obtain a drag movement event during the dragging process, wherein the drag movement event includes the position information of the contact view, and sends the position information to the drag state of the framework layer, and controls the drag state to send the position information to the display module of the framework layer; The display module is further used to display a return animation of the first display element based on the position information.

15. A readable medium, characterized in that The readable medium stores instructions, which, when executed on an electronic device, enable the electronic device to execute the method according to any one of claims 1 to 12.

16. An electronic device, characterized in that: include: a memory for storing instructions to be executed by one or more processors of the electronic device, and The processor is one of the processors of the electronic device, and is used to execute the method according to any one of claims 1 to 12.

Images