Multi-screen interaction method and device, terminal equipment and storage medium

Abstract

The application is suitable for the technical field of terminal equipment, and provides a multi-screen interaction method and device, terminal equipment and storage medium. The method generates an auxiliary instruction in response to a touch event triggered on a second display screen. The type of the auxiliary instruction is related to the gesture type of the touch event. The auxiliary instruction is converted based on a preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction. According to the control instruction, a corresponding action is performed on the display content of the first display screen. When using a multi-display screen device, the main screen can be controlled by mapping the display screen other than the main screen, the linkage between multiple display screens is improved, and the utilization rate of the display screen and the work efficiency of the multi-display screen device are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of terminal devices, and particularly relates to a multi-screen interaction method and device, a terminal device, and a storage medium. BACKGROUND

[0002] With the rapid development of electronic technology and screen technology, more and more devices equipped with multiple display screens (for example, a combination of front and back screens, a large main screen and a small auxiliary screen, or a folding screen) are put into the market, bringing users brand-new use experience and visual effects.

[0003] At present, the use of a multi-display screen device is limited by the design concept of multi-screen independent touch, so that the operations between multiple display screens are independent of each other. Taking a folding screen device as an example, when the folding screen device is in a folded state, the folding screen device controls and outputs display content through an auxiliary screen, and a main screen is folded and does not work. When the folding screen device is in an unfolded state, the folding screen device controls and outputs display content through the main screen, and the auxiliary screen is turned off or independently runs, resulting in a lack of linkage between multiple display screens, an increase in cost, a decrease in display screen utilization, and a waste of resources. Therefore, how to improve the display screen utilization of a multi-display screen device has become a problem to be solved at present. SUMMARY

[0004] In view of this, the embodiments of the present application provide a multi-screen interaction method and device, a terminal device, and a storage medium to solve the problem of low display screen utilization of a multi-display screen device.

[0005] A first aspect of the embodiments of the present application provides a multi-screen interaction method applied to a terminal device, the terminal device including at least two display screens, a first display screen serving as a main screen and outputting display content, and a second display screen being any display screen other than the main screen. The method includes the following steps.

[0006] In response to a touch event triggered on the second display screen, an auxiliary instruction is generated, and the type of the auxiliary instruction is related to the gesture type of the touch event.

[0007] Based on a preset mapping rule between the first display screen and the second display screen, the auxiliary instruction is converted to obtain a corresponding control instruction.

[0008] According to the control instruction, a corresponding action is performed on the display content of the first display screen.

[0009] The first aspect of the embodiment of the present application provides a multi-screen interaction method, which comprises the following steps: generating an auxiliary instruction in response to a touch event triggered on a second display screen; the type of the auxiliary instruction is related to the gesture type of the touch event; converting the auxiliary instruction based on a preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction; and performing a corresponding action on the display content of the first display screen according to the control instruction. When a multi-display device is used, the main screen can be controlled by mapping the display screen other than the main screen, the linkage between the multiple display screens is improved, and the display screen utilization and the work efficiency of the multi-display device are improved.

[0010] The second aspect of the embodiment of the present application provides a multi-screen interaction device, which comprises:

[0011] a response module, configured to generate an auxiliary instruction in response to a touch event triggered on a second display screen; the type of the auxiliary instruction is related to the gesture type of the touch event;

[0012] a mapping module, configured to convert the auxiliary instruction based on a preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction;

[0013] an execution module, configured to perform a corresponding action on the display content of the first display screen according to the control instruction.

[0014] The third aspect of the embodiment of the present application provides a terminal device, which comprises at least two display screens, a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the multi-screen interaction method provided by the first aspect of the embodiment of the present application when executing the computer program.

[0015] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the multi-screen interaction method provided by the first aspect of the embodiment of the present application when executed by a processor.

[0016] It can be understood that the beneficial effects of the second aspect to the fourth aspect described above can be referred to the related description in the first aspect described above, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1is a structural schematic diagram of a terminal device provided by an embodiment of the present application;

[0019] Figure 2 is a first flow schematic diagram of a multi-screen interaction method provided by an embodiment of the present application;

[0020] Figure 3 is an interaction flow schematic diagram between a processor, a first display screen and a second display screen of a terminal device provided by an embodiment of the present application;

[0021] Figure 4 is a second flow schematic diagram of a multi-screen interaction method provided by an embodiment of the present application;

[0022] Figure 5 is a third flow schematic diagram of a multi-screen interaction method provided by an embodiment of the present application;

[0023] Figure 6 is a structural schematic diagram of a multi-screen interaction apparatus provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known devices, circuits, and methods are omitted so as not to obscure the description of the application with unnecessary detail.

[0025] It should be understood that the term "comprises / comprising" when used in this specification and the appended claims, specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0026] It should also be understood that the term "and / or" when used in this specification and the appended claims, means any one or more of the associated listed items can be present, and includes multiples of any one or more of the associated listed items.

[0027] As used in this specification and the appended claims, the term "if' can, in some instances, be interpreted as meaning "when," or "once," or "in response to a determination," or "in response to detecting." Similarly, the phrase "if a determination" or "if detecting [the recited condition or event]" can, in some instances, be interpreted to mean "once a determination" or "in response to a determination" or "once detecting [the recited condition or event]" or "in response to detecting [the recited condition or event]."

[0028] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0029] In the present application, the reference to "one embodiment" or "some embodiments" and the like means that the particular feature, structure or characteristic described in connection with this embodiment is included in at least one embodiment of the present application. Thus, the appearance of the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments" and the like in various places in the specification is not necessarily all referring to the same embodiment, but means "one or more but not all embodiments", unless otherwise specifically stated. The terms "include", "contain", "have" and their variants mean "including but not limited to", unless otherwise specifically stated.

[0030] In the application, at present, the use of multi-display device is limited by the design concept of multi-screen interaction, so that the operation between multiple display screens is independent of each other. Taking a folding screen device as an example, when the folding screen device is in a folded state, the folding screen device controls and outputs display content through the secondary screen, and the primary screen is folded and does not work. When the folding screen device is in an unfolded state, the folding screen device controls and outputs display content through the primary screen, and the secondary screen is turned off or the secondary screen runs independently, resulting in a lack of linkage between multiple display screens, an increase in cost and a decrease in resource utilization. Therefore, how to improve the utilization rate of the display screen of the multi-display device has become a problem to be solved at present.

[0031] To solve the above technical problems, the present application provides a multi-screen interaction method, which generates an auxiliary instruction in response to a touch event triggered on a second display screen. The type of the auxiliary instruction is related to the gesture type of the touch event. The auxiliary instruction is converted based on a preset mapping rule between a first display screen and the second display screen to obtain a corresponding control instruction. The display content of the first display screen is executed according to the control instruction. When using a multi-display device, the primary screen can be controlled by mapping the display screen other than the primary screen, the linkage between multiple display screens is improved, the utilization rate of the display screen and the work efficiency of the multi-display device are improved.

[0032] The multi-screen interaction method provided by the embodiments of the present application can be applied to a terminal device equipped with multiple display screens. The terminal device can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The embodiments of the present application do not make any limitation on the specific type of the terminal device.

[0033] In application, the terminal device suitable for the multi-screen interaction method provided by the embodiments of the present application includes at least two display screens. The installation form of the at least two display screens can include a front-back double screen, a combination of a large-area main screen and a small-area auxiliary screen (for example, the large-area main screen is on the top and the small-area auxiliary screen is on the bottom), a folding screen, a split screen, and the like. The embodiments of the present application do not make any limitation on the installation form of the multiple display screens.

[0034] Figure 1 An exemplary structure schematic diagram of a terminal device 100 is shown. The terminal device 100 includes a processor 10, a memory 20, an audio module 30, a camera module 40, a sensor module 50, an input module 60, a display module 70, a wireless communication module 80, and a power module 90, and the like. The audio module 30 can include a speaker 31 and a microphone 32, and the like. The camera module 40 can include a short-focus camera 41, a long-focus camera 42, and a flash 43, and the like. The sensor module 50 can include an infrared sensor 51, an acceleration sensor 52, a position sensor 53, a fingerprint sensor 54, and an iris sensor 55, and the like. The input module 60 can include a touch panel 61 and an external input unit 62, and the like. The wireless communication module 80 can include a Bluetooth, optical wireless communication (Optical Wireless), mobile communication (Mobile Communications), wireless local area network (WLAN), near field communication (Near Field Communication, NFC), and ZigBee, and the like.

[0035] In applications, the processor 10 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0036] In applications, the memory 20 can be an internal storage unit of the terminal device in some embodiments, for example, a hard disk or a memory of the terminal device. The memory 20 can also be an external storage device of the terminal device in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal device. The memory 20 is used to store computer programs 21 such as an operating system, an application program, a boot loader, etc. The memory 20 can also be used to temporarily store data that has been output or is to be output.

[0037] In applications, the display module 70 can include at least two display screens, Figure 1 The display module 70 is only exemplarily shown to include a first display screen 71 and a second display screen 72, etc. For any one display screen, it can be a straight-face display screen, a curved display screen, a flexible display screen, and specifically can be a folding display screen. The folding display screen can include at least one flexible display screen, or the folding display screen can include at least one flexible display screen and at least one straight-face display screen or curved display screen, wherein the flexible display screen can include a flexible panel and a rotating shaft, the rotating shaft is connected with the flexible panel, and the rotating shaft is used to drive the flexible panel to bend when rotating. The embodiments of the present application do not make any limitation on the specific type of the display module 70.

[0038] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the terminal device 100. In other embodiments of the present application, the terminal device 100 can include more or fewer components than shown, or combine certain components, or different components, for example, can also include a graphics processor, etc. The components shown can be implemented in hardware, software or a combination of software and hardware.

[0039] As Figure 2 shown in the figure, the multi-screen interaction method provided by the embodiment of the present application is applied to the terminal device, the terminal device includes at least two display screens, the first display screen is used as the main screen and outputs display content, and the second display screen is any display screen other than the main screen, and the method includes the following steps S201 to S203:

[0040] Step S201, in response to a touch event triggered on the second display screen, an auxiliary instruction is generated; the type of the auxiliary instruction is related to the gesture type of the touch event;

[0041] In application, the terminal device can listen to whether a touch event is triggered on the second display screen, and the gesture type of the touch event can include clicking and sliding, the clicking can specifically include tapping and long pressing, and the sliding can specifically include sliding belonging to a preset gesture type and sliding not belonging to the preset gesture type. Wherein, the sliding can be triggered by touch screen sliding or by air sliding; the preset gesture type can be composed of arrangement and combination of at least one of the following setting conditions: preset sliding direction, preset sliding times, and preset sliding pattern, etc. For example, the preset gesture can include: sliding once straight line from bottom to top, sliding twice circle in clockwise direction, and sliding twice double wave line from left to right (the sliding pattern of the double wave line can be generated by sliding two fingers). The specific gesture of the preset gesture type can be set according to actual needs, and the embodiment of the present application does not make any limitation on the specific gesture of the preset gesture type.

[0042] In application, after listening to that the touch event is triggered on the second display screen, the auxiliary instruction can be generated according to the touch event, and the type of the auxiliary instruction is determined according to the gesture type of the touch event. It should be noted that the auxiliary instruction does not directly act on the second display screen, but can indirectly act on the first display screen, and the method of indirectly acting on the first display screen by the auxiliary instruction will be described below.

[0043] It should be noted that the terminal device can set any display screen as the main screen according to actual needs, and taking a folding screen device including a flexible display screen and a straight display screen as an example, the flexible display screen can be folded and be off when folded, and the straight display screen can work in both folding state and unfolded state. When the folding screen device is in the unfolded state, the terminal device can set the flexible display screen or the straight display screen as the main screen according to the main screen designation instruction.

[0044] Step S202, based on a preset mapping rule between the first display screen and the second display screen, the auxiliary instruction is converted to obtain a corresponding control instruction.

[0045] In the application, all display screens of the terminal device share a preset mapping rule, or a preset mapping rule corresponding to any two display screens of the terminal device can be set in advance. Specifically, each display screen can have a unique identifier. By reading the identifier of the first display screen and the identifier of the second display screen, the terminal device can determine the preset mapping rule between the first display screen and the second display screen. The preset mapping rule is used to convert the auxiliary instruction into a corresponding control instruction according to the type of the auxiliary instruction, and the type of the control instruction is determined according to the type of the auxiliary instruction.

[0046] In step S203, a corresponding action is performed on the display content of the first display screen according to the control instruction.

[0047] In the application, after obtaining the control instruction corresponding to the auxiliary instruction, the terminal device can perform a corresponding action on the display content of the first display screen according to the control instruction, so that when using a multi-display device, the main screen can be controlled by touching the display screen other than the main screen.

[0048] In one embodiment, step S203 further includes:

[0049] In response to the touch event triggered on the first display screen, a touch instruction is generated;

[0050] In one embodiment, step S203 includes:

[0051] According to the control instruction and the touch instruction, a corresponding action is performed on the display content of the first display screen.

[0052] In the application, the terminal device can also listen to whether a touch event is triggered on the first display screen while listening to whether a touch event is triggered on the second display screen. If so, a touch instruction is generated, which is used to directly control the display content of the first display screen. The terminal device can simultaneously perform a corresponding action on the display content of the first display screen according to the control instruction and the touch instruction, so as to give the cooperation between the multi-display screens and improve the work efficiency when using the terminal device.

[0053] Figure 3 An exemplary interaction flow chart between the processor, the first display screen and the second display screen of the terminal device is shown.

[0054] A specific use scenario of the multi-screen interaction method provided by the embodiment of the application is exemplarily described below:

[0055] Taking the foldable screen device exemplified in step S201 as an example, assuming the foldable screen device is in an unfolded state, the flexible display screen is the main screen, the flat display screen is the secondary screen, and the running application is drawing. The main screen and the secondary screen can jointly perform drawing, and the drawing methods can include dual-pen mode, function selection mode, and transparent mode. Specifically, in dual-pen mode, both touch commands and control commands are used to draw based on gestures and select any drawing function. For example, control commands are used to draw straight lines based on gestures, and touch commands are used to draw curves based on gestures. In function selection mode, touch commands are used to draw based on gestures, and control commands are used to determine the function selected by the gestures. For example, control commands determine the drawing function as an eraser, and touch commands can use the eraser to remove the drawn pattern based on gestures. In transparent mode, touch commands are used to draw based on gestures and select any drawing function, and the position of the gesture on the secondary screen can also be mapped to the position of the corresponding gesture on the main screen to improve the drawing accuracy on the terminal device.

[0056] In the application, auxiliary instructions are generated in response to touch events triggered on the second display screen. The type of auxiliary instructions is related to the gesture type of the touch event. Based on the preset mapping rules between the first and second display screens, the auxiliary instructions are converted to obtain corresponding control instructions. The corresponding actions are performed on the display content of the first display screen according to the control instructions. When using a multi-display device, the main screen can be mapped and controlled by touching a display screen other than the main screen, which improves the linkage between multiple displays, enhances the utilization rate of the displays, and improves the operating efficiency of the multi-display device.

[0057] like Figure 4 As shown, in one embodiment, based on Figure 2 The corresponding embodiment includes the following steps S401 to S406:

[0058] Step S401: When a touch event is triggered on the second display screen, obtain the gesture type of the touch event;

[0059] Step S402: If the gesture type belongs to a preset gesture type, generate the first sub-assistant instruction corresponding to the preset gesture type;

[0060] Step S403: If the gesture type does not belong to the preset gesture type, generate a second sub-auxiliary instruction corresponding to the touch position and / or swipe path of the touch event.

[0061] In the application, when the terminal device listens to the touch event triggered on the second display screen, the terminal device can obtain the gesture type of the touch event and determine the type of the auxiliary instruction based on the gesture type. Specifically, the terminal device can determine whether the gesture type belongs to a preset gesture type. If yes, the terminal device generates a first sub-auxiliary instruction corresponding to the preset gesture type. Each preset gesture type has a corresponding first sub-auxiliary instruction. The corresponding relationship can be pre-stored in the terminal device, so that the first sub-auxiliary instruction corresponding to the preset gesture type can be quickly obtained by table lookup. If no, the terminal device generates a second sub-auxiliary instruction corresponding to the touch position and / or sliding path of the touch event.

[0062] In one embodiment, step S402 comprises:

[0063] If the gesture type belongs to a preset gesture type and the second display screen is in an off-screen state, the terminal device generates a first sub-auxiliary instruction corresponding to the preset gesture type.

[0064] In the application, the second display screen can implement gesture listening and generate a corresponding first sub-auxiliary instruction in an off-screen state. It should be noted that the second display screen can also implement gesture listening and generate a corresponding first sub-auxiliary instruction in a screen-on state. In the screen-on state, the second display screen can only generate a second sub-auxiliary instruction corresponding to the touch position and / or sliding path of the touch event.

[0065] Step S404: When the first sub-auxiliary instruction is received, the terminal device invokes a first sub-mapping rule between the first display screen and the second display screen, converts the first sub-auxiliary instruction, and obtains a corresponding first sub-control instruction.

[0066] Step S405: When the second sub-auxiliary instruction is received, the terminal device invokes a second sub-mapping rule between the first display screen and the second display screen, converts the second sub-auxiliary instruction, and obtains a corresponding second sub-control instruction.

[0067] The first sub-control instruction is used to locally trigger a corresponding preset function, and the second sub-control instruction is used to perform a corresponding click and / or sliding on the display content of the first display screen.

[0068] In the application, the first sub-mapping rule is used to convert the first sub-auxiliary instruction into the first sub-control instruction. The first sub-control instruction is used to locally trigger a corresponding preset function. The type of the preset function is determined according to the preset gesture type corresponding to the first sub-auxiliary instruction. The corresponding relationship between the type of the preset function and the preset gesture type can be pre-stored in the terminal device or can be customized by the user.

[0069] In the application, the second sub-mapping rule is used to convert the second sub-assistant instruction into a second sub-control instruction, where the second sub-control instruction is used to perform a corresponding click and / or slide on the display content of the first display screen, the click position and / or slide path being determined according to the touch position and / or slide path of the touch event, and specifically, the click position and / or slide path of the second sub-control instruction can be determined according to the preset mapping relationship in the second sub-mapping rule. The preset mapping relationship can determine the relative position of the click position and / or slide path of the second sub-control instruction in the first display screen according to the relative position of the touch position and / or slide path of the touch event in the second display screen.

[0070] In step S406, a corresponding action is performed on the display content of the first display screen according to the control instruction.

[0071] In the application, the multi-screen interaction method provided in step S406 can refer to the related description of step S203, which will not be repeated here. The difference is that the control instruction can include the first sub-control instruction and the second sub-control instruction.

[0072] In the application, the terminal device can determine the gesture type of the touch event after triggering the touch event on the second display screen; if it belongs to the preset gesture type, the first sub-assistant instruction is generated and converted into the first sub-control instruction to realize gesture control on the first display screen; if it does not belong to the preset gesture type, the second sub-assistant instruction is generated and converted into the second sub-control instruction to realize mapping click and / or slide on the first display screen, thereby improving the flexibility of multi-screen interaction.

[0073] As shown in FIG. 1, Figure 5 According to the embodiment corresponding to Figure 4 The embodiment includes the following steps S501 to S507:

[0074] In step S501, when the event is triggered on the second display screen, the gesture type of the touch event is obtained.

[0075] In step S502, if the gesture type belongs to the preset gesture type, the first sub-assistant instruction corresponding to the preset gesture type is generated.

[0076] In step S503, if the gesture type does not belong to the preset gesture type, the second sub-assistant instruction corresponding to the touch position and / or slide path of the touch event is generated.

[0077] In step S504, when the first sub-assistant instruction is received, the first sub-mapping rule between the first display screen and the second display screen is called to convert the first sub-assistant instruction to obtain the corresponding first sub-control instruction.

[0078] Step S505. Upon receiving the second sub-assistance instruction, a second sub-mapping rule between the first display screen and the second display screen is invoked to convert the second sub-assistance instruction to obtain a corresponding second sub-control instruction.

[0079] The first sub-control instruction is used to trigger a corresponding preset function locally, and the second sub-control instruction is used to perform a corresponding click and / or swipe on the display content of the first display screen.

[0080] In application, the multi-screen interaction method provided by steps S501 to S505 is consistent with the multi-screen interaction method provided by steps S401 to S405 described above, and will not be repeated here.

[0081] Step S506. Based on a preset mapping range between the first display screen and the second display screen, a valid range of the control instruction in the first display screen is determined.

[0082] In application, the preset mapping range is used to determine the valid range of the control instruction in the first display screen. Specifically, the display range of the first display screen can include a first display range and a second display range (for example, the first display range can correspond to the left half of the main screen of the folding screen device, and the second display range can correspond to the right half of the main screen of the folding screen device). When the preset mapping range is full screen, the valid range of the control instruction in the first display screen includes the first display range and the second display range. When the preset mapping range is the first display range, the valid range of the control instruction in the first display screen is the first display range. When the preset mapping range is the second display range, the valid range of the control instruction in the first display screen is the second display range.

[0083] It should be noted that the division of the display range of the first display screen can be set according to actual needs, and can be divided by the user himself to further improve the flexibility and customization ability of multi-screen interaction, and precise mapping control can be realized in the specified display range.

[0084] Step S507. According to the control instruction, a corresponding action is performed on the display content of the first display screen.

[0085] In application, the multi-screen interaction method provided by step S507 can refer to the related description of step S406 described above, and will not be repeated here.

[0086] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0087] As Figure 6As shown, the embodiments of the present application also provide a multi-screen interaction device for executing the steps of the above-mentioned multi-screen interaction method embodiments. The multi-screen interaction device can be a virtual appliance in a terminal device, run by a processor of the terminal device, or can be the terminal device itself.

[0088] As shown, the embodiments of the present application provide a multi-screen interaction device 6, which includes: Figure 6

[0089] The response module 610 is configured to generate an auxiliary instruction in response to a touch event triggered on the second display screen; the type of the auxiliary instruction is related to the gesture type of the touch event; the type of the auxiliary instruction is related to the gesture type of the touch event;

[0090] The mapping module 620 is configured to convert the auxiliary instruction based on a preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction;

[0091] The execution module 630 is configured to perform a corresponding action on the display content of the first display screen according to the control instruction.

[0092] In one embodiment, the response module 610 includes:

[0093] The gesture type judgment unit is configured to:

[0094] When the touch event is triggered on the second display screen, the gesture type of the touch event is obtained;

[0095] If the gesture type belongs to a preset gesture type, a first sub-auxiliary instruction corresponding to the preset gesture type is generated;

[0096] If the gesture type does not belong to the preset gesture type, a second sub-auxiliary instruction corresponding to the touch position and / or the sliding path of the touch event is generated.

[0097] In one embodiment, the multi-screen interaction device 6 further includes:

[0098] The mapping range unit is configured to determine the effective range of the control instruction in the first display screen based on a preset mapping range between the first display screen and the second display screen.

[0099] In one embodiment, the multi-screen interaction device 6 further includes:

[0100] The synchronous touch unit is configured to generate a touch instruction in response to a touch event triggered on the first display screen;

[0101] The above-mentioned execution module 630 includes:

[0102] ​The synchronization execution unit is configured to execute corresponding actions on the display content of the first display screen according to the control instruction and the touch instruction.

[0103] In the application, each module in the multi-screen interaction device 6 can be a software program module, can be realized by different logical circuits integrated in the processor, or can be realized by multiple distributed processors.

[0104] It should be noted that the information interaction, execution process and the like between the above modules are based on the same concept as the method embodiments of the present application, and the specific functions and technical effects brought by them can be referred to the multi-screen interaction method embodiments, which will not be described here.

[0105] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. Each functional module in the embodiment can be integrated in one processing module, or each module can be physically present separately, or two or more modules can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of software functional module. In addition, the specific name of each functional module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0106] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps in each of the above multi-screen interaction method embodiments.

[0107] The integrated module, if implemented in the form of a software function module and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can be instructed by a computer program to relevant hardware to complete, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium at least includes any entity or device capable of carrying the computer program code to the photographing terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0108] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0109] Those of ordinary skill in the art can realize that the modules and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0110] In the embodiments provided in the present application, it should be understood that the disclosed terminal device and method can be implemented in other ways. For example, the terminal device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutually can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0111] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A multi-screen interaction method, characterized in that, The method is applied to a terminal device, the terminal device comprising at least two display screens, a first display screen serving as a main screen and outputting display content, and a second display screen being any display screen other than the main screen, the method comprising: generating an auxiliary instruction in response to a touch event triggered on the second display screen, the type of the auxiliary instruction being related to the gesture type of the touch event; converting the auxiliary instruction based on a preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction; performing a corresponding action on the display content of the first display screen according to the control instruction; the method further comprising: determining the effective range of the control instruction in the first display screen based on a preset mapping range between the first display screen and the second display screen; the display range of the first display screen comprising a first display range and a second display range, and the determination of the effective range of the control instruction in the first display screen based on the preset mapping range between the first display screen and the second display screen comprising: when the preset mapping range is full screen, the effective range of the control instruction in the first display screen comprising the first display range and the second display range; when the preset mapping range is the first display range, the effective range of the control instruction in the first display screen being the first display range; when the preset mapping range is the second display range, the effective range of the control instruction in the first display screen being the second display range. 2.The multi-screen interaction method of claim 1, wherein, the generation of the auxiliary instruction in response to the touch event triggered on the second display screen comprising: when the touch event is triggered on the second display screen, obtaining the gesture type of the touch event; if the gesture type belongs to a preset gesture type, generating a first sub-auxiliary instruction corresponding to the preset gesture type; if the gesture type does not belong to a preset gesture type, generating a second sub-auxiliary instruction corresponding to the touch position and / or sliding path of the touch event. 3.The multi-screen interaction method of claim 2, wherein, the preset mapping rule comprising a first sub-mapping rule and a second sub-mapping rule, and the conversion of the auxiliary instruction based on the preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction comprising: when the first sub-auxiliary instruction is received, the first sub-mapping rule between the first display screen and the second display screen being called to convert the first sub-auxiliary instruction to obtain a corresponding first sub-control instruction; when the second sub-auxiliary instruction is received, the second sub-mapping rule between the first display screen and the second display screen being called to convert the second sub-auxiliary instruction to obtain a corresponding second sub-control instruction; wherein the first sub-control instruction is used to trigger a corresponding preset function locally, and the second sub-control instruction is used to perform a corresponding click and / or slide on the display content of the first display screen.

4. The multi-screen interaction method of any one of claims 1 to 3, wherein, the method further comprising: generating a touch instruction in response to a touch event triggered on the first display screen; the performance of a corresponding action on the display content of the first display screen according to the control instruction comprising: the performance of a corresponding action on the display content of the first display screen according to the control instruction and the touch instruction.

5. A multi-screen interaction device, characterized in that, The method comprises: a response module, configured to generate an auxiliary instruction in response to a touch event triggered on the second display screen; the type of the auxiliary instruction is related to the gesture type of the touch event; a mapping module, configured to convert the auxiliary instruction based on a preset mapping rule between the first display screen and the second display screen to obtain a corresponding control instruction; an execution module, configured to perform a corresponding action on the display content of the first display screen according to the control instruction. The multi-screen interaction device further comprises: a mapping range unit, configured to determine the effective range of the control instruction in the first display screen based on a preset mapping range between the first display screen and the second display screen. The display range of the first display screen comprises a first display range and a second display range, and the determination of the effective range of the control instruction in the first display screen based on the preset mapping range between the first display screen and the second display screen comprises: when the preset mapping range is full screen, the effective range of the control instruction in the first display screen comprises the first display range and the second display range; when the preset mapping range is the first display range, the effective range of the control instruction in the first display screen is the first display range; when the preset mapping range is the second display range, the effective range of the control instruction in the first display screen is the second display range.

6. A terminal device, characterized by comprising: The computer program is executed by the processor to implement the steps of the multi-screen interaction method according to any one of claims 1 to 4.

7. The terminal device according to claim 6, characterized by At least one display screen comprises a flexible panel and a rotating shaft, and the rotating shaft is connected to the flexible panel. The rotating shaft is used to drive the flexible panel to bend when rotating.

8. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the multi-screen interaction method according to any one of claims 1 to 4.

Images