Split-screen interaction method and electronic device

Abstract

Embodiments of the present application provide a split-screen interaction method and an electronic device. The method is applied to an electronic device. The method comprises: displaying a first split-screen interface and a second split-screen interface, the first split-screen interface displaying content of a first application, the second split-screen interface displaying content of a second application, and the first application being different from the second application; and, in response to an operation by a user in an interaction region of the first split-screen interface, the electronic device displaying on the second split-screen interface second content associated with first content displayed in the interaction region, and / or the electronic device displaying on the second split-screen interface a recommended service associated with the first content displayed in the interaction region. By means of the method or the electronic device, interaction content selected by the user in one split screen can be sensed by the other split screen, and a response can be made to the interaction content in the other split screen, so that the efficiency of split-screen processing of a plurality of tasks by the user can be improved and the split screen use experience of the user can be improved.

Description

Split-screen interaction methods and electronic devices

[0001] This application claims priority to Chinese Patent Application No. 202411456967.0, filed on October 17, 2024, entitled “Method and Electronic Device for Split-Screen Interaction”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of split-screen interaction, and more specifically, to a method and electronic device for split-screen interaction. Background Technology

[0003] With the development of device intelligence, the split-screen function of foldable screen devices and large screen devices has gradually become a common function for users in learning, office and other scenarios. Through the device's split-screen function, users can handle multiple tasks at the same time, which significantly improves learning and work efficiency.

[0004] However, in the current split-screen solutions, the interactive experience between multiple split screens still needs to be improved. Summary of the Invention

[0005] This application provides a method and electronic device for split-screen interaction. Through this method and electronic device, the content selected by the user in one split-screen application can be perceived by another split-screen application, and the user can respond to the content in the interface of the other split-screen application. This can improve the efficiency of users in handling multiple tasks in split-screen mode and enhance the user's split-screen experience.

[0006] In a first aspect, a method for split-screen interaction is provided, applied to an electronic device. The method includes: displaying a first split-screen interface and a second split-screen interface, wherein the first split-screen interface displays content of a first application, and the second split-screen interface displays content of a second application, wherein the first application is different from the second application; in response to an operation by a user in an interactive area of ​​the first split-screen interface, the electronic device displays second content associated with the first content displayed in the interactive area on the second split-screen interface, and / or, the electronic device displays a recommended service associated with the first content displayed in the interactive area on the second split-screen interface.

[0007] In this embodiment, the content selected by the user in one split-screen application of the electronic device can be perceived by another split-screen application, and the user can respond to the content in the interface of the other split-screen application. For example, the selected content and / or recommended services associated with the selected content can be displayed in the interface of the other split-screen application, which can improve the efficiency of the user in handling multiple tasks in split-screen and enhance the user's split-screen experience.

[0008] In conjunction with the first aspect, in one possible implementation, the second content associated with the first content includes: content obtained by copying the first content, content obtained by completing the first content, or content obtained by format conversion of the first content.

[0009] In this embodiment, the content displayed in the interactive area of ​​the first split-screen interface of the electronic device may differ from the related content displayed in the second split-screen interface in terms of text completeness and / or format. For example, when the content displayed in the interactive area is incomplete, the electronic device can display the completed content on the second split-screen interface. In this way, the user does not need to spend a long time accurately selecting the interactive text. For another example, when an uneditable table is displayed in the interactive area, the electronic device can display the corresponding editable table on the second split-screen interface. This can improve the user's efficiency in multitasking with split-screen mode and thus enhance the user's split-screen interaction experience.

[0010] In conjunction with the first aspect, in one possible implementation, the recommendation service associated with the first content includes: a recommendation service associated with the modal type of the first content and / or a recommendation service associated with the meaning of the first content, wherein the modal type is used to indicate the carrying form and / or expression form of the first content.

[0011] For example, the format of the content can include sound, document, vibration, control, video, image, photo, etc. For instance, the first content may be carried in the form of a document (which can be further divided into various subtypes such as .txt, .doc, .xlsx, .pdf, etc.), or the first content may be carried in the form of an image (which can be further divided into subtypes such as .jpg, .png, etc.).

[0012] Within the same format, multiple forms of expression can be included. For example, a .doc document can include links and images, where the content expressed by the links and the content expressed by the images belong to different modalities.

[0013] Content can also be carried in different formats in the same form. For example, a table can be carried in an .xlsx document, a .pdf document, and a .doc document. These table information carried in different ways belong to different modal types.

[0014] In this embodiment, cross-screen perception of multiple modal types of display information can be achieved between the split-screen interfaces of different applications displayed on the electronic device. This enables users to achieve cross-screen interaction of multiple modal types of display information between the split-screen interfaces of different applications, which can further improve the efficiency of users in handling multiple tasks in split-screen mode and enhance the user's split-screen experience.

[0015] Furthermore, in the second split-screen interface, recommended services can be suggested to the user that are related to the modal type of the content displayed in the interactive area, allowing the user to quickly perform some general services on the content displayed in the interactive area in the second split-screen interface; recommended services can also be suggested to the user that are related to the meaning of the content displayed in the interactive area, allowing the user to quickly perform some targeted services on the content displayed in the interactive area in the second split-screen interface.

[0016] In conjunction with the first aspect, in one possible implementation, when the modal type of the first content includes text, the recommendation service associated with the modal type of the first content includes one or more of text interpretation service, text editing service, text translation service, and text sharing service; and / or when the modal type of the first content includes table, the recommendation service associated with the modal type of the first content includes one or more of table file generation service, table editing service, table interpretation service, and data analysis service; and / or when the modal type of the first content includes image, the recommendation service associated with the modal type of the first content includes one or more of image interpretation service, image editing service, text extraction service, image translation service, and image search service.

[0017] In conjunction with the first aspect, in one possible implementation, when the modal type of the first content includes multiple modal types, the recommendation service associated with the modal type of the first content includes recommendation services associated with at least one of the multiple modal types.

[0018] In this embodiment of the application, when the modal types of the content displayed in the interactive area include multiple types, the second split-screen interface can recommend services associated with the multiple modal types of the content displayed in the interactive area to the user. This allows the user to quickly perform some comprehensive services on the content displayed in the interactive area in the second split-screen interface, such as data aggregation services and comprehensive analysis services.

[0019] In conjunction with the first aspect, in one possible implementation, when the modal type of the first content includes a first modal type and a second modal type, the method further includes: in response to a user's operation of editing content corresponding to the first modal type in the second split-screen interface, the electronic device synchronously updates the content displayed in the second split-screen interface corresponding to the second modal type according to the edited content.

[0020] In this embodiment, when the modal types of the content displayed in the interactive area include multiple types, the associated second content displayed in the second split-screen interface also corresponds to multiple modal types. The second content can be editable. When the user edits the content corresponding to one of the multiple modal types of content displayed in the second split-screen interface, the content associated with the edited content in the other modal types will also be adaptively updated. This allows the user to quickly organize data.

[0021] In conjunction with the first aspect, in one possible implementation, the second content is the content after the electronic device performs sentence completion and / or paragraph completion on the first content.

[0022] In this embodiment, when the content displayed in the interactive area is incomplete, the related content displayed by the electronic device on the second split-screen interface can be the content after the content displayed in the interactive area has been completed. In this way, the user does not need to spend a long time to accurately select the interactive text, which can improve the user's work efficiency in split-screen multitasking and thus enhance the user's split-screen interaction experience.

[0023] In conjunction with the first aspect, in one possible implementation, the second split-screen interface also displays a voice control, and the method further includes: in response to a user's touch operation on the voice control, the electronic device acquires a first voice command; the electronic device displays a first result on the second split-screen interface based on the first voice command and the first content.

[0024] In this embodiment, users can ask and answer questions on the second split-screen interface based on the content displayed in the interactive area of ​​the first split-screen interface. Correspondingly, the second split-screen interface displays the question and answer results generated based on the content displayed in the interactive area, which can improve the user's efficiency in handling multiple tasks in split-screen mode and thus enhance the user's split-screen interaction experience.

[0025] In conjunction with the first aspect, in one possible implementation, the electronic device displays a first result on the second split-screen interface based on the first voice command and the first content, including: the electronic device displays the first result on the second split-screen interface based on the content displayed on the first split-screen interface, the first voice command, and the first content.

[0026] In this embodiment, the user can ask and answer questions on the second split-screen interface based on the content displayed in the interactive area of ​​the first split-screen interface. Correspondingly, the second split-screen interface displays the question and answer results generated based on the content displayed in the interactive area and the content displayed in the first split-screen interface. In this embodiment, the question and answer results obtained through the question and answer format can be generated based on the contextual semantics of the content displayed on the first split-screen interface, which makes the accuracy of the obtained question and answer results higher.

[0027] In conjunction with the first aspect, in one possible implementation, the method further includes: establishing a correspondence between the display area and modal type in the current display interface of the first split-screen interface through controls in the first split-screen interface, or by performing layout analysis on the first split-screen interface, or by performing document parsing on the first split-screen interface; and obtaining the modal type corresponding to the first content displayed in the interactive area based on the correspondence.

[0028] In conjunction with the first aspect, in one possible implementation, in response to a user's operation in the interactive area of ​​the first split-screen interface, the electronic device displays second content associated with the first content displayed in the interactive area on the second split-screen interface, and / or, the electronic device displays a recommendation service associated with the first content displayed in the interactive area on the second split-screen interface, including: in response to a user's operation in the interactive area of ​​the first split-screen interface, the electronic device acquires the first content displayed in the interactive area; the electronic device identifies the first content according to the modal type corresponding to the first content to acquire the second content; the electronic device generates the recommendation service associated with the first content based on the first content and / or the second content; the electronic device displays the second content on the second split-screen interface, and / or, the electronic device displays the recommendation service associated with the first content on the second split-screen interface.

[0029] In a second aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the first aspect or any possible implementation thereof.

[0030] Thirdly, a computer-readable storage medium is provided, which stores a computer program or instructions that, when executed, implement the method described in the first aspect or any possible implementation thereof.

[0031] Fourthly, a chip is provided, wherein instructions are stored that, when executed on a device, cause the chip to perform the methods described in the first aspect or any possible implementation thereof.

[0032] Fifthly, a computer program product is provided, which stores a computer program or instructions that, when executed, implement the method in the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0033] Figure 1 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application;

[0034] Figure 2 is a software structure block diagram of the electronic device provided in an embodiment of this application;

[0035] Figure 3 is a schematic flowchart of a split-screen interaction method provided in an embodiment of this application;

[0036] Figure 4 is a schematic diagram of the interface corresponding to a split-screen interaction method provided in an embodiment of this application;

[0037] Figure 5 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application;

[0038] Figure 6 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0039] Figure 7 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0040] Figure 8 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0041] Figure 9 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0042] Figure 10 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0043] Figure 11 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0044] Figure 12 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0045] Figure 13 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0046] Figure 14 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiment of this application;

[0047] Figure 15 is a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application;

[0048] Figure 16 is a schematic flowchart of another split-screen interaction method provided in an embodiment of this application;

[0049] Figure 17 is a schematic architecture diagram of a split-screen interaction device provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments.

[0051] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "plural" or "multiple" refers to two or more than two.

[0052] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0053] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0054] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "one embodiment," "some embodiments," "another embodiment," "other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0055] The methods provided in this application can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0056] For example, Figure 1 shows a schematic diagram of the structure of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0057] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0058] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0059] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0060] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0061] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0062] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0063] The charging management module 140 receives charging input from the charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0064] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be housed in the same device. The wireless communication function of the electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0065] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0066] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0067] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0068] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0069] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0070] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0071] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0072] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0073] The SIM card interface 195 is used to connect a SIM card. The electronic device 100 interacts with the network through the SIM card to realize functions such as making calls and data communication.

[0074] It should be understood that the phone cards in the embodiments of this application include, but are not limited to, SIM cards, eSIM cards, universal subscriber identity modules (USIM), universal integrated circuit cards (UICC), etc.

[0075] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0076] Figure 2 is a software structure block diagram of an electronic device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0077] As shown in Figure 2, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0078] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0079] As shown in Figure 2, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0080] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0081] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0082] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0083] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0084] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0085] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0086] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0087] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0088] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0089] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0090] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0091] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0092] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0093] A 2D graphics engine is a graphics engine for 2D drawing.

[0094] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0095] It should be understood that the technical solutions in the embodiments of this application can be used in systems such as Android, iOS, and HarmonyOS.

[0096] The technical solutions of this application embodiment can be applied to electronic devices with display capabilities, for example. Exemplarily, they can be applied to televisions, desktop computers, laptops, in-vehicle screens, and portable electronic devices such as mobile phones, foldable screens, tablets, cameras, camcorders, video recorders, watches, and wristbands. They can also be applied to other electronic devices with display capabilities, and to electronic devices in future networks or in future evolved public land mobile networks (PLMNs), etc. The main application scenario can be split-screen interaction scenarios for electronic devices.

[0097] As smart devices become increasingly popular, foldable screen devices and large-screen devices are gradually becoming common devices for users in scenarios such as learning and working. Among them, foldable screen devices and large-screen devices, due to their large display space, frequently use their split-screen display function. Through the split-screen display function of foldable screen devices and large-screen devices, users can handle multiple tasks at the same time, which can significantly improve users' learning and work efficiency.

[0098] When an electronic device's screen is split into multiple independent operating areas, the user's actions on one split screen will only be responded to on that split screen. However, when users use electronic devices with split screens, the multiple processing tasks opened simultaneously through multiple split screens are often related. Therefore, the interactive experience between multiple split screens has become an increasingly important issue for users.

[0099] In some embodiments, taking the two split-screen interfaces displayed on the electronic device as a gallery application interface and a shopping application interface as an example, the user needs to manually fill in the address information from the gallery application interface into the shopping application interface in order to save the address as the shopping application's delivery address. For example, the user can drag and drop the address information recognized in the gallery application interface across screens to the shopping application interface to fill in the address information recognized in the gallery application interface into the shopping application's add delivery address page. Alternatively, the user can copy the recognized address information from the gallery application interface and then paste the address information into the corresponding position on the shopping application's add delivery address page. Or, the user can manually enter the address information into the corresponding position on the shopping application's add delivery address page by referring to the address information in the gallery application interface.

[0100] Users can obtain text from the text source interface by copying or long-pressing to select, and copy the text obtained in one application interface to another application interface by pasting or dragging across screens. Furthermore, users can only manually interact with the text content in the interface.

[0101] Therefore, in current split-screen interaction methods, the flow of information and data between split screens requires manual operation by the user. The device cannot perceive and synchronize user interactions across screens, making it difficult for users to easily synchronize relevant information or perform related operations across screens when handling multiple tasks in split-screen mode. This increases the complexity of split-screen operations and negatively impacts the user experience. For example, when using a smart assistant app on one split screen to ask and answer questions about a document displayed on another split screen, the user needs to manually input the document information from the other split screen into the smart assistant app's interface before they can ask and answer questions based on that document information.

[0102] In view of this, embodiments of this application provide a method and electronic device for split-screen interaction. Through this method and electronic device, cross-screen perception between multiple split screens can be achieved. The interactive content of a user in one split screen can be perceived by another split screen, and the user can respond to the interactive content in the other split screen. For example, the user can display the interactive content in the other split screen and / or actively recommend services related to the interactive content to the user in the other split screen. This can improve the efficiency of users in handling multiple tasks in split screen and enhance the user's split-screen experience.

[0103] Furthermore, in the method provided in this application embodiment, the device can obtain the modal types of display information in different areas within the interactive screen. When a user selects interactive content in one split screen, the device can specifically identify the interactive content based on its corresponding modal type. Based on the interactive content and its corresponding modal type, the device can recommend related services to the user in another split screen. For example, when the user selects a table in one split screen, table-related recommendations can be provided on another split screen; when the user selects text in one split screen, text-related recommendations can be provided on another split screen; when the user selects an image in one split screen, image-related recommendations can be provided on another split screen; when the user selects multiple modal types in one split screen, recommendations associated with at least one of those modal types can be provided on another split screen. This enables interaction between multiple split screens based on multiple modal types of content, thereby providing users with more accurate and diversified services and further enhancing the user's split-screen experience.

[0104] The modal type may include text type, image type, table type, control type, link type, etc., and may also include other modal content, which is not limited in this application.

[0105] It should be understood that the modal type described in the embodiments of this application can refer to the form in which information is carried, such as sound, document, vibration, control, video, image, photograph, etc. For example, information can be carried in the form of a document (which can be further divided into various subtypes such as .txt, .doc, .xlsx, .pdf, etc.), or in the form of an image (which can be further divided into subtypes such as .jpg, .png, etc.). Modal type can also refer to the form in which information is expressed. For example, within the same carrying form, multiple forms of information expression can be included. For instance, a .doc document can include links and images, and the information expressed by the links and the information expressed by the images in the document belong to different modal types. Information can also be carried in the same form by different carrying forms. For example, a table can be carried in an .xlsx document, a .pdf document, and a .doc document; these table information carried in different ways belong to different modal types.

[0106] In an optional example, an image 1 is displayed in a split-screen interface of an electronic device. This image includes Table 1, Image 2, and Link 1. This can be understood as the split-screen interface containing information of three modalities: the table type corresponding to Table 1, the image type corresponding to Image 2, and the link type corresponding to Link 1. In other words, when the content displayed on the split-screen interface corresponds to one format, or when this content corresponds to multiple formats, it can also be understood as the content displayed on the split-screen interface being multi-modal.

[0107] In another optional example, when Table 1 is displayed as a document in the split-screen interface, Table 1 is displayed as a table, and the modal type of Table 1 is Table type; when Table 1 is displayed as an image in the split-screen interface, Table 1 is displayed as a table, and the modal type of Table 1 is Table type; when Table 1 is displayed as a photo in the split-screen interface, Table 1 is displayed as a table, and the modal type of Table 1 is Table type.

[0108] For example, Figure 3 shows a schematic flowchart of a split-screen interaction method 300 provided in an embodiment of this application.

[0109] The method 300 is applied to an electronic device whose display screen shows a first split-screen interface and a second split-screen interface. The first split-screen interface displays content from a first application, and the second split-screen interface displays content from a second application. The first application is different from the second application. As shown in Figure 3, the method 300 includes:

[0110] S301: In response to a user's operation in the interactive area of ​​the first split-screen interface of the electronic device, the electronic device displays second content associated with the first content displayed in the interactive area on the second split-screen interface, and / or, the electronic device displays recommended services associated with the first content displayed in the interactive area on the second split-screen interface.

[0111] The electronic device displays second content on the second split-screen interface that is associated with the first content displayed in the interactive area. The relationship between the first content and the second content can include any of the following relationships:

[0112] (1) The second content is obtained by copying the first content.

[0113] In one example, the first content could be the entirety of the first statement, and correspondingly, the second content could be the entirety of the first statement.

[0114] In some examples, the first content could be the entire first paragraph, and correspondingly, the second content could be the entire first paragraph.

[0115] In some examples, the user's operation in the interactive area of ​​the first split-screen interface could be, for example, triggering the recognition of the content displayed on the first split-screen interface and then copying the first content from the recognized content, or selecting the first content through clicking, long pressing, voice commands, selection, etc.

[0116] (2) Copy the first content and complete the copied first content according to the content displayed in the first split screen to obtain the second content.

[0117] In some examples, the first content may be a part of the first statement, and correspondingly, the second content may be the entire first statement.

[0118] In some examples, the first content may be a part of the first paragraph, and correspondingly, the second content may be the entire first paragraph.

[0119] (3) By recognizing the first content and converting the format of the recognized first content, the second content is obtained.

[0120] In one example, the first content could be a table in the form of an image, and the second content could be an HTML table obtained by recognizing the table in the image, converting the table to a new format.

[0121] In one example, the first content could be an image, which could be recognized and converted into a format that supports display on the second split screen or into an editable format. Correspondingly, the second content would be the converted image.

[0122] (4) By identifying the first content and copying the identified first content, the second content is obtained.

[0123] In one example, the first content could be an image representing the first address information. This first content can be identified and copied. Correspondingly, the second content could be the first address information displayed as editable text.

[0124] In some embodiments, the second content associated with the first content includes: content obtained by copying the first content, content obtained by completing the first content, and / or content obtained by format conversion of the first content.

[0125] The first content can correspond to one or more modal types, such as text, image, table, or link types.

[0126] The first content may include one or more of text, images, tables, and links. In addition, the first content may also include content of other modal types, which are not limited in this application.

[0127] In some embodiments, the recommendation service associated with the first content may include a recommendation service associated with the modal type of the first content and / or a recommendation service associated with the meaning of the first content. The modal type is used to indicate the carrying format and / or expression format of the first content.

[0128] For example, the format of the content can include sound, document, vibration, control, video, image, photo, etc. For instance, the first content may be carried in the form of a document (which can be further divided into various subtypes such as .txt, .doc, .xlsx, .pdf, etc.), or the first content may be carried in the form of an image (which can be further divided into subtypes such as .jpg, .png, etc.).

[0129] Within the same format, multiple forms of expression can be included. For example, a .doc document can include links and images, where the content expressed by the links and the content expressed by the images belong to different modalities.

[0130] Content can also be carried in different formats in the same form. For example, a table can be carried in an .xlsx document, a .pdf document, and a .doc document. These table information carried in different ways belong to different modal types.

[0131] In an optional example, when the modal type of the first content includes text, the recommendation service associated with the modal type of the first content may include one or more of the following: text interpretation service, text editing service, text translation service, and text sharing service.

[0132] In an optional example, when the modal type of the first content includes a table type, the recommendation services associated with the modal type of the first content include one or more of the following: generating table files service, table editing service, interpreting table service, and data analysis service.

[0133] In an optional example, when the modal type of the first content includes an image type, the recommendation services associated with the modal type of the first content include one or more of the following: image interpretation service, image editing service, text extraction service, image translation service, and image search service.

[0134] In an optional example, when the first content includes address information, the recommendation services associated with the meaning of the first content may include navigation services, route viewing services, etc.

[0135] In an optional example, when the first content includes account information, the recommendation services associated with the meaning of the first content may include transfer services, save services, etc.

[0136] In an optional example, when the first content includes flight information, the recommended services associated with the meaning of the first content may include flight booking services, flight search services, etc.

[0137] In some embodiments, when the modal type of the first content includes multiple modal types, the recommendation service associated with the modal type of the first content includes recommendation services associated with at least one of the multiple modal types.

[0138] In an optional example, when the modal type of the first content includes text type and table type, the recommendation service associated with the modal type of the first content may include one or more of the following: comprehensive interpretation service, comprehensive analysis service, and data aggregation service.

[0139] In some embodiments, when the modal type of the first content includes a first modal type and a second modal type, the method 300 may further include: in response to a user's operation of editing content corresponding to the first modal type in the second split-screen interface, the electronic device synchronously updates the content displayed in the second split-screen interface corresponding to the second modal type according to the content corresponding to the edit.

[0140] In an optional example, when the first content includes a first table and a first image, the second split-screen interface of the electronic device can display the first table in an editable state. When the user modifies a piece of data in the first table, correspondingly, other data in the table associated with that data are also updated adaptively, and the content displayed in the first image associated with that data is also updated adaptively.

[0141] In some embodiments, the second content may be the content obtained by the electronic device after performing sentence completion and / or paragraph completion on the first content.

[0142] In some embodiments, a voice control may also be displayed on the second split-screen interface, and the method 300 may further include: in response to a user's touch operation on the voice control, the electronic device acquires a first voice command; the electronic device displays a first result on the second split-screen interface according to the first voice command and the first content.

[0143] In some embodiments, the electronic device displays a first result on a second split-screen interface based on the first voice command and the first content, including: the electronic device displays the first result on the second split-screen interface based on the content displayed on the first split-screen interface, the first voice command, and the first content.

[0144] The user's touch operation on the voice control can be a click, a long press, or a swipe.

[0145] For example, Figure 4 shows a schematic diagram of the interface corresponding to a split-screen interaction method provided in an embodiment of this application.

[0146] As shown in Figure 4(a), the display screen of the electronic device 400 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays one or more of the following: text 401, table 402 and picture 403. The second split screen displays, for example, a smart assistant page.

[0147] As shown in Figure 4(b), when the user selects all the content of text 401 in the first split screen, the second split screen of the electronic device 400 displays all the content of text 401, and in an optional example, one or more recommendation services associated with text 401 may also be displayed in the second split screen.

[0148] The one or more recommendation services associated with the text 401 may include, for example, recommendation services associated with the modality type of the text 401, and may also include recommendation services associated with the semantics of the text 401.

[0149] Optionally, the recommendation services associated with the modality type of the text 401 may include text interpretation services, text translation services, text sharing services, etc., and may also include other recommendation services, such as text saving services, etc., which are not limited in this application.

[0150] Optionally, the recommendation service associated with the semantics of the text 401 may include a currency conversion service, through which a user can convert the content "734748 million yuan" in the text 401 into other currency units, such as US dollars.

[0151] In some optional examples, when a user selects the text sharing service in the second split screen, the display interface of the second split screen can jump to the display interface for sharing text 401 with a friend. After the user selects a friend to share with in this display interface, the electronic device 400 can send the content of text 401 to that friend.

[0152] In some embodiments, the user can select text 401 in the first split screen by performing a selection operation directly on the first split screen. For example, the user can select text 401 by long-pressing and selecting text content on the first split screen; the user can select text 401 by circling text content on the first split screen; the user can select text 401 by directly clicking on text content on the first split screen; the user can select text 401 by voice instruction; or the user can select text 401 in other ways, such as selecting text by multi-finger swipe gestures, selecting text by stylus, selecting text by hover touch, selecting text by visual tracking, etc. This application does not limit this method.

[0153] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. This trigger operation may be, for example, a long press operation, a click operation to recognize the control, or other specific gesture operations. After the displayed content in the first split screen is converted into a selectable state, the user can select the text 401 by clicking, swiping to select, dragging, voice instruction, stylus selection, hover touch, visual following, or circle selection.

[0154] For example, Figure 5 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0155] As shown in Figure 5(a), the display screen of the electronic device 400 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays one or more of the following: text 401, table 402 and picture 403. The second split screen displays, for example, a smart assistant page.

[0156] As shown in Figure 5(b), when a user selects a portion of text 401 in the first split screen, the second split screen of the electronic device 400 can display the completed content of the portion of text 401. For example, if the first paragraph is selected in the first split screen, the entire first paragraph is displayed in the second split screen. Or, if the first sentence is selected in the first split screen, the entire first sentence is displayed in the second split screen. In an optional example, one or more recommendation services associated with text 401 can also be displayed in the second split screen.

[0157] The one or more recommendation services associated with the text 401 may include, for example, recommendation services associated with the modality type of the text 401, and may also include recommendation services associated with the semantics of the text 401.

[0158] Optionally, the recommendation services associated with the modality type of the text 401 may include text interpretation services, text translation services, text sharing services, etc., and may also include other recommendation services, such as text saving services, text editing services, etc., which are not limited in this application.

[0159] Optionally, the recommendation service associated with the semantics of the text 401 may include a currency conversion service, through which a user can convert the content "734748 million yuan" in the text 401 into other currency units, such as US dollars.

[0160] In some optional examples, when a user selects the text sharing service in the second split screen, the display interface of the second split screen can jump to the display interface for sharing text 401 with a friend. After the user selects a friend to share with in this display interface, the electronic device 400 can send the content of text 401 to that friend.

[0161] In some embodiments, the user can select text 401 in the first split screen by performing a selection operation directly on the first split screen. For example, the user can select text 401 by long-pressing and selecting text content on the first split screen; the user can select text 401 by circling text content on the first split screen; the user can select text 401 by directly clicking on text content on the first split screen; the user can select text 401 by voice instruction; or the user can select text 401 in other ways, such as selecting text by multi-finger swipe gestures, selecting text by stylus, selecting text by hover touch, selecting text by visual tracking, etc. This application does not limit this method.

[0162] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. This trigger operation may be, for example, a long press operation, a click operation to recognize the control, or other specific gesture operations. After the displayed content in the first split screen is converted into a selectable state, the user can select the text 401 by clicking, swiping to select, dragging, voice instructions, circling, stylus, hovering touch, or visual following.

[0163] In this embodiment, when a user selects part of the text content in the first split screen, the electronic device can sense the selected text content and complete the selected text content into a complete content based on the page content, and display the complete content in the second split screen. In this way, the user does not need to spend a long time accurately selecting interactive text, which can improve the user's work efficiency in handling multiple tasks in the split screen, and thus enhance the user's split screen interaction experience.

[0164] For example, based on the embodiments shown in Figure 4 or Figure 5, Figure 6 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0165] As shown in Figure 6(a), the second screen of the electronic device 400 displays one or more recommended services associated with the text 401 on the first screen. These one or more recommended services may include, for example, text interpretation services, text translation services, and text sharing services.

[0166] As shown in Figure 6(b), when the user selects the text interpretation service, the interpretation result of text 401 is displayed on the second split screen of the electronic device 400.

[0167] In some optional examples, users can select a text interpretation service from one or more recommended services by clicking on the corresponding control, using voice instructions, or other methods.

[0168] In one example, Text 401 could be something like, “In 2023, the company adhered to its strategic focus and fully leveraged its comprehensive advantages in industry portfolio, technological innovation, and complex software and hardware platforms, achieving annual revenue of RMB 734,748 million, a year-on-year increase of 9.6%.” Correspondingly, the interpretation of Text 401 displayed on the second screen could be, “In 2023, the company’s annual revenue was RMB 734,748 million, a year-on-year increase of 9.6%”; furthermore, the interpretation could also be, “In 2023, the company’s annual revenue was RMB 734,748 million, a year-on-year increase of 9.6%. According to public data, among companies of similar size and type, the company’s annual revenue and growth rate are both in the top 20%.”

[0169] For example, Figure 7 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0170] As shown in Figure 7(a), the display screen of the electronic device 700 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays one or more of the following: text 701, table 702 and picture 703. The second split screen displays, for example, a smart assistant page.

[0171] As shown in Figure 7(b), when the user selects table 702 in the first split screen, the second split screen of the electronic device 700 displays the content of table 702, and in an optional example, one or more recommendation services associated with table 702 may also be displayed in the second split screen.

[0172] The one or more recommendation services associated with the table 702 may include, for example, recommendation services associated with the modality type of the table 702, and may also include recommendation services associated with the meaning expressed by the table 702.

[0173] Optionally, the recommended services associated with the modal type of the table 702 may include table file generation services, table interpretation services, data analysis services, table editing services, etc., and may also include other recommended services, such as table saving services, table sharing services, etc., which are not limited in this application.

[0174] Optionally, the recommendation service associated with the meaning expressed in Table 702 may include a currency conversion service, through which users can convert the currency unit of the amounts shown in Table 702 from RMB to other currency units, such as USD. The recommendation service associated with the semantics of Table 702 may also include a comparison chart generation service, through which users can obtain a revenue comparison chart of each business based on the revenue data of each business in each year in Table 702, so as to see the revenue distribution more intuitively.

[0175] In some embodiments, the user may select table 702 in the first split screen by performing a selection operation directly on the first split screen, such as by long-pressing table 702 on the first split screen; by circling table 702 on the first split screen; by directly clicking table 702 on the first split screen; by using voice commands; or by using other methods, such as copying after recognition, multi-finger swipe gestures, stylus, hover touch, or visual tracking. This application does not limit the scope of these methods.

[0176] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. The trigger operation may be a long press operation, a click operation to recognize the control, or other specific gesture operations. After the displayed content in the first split screen is converted into a selectable state, the user can select table 702 by clicking, swiping to select, dragging, voice instructions, circle selection, stylus, hover touch or visual follow.

[0177] In some optional examples, when the user selects the table interpretation service in the second split screen, the display interface of the second split screen can jump to the display interface shown in Figure 7(c), where the interpretation result of table 702 is displayed.

[0178] In some optional examples, users can select the table interpretation service from one or more recommended services by clicking the corresponding control, using voice instructions, or other methods.

[0179] In one example, Table 702 could be:

[0180] Correspondingly, the interpretation result for Table 702 could be, for example:

[0181] 1. Revenue of Business 1 in 2023 was 361,997, and revenue of Business 1 in 2022 was 353,978, representing a 2.3% increase in revenue of Business 1 in 2023 compared to 2022.

[0182] 2. Revenue of Business 2 in 2023 was 251,496, while revenue of Business 2 in 2022 was 214,463, representing a 17.3% increase in revenue of Business 2 in 2023 compared to 2022.

[0183] 3. Revenue of Business 3 in 2023 was 107,894, and revenue of Business 3 in 2022 was 96,148, representing a 12.2% increase in revenue of Business 3 in 2023 compared to 2022.

[0184] 4. Revenue of Business 4 in 2023 was 13,361, and revenue of Business 4 in 2022 was 6,055, representing a 120.7% increase in revenue of Business 4 in 2023 compared to 2022.

[0185] 5. Total revenue in 2023 was 734,748, and total revenue in 2022 was 670,644, representing a 9.6% increase in total revenue in 2023 compared to 2022.

[0186] For example, the interpretation result corresponding to Table 702 could also be:

[0187] In 2023, among all the company's businesses, Business 1 generated the most revenue, accounting for 49.3%, while Business 4 generated the least, accounting for 1.8%. It is encouraging that all businesses in 2023 showed growth compared to 2022, with Business 4 showing the fastest growth at 120.7%. However, it should be noted that Business 1, as the main source of revenue, had the slowest growth, with a growth rate of only 2.3%.

[0188] In some optional examples, when a user selects the generate table file service in the second split screen, the display interface of the second split screen can jump to the table file creation interface. In this table file creation interface, the content of table 702 has been automatically entered into the table file, and the user can directly save the table file without having to manually enter the content of table 702 into the table file.

[0189] For example, Figure 8 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0190] As shown in Figure 8(a), the display screen of the electronic device 800 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays one or more of the following: text 801, table 802 and picture 803. The second split screen displays, for example, a smart assistant page.

[0191] As shown in Figure 8(b), when the user selects image 803 in the first split screen, the second split screen of the electronic device 800 displays the content of image 803. In an optional example, one or more recommendation services associated with image 803 may also be displayed in the second split screen.

[0192] The one or more recommendation services associated with the image 803 may include, for example, recommendation services associated with the modality type of the image 803, and may also include recommendation services associated with the meaning expressed by the image 803.

[0193] Optionally, the recommendation services associated with the modal type of the image 803 may include image interpretation services, text extraction services, image translation services, image search services, etc., and may also include other recommendation services, such as image saving services, image sharing services, image editing services, etc., which are not limited in this application.

[0194] Optionally, the recommendation service associated with the meaning expressed by the image 803 may include a business distribution rationality analysis service. Users can obtain the distribution rationality analysis results of each business in the image 803 through this service, so that users can more intuitively understand the rationality of the distribution of each business. The distribution rationality analysis results of each business in the image 803 may be generated by electronic devices through big data retrieval and analysis, for example, by comparing and analyzing the distribution of business 1, business 2, business 3 and business 4 shown in the image 803 with the business distribution of similar companies.

[0195] In some embodiments, the user can select image 803 in the first split screen by performing a selection operation directly on the first split screen, such as by long-pressing image 803 on the first split screen; by circling image 803 on the first split screen; by directly clicking image 803 on the first split screen; by selecting image 803 via voice command; or by selecting image 803 in other ways, such as by copying after recognition, multi-finger swipe gestures, stylus, hover touch, or visual tracking. This application does not limit the scope of the methods described.

[0196] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. This trigger operation may be, for example, a long press operation, a click operation to recognize the control, or other specific gesture operations. After the displayed content in the first split screen is converted into a selectable state, the user can select the image 803 by clicking, swiping to select, dragging, voice instructions, circling, using a stylus, hovering touch, or visual following.

[0197] In some optional examples, when the user selects the image interpretation service in the second split screen, the display interface of the second split screen can jump to the display interface shown in Figure 8(c), where the interpretation result of image 803 is displayed.

[0198] In some optional examples, users can select an image interpretation service from one or more recommended services by clicking on the corresponding control, using voice instructions, or other methods.

[0199] In one example, the interpretation result corresponding to image 803 could be: "The revenue share of business 1 is 49.3%; the revenue share of business 2 is 34.2%; the revenue share of business 3 is 14.7%; and the revenue share of business 4 is 1.8%."

[0200] In one example, the interpretation of image 803 could be: "Business 1's revenue share is 49.3%; Business 2's revenue share is 34.2%; Business 3's revenue share is 14.7%; and Business 4's revenue share is 1.8%. Among them, Business 1 has the highest revenue share, and Business 4 has the lowest revenue share. The difference between the revenue share of Business 1 and Business 4 is 47.5%. According to public data, the revenue share of our company's Business 4 is lower than that of similar companies. We can consider increasing the development of Business 4 in the next year."

[0201] In some optional examples, when a user selects the text extraction service in the second split screen, the display interface of the second split screen can jump to the text extraction interface corresponding to image 803. In the text extraction interface, the text extracted from image 803 can be displayed as "Business 1 49.3%, Business 2 34.2%, Business 3 14.7%, Business 4 1.8%".

[0202] For example, Figure 9 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0203] As shown in Figure 9(a), the display screen of the electronic device 900 displays a first split screen and a second split screen. The first split screen displays the interface of the first application, and the second split screen displays the interface of the second application. The first application and the second application are different. For example, the first split screen displays the gallery application interface, which displays image 901. The second split screen displays the smart assistant page, for example.

[0204] When a user selects image 901 in the first split screen, image 901 can be displayed on the second split screen of electronic device 900. In an optional example, one or more recommended services associated with image 901 can also be displayed in the second split screen.

[0205] The one or more recommendation services associated with the image 901 may include, for example, recommendation services associated with the modal type of the image 901, and may also include recommendation services associated with the meaning expressed by the image 901.

[0206] Optionally, the one or more recommended services may include image interpretation services, text extraction services, image translation services, money transfer services, etc., and may also include other recommended services, such as image saving services, image sharing services, image search services, etc., which are not limited in this application.

[0207] In some embodiments, the user may select image 901 in the first split screen by performing a selection operation directly on the first split screen, such as by long-pressing image 901 on the first split screen; by circling image 901 on the first split screen; by directly clicking image 901 on the first split screen; by selecting image 901 via voice command; or by selecting image 901 in other ways, such as by copying after recognition, multi-finger swipe gestures, stylus, hover touch, or visual tracking. This application does not limit the scope of the methods described.

[0208] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. This trigger operation may be, for example, a long press operation, a click operation to recognize the control, or other specific gesture operations. After the displayed content in the first split screen is converted into a selectable state, the user can select image 901 by clicking, swiping to select, dragging, voice instructions, circling, stylus, hovering touch, or visual following.

[0209] In some optional examples, when the user selects the text extraction service in the second split screen, the display interface of the second split screen can jump to the display interface shown in Figure 9(b), in which the text extracted from the image 901, "Bank account number: 1234567891234567; Bank: xxxxxx; Payee: Zhang San", can also be displayed in this interface one or more recommended services determined based on the extracted text information, such as transfer service, copy all text information service, etc.

[0210] When a user selects the transfer service in the second split screen, the display interface of the second split screen can jump to the transfer application interface shown in Figure 9(c). In this transfer application interface, the text information extracted from Figure 901 has been automatically filled into the corresponding position. The user only needs to verify the transfer information and enter the transfer amount to directly complete the transfer to the target.

[0211] In another implementation, the user can directly click on the transfer service associated with the meaning expressed by the image 901 in the second split screen shown in Figure 9(a), so that the interface of the second split screen will directly jump to the transfer interface shown in Figure 9(c).

[0212] For example, Figure 10 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0213] As shown in Figure 10(a), the display screen of the electronic device 1000 displays a first split screen and a second split screen. The first split screen displays the interface of the first application, and the second split screen displays the interface of the second application. The first application and the second application are different.

[0214] Taking the first split screen as the interface of a video application and the second split screen as the interface of a notebook application as an example, the first split screen displays the video course playback interface, which displays course content such as "abandon (to abandon, discard, give up), ability (ability; talent), able (capable), abnormal (abnormal, distorted)". The second split screen displays the note-taking interface.

[0215] As shown in Figure 10(b), when a user wants to record the course content "able; capable, abnormal, perverted" displayed on the first split screen as notes, the user can select the course content "able; capable, abnormal, perverted" on the first split screen. In response to the user's selection of the course content "able; capable, abnormal, perverted" on the first split screen, the course content "able; capable, abnormal, perverted" is automatically recorded as notes on the second split screen of the electronic device 1000.

[0216] It should be understood that this application does not limit the way users select course content on the first split screen. For example, they can select by circling, clicking, long-pressing, voice instructions, stylus operation, or visual following.

[0217] In some embodiments, when a user selects interactive content on the interface of a first application displayed on the first split screen of an electronic device, the selected interactive content may not be displayed on the interface of a second application displayed on the second split screen of the electronic device. Instead, a recommendation service associated with the selected interactive content may be directly displayed. Furthermore, in response to the user's selection of a target recommendation service, the target recommendation service is executed on the selected interactive content on the second split screen.

[0218] For example, Figure 11 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0219] As shown in Figure 11(a), the display screen of the electronic device 1100 displays a first split screen and a second split screen. The first split screen displays the interface of the first application, and the second split screen displays the interface of the second application. The first application and the second application are different.

[0220] Taking the first split screen as the interface of the gallery application and the second split screen as the interface of the shopping application as an example, the first split screen displays, for example, image 1101, which includes, for example, an image of the smart speaker 1102, and the second split screen displays, for example, the shopping application page, which may include, for example, sales information of multiple products.

[0221] As shown in Figure 11(b), when a user wants to obtain the sales information of the smart speaker 1102 included in picture 1101, the user can select picture 1101 on the first split screen. In response to the user's operation of selecting picture 1101 on the first split screen, the second split screen of the electronic device 1000 can display the same or similar smart speaker sales information as the smart speaker 1102 included in picture 1101, so that the user can conveniently view the sales information of the smart speaker 1102 included in picture 1101 without the user having to search separately.

[0222] It should be understood that this application does not limit the way users select image 1101 on the first split screen. For example, it can be done by copying, selecting, clicking, long pressing, voice instructions, stylus operation, or visual following.

[0223] For example, Figure 12 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0224] As shown in Figure 12(a), the display screen of the electronic device 1200 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays text 1201, table 1202 and picture 1203, and the second split screen displays a smart assistant page.

[0225] As shown in Figure 12(b), when the user selects table 1202 and picture 1203 in the first split screen, the second split screen of the electronic device 1200 can display the contents of table 1202 and picture 1203.

[0226] Furthermore, one or more recommendation services associated with the table 1202 and the image 1203 can also be displayed in this second split screen.

[0227] The one or more recommendation services associated with the table 1202 and the image 1203 may include, for example, recommendation services associated with the modal type of the table 1202 and the image 1203, and may also include recommendation services associated with the meaning expressed by the table 1202 and the image 1203.

[0228] Optionally, the recommendation services associated with the modal types of the table 1202 and the image 1203 may include comprehensive interpretation services, comprehensive data analysis services, data summarization services, etc., and may also include other recommendation services, such as table and image saving services, table and image sharing services, etc., which are not limited in this application.

[0229] Optionally, the recommendation services associated with the meaning expressed in Table 1202 and Image 1203 may include business comprehensive analysis services, currency conversion services, and / or comparison chart generation services.

[0230] In some embodiments, users can select table 1202 and image 1203 in the first split screen by circling; they can also select table 1202 and image 1203 in other ways, such as by copying after recognition, multi-finger swipe gesture operation, stylus, hover touch, voice instruction or visual following, etc., which are not limited in this application.

[0231] In some optional examples, when the user selects the comprehensive interpretation service in the second split screen, the display interface of the second split screen can jump to the display interface shown in Figure 12(c), in which the comprehensive interpretation results of Table 1202 and Picture 1203 are displayed.

[0232] In some optional examples, users can select the comprehensive table and image interpretation service from one or more recommended services by clicking the corresponding controls, using voice prompts, or other methods.

[0233] In one example, Table 1202 could be:

[0234] Image 1203, for example, could represent that in 2023, Business 1 accounted for 49.3%; Business 2 accounted for 34.2%; Business 3 accounted for 14.7%; and Business 4 accounted for 1.8%.

[0235] Correspondingly, the comprehensive interpretation results for Table 1202 and Image 1203 could be, for example:

[0236] 1. Revenue from all business segments showed a significant increase compared to the previous year.

[0237] Compared to 2022, revenue from all business segments increased in 2023, with total revenue growing by 9.6%. Among them, business segment 4 saw the fastest growth at 120.7%, while business segment 1 experienced the slowest growth at 2.3%.

[0238] 2. The rationality of business distribution needs to be improved.

[0239] In 2023, Business 1 had the highest revenue, accounting for 49.3%, while Business 4 had the lowest revenue, accounting for 1.8%. The difference in revenue share between the two was as high as 47.5%, which is relatively large compared with similar companies.

[0240] 3. Business 4 has significant room for expansion.

[0241] According to publicly available data, among similar companies, the revenue share of business 4 generally increased significantly in 2023, with a considerable number of companies experiencing growth rates exceeding 150%. It is recommended that our company consider further expanding business 4 in the next year, but a risk analysis should be conducted.

[0242] In yet another example, the combined interpretation result corresponding to Table 1202 and Image 1203 could be:

[0243] 1. In 2023, the revenue of Business 1 was 361,997, accounting for 49.3% of the total revenue. In 2022, the revenue of Business 1 was 353,978, and the revenue of Business 1 in 2023 increased by 2.3% compared with 2022.

[0244] 2. In 2023, the revenue of Business 2 was 251,496, accounting for 34.2% of the total revenue. In 2022, the revenue of Business 2 was 214,463, and the revenue of Business 2 in 2023 increased by 17.3% compared with 2022.

[0245] 3. In 2023, the revenue of business segment 3 was 107,894, accounting for 14.7% of the total revenue. In 2022, the revenue of business segment 3 was 96,148, and the revenue of business segment 3 in 2023 increased by 12.2% compared with 2022.

[0246] 4. In 2023, the revenue of business segment 4 was 13,361, accounting for 1.8% of the total revenue. In 2022, the revenue of business segment 4 was 6,055, and the revenue of business segment 4 in 2023 increased by 120.7% compared with 2022.

[0247] 5. Total revenue in 2023 was 734,748, and total revenue in 2022 was 670,644, representing a 9.6% increase in total revenue in 2023 compared to 2022.

[0248] For example, Figure 13 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0249] As shown in Figure 13(a), the display screen of the electronic device 1200 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays text 1201, table 1202 and picture 1203, and the second split screen displays a smart assistant page.

[0250] As shown in Figure 13(b), when the user selects table 1202 and picture 1203 in the first split screen, the second split screen of the electronic device 1200 can display the content of table 1202 and picture 1203.

[0251] Furthermore, one or more recommendation services associated with the table 1202 and the image 1203 can also be displayed in this second split screen.

[0252] In some optional examples, table 1202 displayed in the second split screen is editable. When the user performs an editing operation on table 1202 in the second split screen, the other content displayed in table 1202 and the content displayed in image 1203 can adapt to the changes in the content of table 1202.

[0253] For example, as shown in Figure 13(c), when a user modifies the revenue of Business 4 in Table 1202 for 2023 from "13361" to "20157" in the second split screen, correspondingly, in Table 1202, the growth rate of Business 4 in 2023 relative to 2022 changes from "120.7%" to "232.9%", and the total revenue for 2023 changes from "734748" to "741544". The growth rate of total revenue in 2023 compared to 2022 changed from "9.6%" to "10.6%"; accordingly, in Figure 1203, the revenue share of Business 1 changed from "49.3%" to "48.8%", the revenue share of Business 2 changed from "34.2%" to "33.9%", the revenue share of Business 3 changed from "14.7%" to "14.6%", and the revenue share of Business 4 changed from "1.8%" to "2.7%".

[0254] For example, Figure 14 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0255] As shown in Figure 14(a), the display screen of the electronic device 1400 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays text 1401, table 1402 and picture 1403, and the second split screen displays a smart assistant page.

[0256] As shown in Figure 14(b), when the user selects table 1402 and picture 1403 in the first split screen, the second split screen of the electronic device 1400 can display the contents of table 1402 and picture 1403.

[0257] Furthermore, in this second split screen, one or more recommended services associated with the table 1402 and one or more recommended services associated with the image 1403 can also be displayed.

[0258] The one or more recommendation services associated with the table 1402 may include, for example, recommendation services associated with the modality type of the table 1402, and may also include recommendation services associated with the meaning expressed by the table 1402.

[0259] Optionally, the recommended services associated with the modal type of the table 1402 may include table file generation services, table interpretation services, data analysis services, table editing services, etc., and may also include other recommended services, such as table saving services, table sharing services, etc., which are not limited in this application.

[0260] Optionally, the recommended services associated with the meaning expressed in Table 1402 may include currency conversion services or comparison chart generation services.

[0261] The one or more recommendation services associated with the image 1403 may include, for example, recommendation services associated with the modal type of the image 1403, and may also include recommendation services associated with the meaning expressed by the image 1403.

[0262] Optionally, the recommendation services associated with the modal type of the image 1403 may include image interpretation services, text extraction services, image translation services, image search services, etc., and may also include other recommendation services, such as image saving services, image sharing services, image editing services, etc., which are not limited in this application.

[0263] For example, the recommendation service associated with the meaning expressed by the image 1403 may include business distribution rationality analysis service, etc.

[0264] In some embodiments, users can select table 1402 and image 1403 in the first split screen by circling; they can also select table 1402 and image 1403 in other ways, such as by copying after recognition, multi-finger swipe gesture operation, stylus, hover touch, voice instruction or visual follow, etc., which are not limited in this application.

[0265] It is understood that the embodiments described in Figure 12 and the embodiments shown in Figure 14 can be implemented in combination. That is, when the interactive content selected by the user in the first split screen includes display information of multiple modal types (e.g., including two or more of text, images, tables and links), the recommended services associated with each of the multiple display information can be displayed in the second split screen, and the recommended services associated with the entire multiple display information can also be displayed.

[0266] For example, based on Figure 12(a), when a user selects table 1202 and image 1203 in the first split screen, the second split screen of the electronic device 1200 can display the content of table 1202 and image 1203. In the second split screen, recommendation services associated with table 1202 and image 1203 can also be displayed.

[0267] in,

[0268] Recommendation services associated with the table 1202 and the image 1203 may include, for example, recommendation services associated with the modal type of the table 1202 and the image 1203, such as comprehensive interpretation services, comprehensive data analysis services, and data summarization services; they may also include recommendation services associated with the meaning expressed by the table 1202 and the image 1203, such as comprehensive business analysis services, currency conversion services, and / or comparison chart generation services.

[0269] Recommended services associated with the table 1202 may include, for example, recommended services associated with the modality type of the table 1202, such as table file generation service, table interpretation service, data analysis service, table editing service, etc.; they may also include recommended services associated with the meaning expressed by the table 1202, such as currency conversion service or comparison chart generation service, etc.

[0270] Recommendation services associated with the image 1203 may include, for example, recommendation services associated with the modal type of the image 1203, such as image interpretation services, text extraction services, image translation services, and image search services; they may also include recommendation services associated with the meaning expressed by the image 1203, such as business distribution rationality analysis services.

[0271] It should be understood that the embodiments of this application use text, tables or images as examples of interactive content modal types for illustration, but this does not limit the modal types of interactive content in this application. For example, the modal types of interactive content in this application may also include web links, etc.

[0272] For example, Figure 15 shows a schematic diagram of the interface corresponding to another split-screen interaction method provided in the embodiments of this application.

[0273] As shown in Figure 15(a), the display screen of the electronic device 1500 displays a first split screen and a second split screen. The first split screen displays the interface of a first application, and the second split screen displays the interface of a second application. The first application and the second application are different. For example, the first split screen displays one or more of the following: text 1501, table 1502 and picture 1503. The second split screen displays, for example, a smart assistant page.

[0274] As shown in Figure 15(b), when the user selects table 1502 in the first split screen, the second split screen of the electronic device 1500 displays the contents of table 1502.

[0275] In an optional example, one or more recommended services associated with the table 1502 may also be displayed in this second split screen.

[0276] The explanation of one or more recommendation services associated with Table 1502 is described in detail in the embodiment shown in Figure 7, and will not be repeated here for the sake of brevity.

[0277] For an explanation of how the user selects table 1502 in the first split screen, please refer to the explanation of how the user selects table 702 in the first split screen in the embodiment shown in Figure 7. For the sake of brevity, it will not be repeated here.

[0278] In some optional examples, when a user clicks on the voice control 1504 in the smart assistant interface displayed in the second split screen as shown in Figure 15(b), a voice input box 1505 can be displayed on the second split screen display interface. The voice input box 1505 may include a prompt to guide the user to issue a voice command, such as "Please say your question...".

[0279] When a user issues a voice command based on Table 1502, the electronic device can execute the user's voice command based on Table 1503 and output the execution result of the voice command to the user.

[0280] As shown in Figure 15(c) and Figure 15(d), when the user triggers the voice control 1504 and issues the voice command "Help me interpret this table", the second split screen displays the voice command "Help me interpret this table" and the interpretation result of the table 1502 in the form of a dialogue.

[0281] In some embodiments, the interpretation result of the table 1502 displayed on the second split screen may be generated based on the table 1502; it may also be generated based on the table 1502 and other content displayed on the first split screen (e.g., text 1501 and / or image 1503); or it may be generated based on the table 1502, other content displayed on the first split screen, and / or relevant information in the database.

[0282] In this embodiment, the question-and-answer results obtained through the question-and-answer format can be generated based on the contextual semantics of the content displayed on the first split screen, which makes the accuracy of the question-and-answer results higher.

[0283] In some embodiments, based on what is shown in Figure 15(d), the user can issue further voice commands through the voice control 1504 to enable the user to conduct multi-round voice interaction with the electronic device on the second split screen based on the table 1502.

[0284] In some embodiments, the user may further select image 1503 on the first split screen and issue voice commands based on image 1503 and the previously selected table 1502 to enable multi-round voice interaction between the user and the electronic device on the second split screen based on table 1502 and image 1503.

[0285] For example, Figure 16 shows a schematic flowchart of another split-screen interaction method 1600 provided in an embodiment of this application.

[0286] The method 1600 is applied to an electronic device whose display screen shows a first split-screen interface and a second split-screen interface, wherein the first split-screen interface displays the content of a first application, and the second split-screen interface displays the content of a second application, the first application being different from the second application, as shown in Figure 16. The method 1600 includes:

[0287] S1601: In response to a user's operation in the interactive area of ​​the first split-screen interface, the electronic device acquires the first content displayed in that interactive area.

[0288] S1602: The electronic device identifies the first content based on the modal type corresponding to the first content and obtains the second content associated with the first content.

[0289] In some embodiments, the electronic device establishes a correspondence between the display area and the modal type in the current display interface of the first split-screen interface through controls in the first split-screen interface, or by performing layout analysis on the first split-screen interface, or by performing document parsing on the first split-screen interface; then the electronic device obtains the modal type corresponding to the interactive area based on the correspondence, and thus obtains the modal type corresponding to the first content displayed in the interactive area.

[0290] In some embodiments, the modal type may include one or more of the following: text type, table type, image type, and link type.

[0291] In some examples, the electronic device establishes a correspondence between the display area and modal type in the current display interface of the first split-screen interface by performing layout analysis on the first split-screen interface. For example, this could refer to establishing a correspondence between the coordinate position of each text displayed in the current display interface of the first split-screen interface and the modal type, and / or establishing a correspondence between the coordinate position of each image displayed in the current display interface of the first split-screen interface and the modal type, and / or establishing a correspondence between the coordinate position of each table displayed in the current display interface of the first split-screen interface and the modal type, and / or establishing a correspondence between the coordinate position of each link displayed in the current display interface of the first split-screen interface and the modal type, etc.

[0292] In some embodiments, different modal types correspond to different recognition models. The electronic device can determine the model for recognizing the first content based on the modal type of the first content, and then the electronic device recognizes the first content through the determined model.

[0293] In one example, the modality type corresponding to the first content is text. After acquiring the first content, the electronic device uses a text recognition model to identify the first content in order to obtain the second content. Alternatively, the electronic device can directly obtain the second content through layout analysis or document parsing, without needing to use a model for recognition.

[0294] In one example, the modal type corresponding to the first content is a table. After acquiring the first content, the electronic device recognizes it using the recognition model corresponding to the table to obtain the second content. For example, the electronic device can obtain a cropped image of the selected table from the first split-screen interface, input the cropped image into the recognition model corresponding to the table to obtain the structure of the selected table and the text content within the cells, and represent it in HTML format to obtain the second content.

[0295] In one example, the modality of the first content is an image. After acquiring the first content, the electronic device recognizes it using the image recognition model to obtain the second content. For instance, the electronic device can call an optical character recognition (OCR) module to obtain text, and then call a vision-language model (VLM) or a large multimodal model (LMM) to obtain the image-related content and description to obtain the second content, which is then displayed on the second split-screen interface.

[0296] In one example, the modal types corresponding to the first content are table and image. After the electronic device acquires the first content, it recognizes the first content using the recognition model corresponding to the table and the recognition model corresponding to the image.

[0297] In some embodiments, the electronic device may obtain the first content associated with the interactive area through a recognition control in the first split-screen interface, through layout analysis of the first split-screen interface, or through document parsing of the first split-screen interface.

[0298] S1603: The electronic device generates a recommendation service associated with the first content based on the first content and / or the second content.

[0299] The explanation of the recommendation service associated with the first content is given in the embodiment shown in Figure 3, and will not be repeated here for the sake of brevity.

[0300] In one example, the first content includes first text and first image. Correspondingly, the acquired second content includes first text and first image. The modal types corresponding to the interactive area include text type and image type. After acquiring the first content, the electronic device recognizes the first text using the model corresponding to the text type and recognizes the first image using the model corresponding to the image type. The electronic device generates a recommendation service associated with the first text based on the recognition results of the first text and / or the first text, and generates a recommendation result associated with the first image based on the recognition results of the first image and / or the first image; or the electronic device generates a recommendation service associated with the first text and the first image based on the recognition results of the first text and the first image.

[0301] S1604: The electronic device displays second content on the second split-screen interface, and / or the electronic device displays a recommended service corresponding to the first content on the second split-screen interface.

[0302] For an explanation of this step, please refer to the explanation of S301 in the embodiment shown in Figure 3. For the sake of brevity, it will not be repeated here.

[0303] For example, FIG17 shows a schematic architecture diagram of a split-screen interaction device 1700 provided in an embodiment of the present application.

[0304] As shown in Figure 17, the device 1700 includes a sensing module 1710, multiple identification modules (e.g., a first identification module 1720, a second identification module 1730, a third identification module 1740, and a fourth identification module 1750), and an intelligent analysis module 1760. The sensing module 1710 may be located, for example, in the sensing layer of the electronic device, and the multiple identification modules and the intelligent analysis module 1760 may be located, for example, in the processing layer of the electronic device.

[0305] The sensing module 1710 is used to respond to the user's operation in the interactive area of ​​the first split-screen interface and obtain the first content displayed in the interactive area.

[0306] The perception module 1710 is also used to send the acquired first content to the corresponding recognition module.

[0307] The perception module 1710 is also used to send the acquired first content to the intelligent analysis module 1760.

[0308] The identification module is used to receive the first content sent by the perception module 1710, and is also used to obtain, by identifying the first content, the second content associated with the first content.

[0309] In one example, the recognition module may include a first recognition module 1720, a second recognition module 1730, a third recognition module 1740, and a fourth recognition module 1750, wherein the first recognition module 1720 is used to recognize interactive content with a text modality, the second recognition module 1730 is used to recognize interactive content with a table modality, the third recognition module 1740 is used to recognize interactive content with an image modality, and the fourth recognition module 1750 is used to recognize interactive content with a link modality.

[0310] The device 1700 may further include a background processing module, wherein the sensing module 1710 sends the first content to the background processing module, and the background processing module distributes the first content to the corresponding recognition module according to the modal type corresponding to the first content; or, the sensing module 1710 directly distributes the first content to the corresponding recognition module according to the modal type corresponding to the first content.

[0311] In one example, the modal type corresponding to the first content is a table. Accordingly, the first content is table content. The perception module 1710 sends the first content to the recognition module corresponding to the table through the background processing module. Alternatively, the perception module 1710 directly sends the first content to the recognition module corresponding to the table based on the modal type corresponding to the first content.

[0312] The intelligent analysis module 1760 is used to receive the first content sent by the sensing module 1710.

[0313] In some embodiments, when the modal type corresponding to the first content is text, the perception module 1710 can directly send the first content to the intelligent analysis module 1760.

[0314] The intelligent analysis module 1760 is also used to receive the second content sent by the recognition module.

[0315] The intelligent analysis module 1760 is also used to generate recommendation services associated with the first content based on the first content and / or the first and second content.

[0316] The device 1700 may also include a display module for displaying second content and / or displaying recommended services associated with the first content on a second split-screen interface of the electronic device.

[0317] One or more modules or units described herein can be implemented in software, hardware, or a combination of both. When any of the above modules or units are implemented in software, the software exists as computer program instructions and is stored in memory. A processor can be used to execute the program instructions and implement the above method flow. The processor can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., and various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor can be built into a SoC (System-on-a-Chip) or an application-specific integrated circuit (ASIC), or it can be a separate semiconductor chip. In addition to the cores for executing software instructions to perform calculations or processing, the processor may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0318] When the modules or units described herein are implemented in hardware, the hardware may be any one or any combination of a CPU, microprocessor, DSP, MCU, artificial intelligence processor, ASIC, SoC, FPGA, PLD, application-specific digital circuit, hardware accelerator, or non-integrated discrete device, which may run the necessary software or perform the above method flow independently of software.

[0319] When the modules or units described herein are implemented using software, they can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0320] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0321] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0322] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0323] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0324] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0325] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0326] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for split-screen interaction, the method comprising: The method is applied to an electronic device, and the method includes: The system displays a first split-screen interface and a second split-screen interface. The first split-screen interface displays the content of a first application, and the second split-screen interface displays the content of a second application. The first application is different from the second application. In response to a user's operation in the interactive area of ​​the first split-screen interface, the electronic device displays second content associated with the first content displayed in the interactive area on the second split-screen interface, and / or, the electronic device displays recommended services associated with the first content displayed in the interactive area on the second split-screen interface.

2. The method of claim 1, wherein, The second content associated with the first content includes: Content obtained by copying the first content, content obtained by completing the first content, or content obtained by converting the format of the first content.

3. The method according to claim 1 or 2, characterized in that, The recommendation service associated with the first content includes: Recommendation services associated with the modality type of the first content and / or recommendation services associated with the meaning of the first content, wherein the modality type is used to indicate the carrying form and / or expression form of the first content.

4. The method according to claim 3, characterized in that, When the modal type of the first content includes text, the recommendation service associated with the modal type of the first content includes one or more of the following: text interpretation service, text editing service, text translation service, and text sharing service; and / or When the modal type of the first content includes a table type, the recommendation service associated with the modal type of the first content includes one or more of the following: table file generation service, table editing service, table interpretation service, and data analysis service; and / or When the modal type of the first content includes an image, the recommendation service associated with the modal type of the first content includes one or more of the following: image interpretation service, image editing service, text extraction service, image translation service, and image search service.

5. The method according to claim 3 or 4, characterized in that, When the modal type of the first content includes multiple modal types, the recommendation service associated with the modal type of the first content includes recommendation services associated with at least one of the multiple modal types.

6. The method according to any one of claims 1 to 5, characterized in that, When the modal type of the first content includes a first modal type and a second modal type, the method further includes: In response to a user's editing operation on the content corresponding to the first modal type in the second split-screen interface, the electronic device synchronously updates the content displayed on the second split-screen interface corresponding to the second modal type according to the edited content.

7. The method according to any one of claims 1 to 6, characterized in that, The second content is the content obtained by the electronic device after performing sentence completion and / or paragraph completion on the first content.

8. The method according to any one of claims 1 to 7, characterized in that, The second split-screen interface also displays a voice control, and the method further includes: In response to a user's touch operation on the voice control, the electronic device acquires a first voice command; The electronic device displays the first result on the second split-screen interface based on the first voice command and the first content.

9. The method of claim 8, wherein, The electronic device displays a first result on the second split-screen interface based on the first voice command and the first content, including: The electronic device displays the first result on the second split-screen interface based on the content displayed on the first split-screen interface, the first voice command, and the first content.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: By using the controls in the first split-screen interface, or by performing layout analysis on the first split-screen interface, or by performing document parsing on the first split-screen interface, a correspondence between the display area and modal type in the current display interface of the first split-screen interface is established. Based on the correspondence, the modal type corresponding to the first content displayed in the interactive area is obtained.

11. An electronic device, comprising: include: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 10.

13. A chip, characterized by The chip stores instructions that, when executed, implement the method as described in any one of claims 1 to 10.

14. A computer program product, characterised in that, The computer program product stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 10.

Images