Split-screen interaction method and electronic equipment

Abstract

The embodiment of the invention provides a split-screen interaction method and electronic equipment, the method is applied to the electronic equipment, the method comprises the steps that a first split-screen interface and a second split-screen interface are displayed, the first split-screen interface displays the content of a first application, the second split-screen interface displays the content of a second application, and the first application is different from the second application; in response to operation of a user in the interaction area of the first split-screen interface, the electronic equipment displays second content associated with first content displayed in the interaction area on the second split-screen interface, and / or the electronic equipment displays recommendation service associated with the first content displayed in the interaction area on the second split-screen interface. Through the method or the electronic equipment, the interactive content selected by a user in one split screen can be perceived by another split screen, and the interactive content can be responded in the other split screen, so that the efficiency of processing multiple tasks by the user in a split screen manner can be improved, and the split screen use experience of the user is improved.

Description

[0001] This application is a divisional application. The application number of the original application is 202411456967.0, and the original application date is October 17, 2024. The entire content of the original application is incorporated into this application by reference. Technical Field

[0002] Embodiments of the present application relate to the field of split-screen interaction, and more specifically, to a split-screen interaction method and electronic device. Background Art

[0003] With the development of intelligent devices, the split-screen function of foldable screen devices and large-screen devices has gradually become a commonly used function for users in learning, office and other scenarios. Through the split-screen function of the device, users can handle multiple tasks at the same time, significantly improving learning and work efficiency.

[0004] However, in the current split-screen solution, the interactive experience between multiple split screens needs to be improved. Summary of the Invention

[0005] The present application provides a split-screen interaction method and electronic device, through which the content selected by a user in one split-screen application can be perceived by another split-screen application, and the user can respond to the content in the interface of the other split-screen application, which can improve the user's efficiency in split-screen multitasking and enhance the user's split-screen usage experience.

[0006] In a first aspect, a split-screen interaction method is provided, which is applied to an electronic device, and the method includes: displaying a first split-screen interface and a second split-screen interface, the first split-screen interface displaying the content of a first application, and the second split-screen interface displaying the content of a second application, the first application being different from the second application; in response to a user's operation in an interaction area of ​​the first split-screen interface, the electronic device displays second content associated with the first content displayed in the interaction area on the second split-screen interface, and / or the electronic device displays recommended services associated with the first content displayed in the interaction area on the second split-screen interface.

[0007] In an embodiment of the present application, the content selected by a user in a split-screen application of an electronic device can be perceived by another split-screen application, and a response can be made to the content in the interface of the other split-screen application. For example, the selected content and / or recommended services associated with the selected content can be displayed in the interface of the other split-screen application, which can improve the efficiency of the user's split-screen multi-tasking and enhance the user's split-screen usage experience.

[0008] In combination with the first aspect, in a possible implementation, the second content associated with the first content includes: content obtained by copying the first content, content obtained by completing the first content, or content obtained by formatting the first content.

[0009] In an embodiment of the present application, the content displayed in the interactive area of ​​the first split-screen interface of the electronic device may be different from the related content displayed in the second split-screen interface in terms of text completeness and / or format. For example, when the content displayed in the interactive area is incomplete, the electronic device may display the completed content displayed in the interactive area on the second split-screen interface. In this way, the user does not need to spend a long time to accurately select the interactive text. For another example, when an uneditable table is displayed in the interactive area, the electronic device may display an editable table corresponding to the uneditable table on the second split-screen interface, which can improve the user's work efficiency in handling multiple tasks in split-screen, and thereby enhance the user's split-screen interactive experience.

[0010] In combination with the first aspect, in one possible implementation, the recommendation service associated with the first content includes: a recommendation service associated with the modal type of the first content and / or a recommendation service associated with the meaning of the first content, the modal type being used to indicate the carrier form and / or expression form of the first content.

[0011] Exemplarily, the carrying form may include sound, document, vibration, control, video, picture, photo, etc. For example, the first content is carried in the form of a document (e.g., further divided into multiple subtypes such as .txt, .doc, .xlsx, .pdf, etc.), the first content is carried in the form of a picture (e.g., further divided into subtypes such as .jpg, .png, etc.), etc.

[0012] The same carrier form may include multiple forms of expression. For example, a .doc document may include links and pictures. The content expressed by the links and the content expressed by the pictures in the document belong to different modal types.

[0013] Content can also be carried in different forms in the same form. For example, a table can be carried in an .xlsx document, a .pdf document, and a .doc document. Then the table information carried in these different ways belongs to different modal types.

[0014] In an embodiment of the present application, cross-screen perception of display information of multiple modal types can be achieved between the split-screen interfaces of different applications displayed on the electronic device. This enables users to achieve cross-screen interaction of display information of multiple modal types between the split-screen interfaces of different applications, which can further improve the efficiency of users' split-screen multi-tasking and enhance the user's split-screen usage experience.

[0015] In addition, in the second split-screen interface, recommended services associated with the modal type of the content displayed in the interactive area can be recommended to the user, allowing the user to quickly perform some general services on the content displayed in the interactive area in the second split-screen interface; recommended services associated with the meaning of the content displayed in the interactive area can also be recommended to the user, allowing the user to quickly perform some targeted services on the content displayed in the interactive area in the second split-screen interface.

[0016] In combination with the first aspect, in one possible implementation, when the modal type of the first content includes a text type, the recommended services associated with the modal type of the first content include one or more of a text interpretation service, a text editing service, a text translation service, and a text sharing service; and / or when the modal type of the first content includes a table type, the recommended services associated with the modal type of the first content include one or more of a table file generation service, a table editing service, a table interpretation service, and a data analysis service; and / or when the modal type of the first content includes a picture type, the recommended services associated with the modal type of the first content include one or more of a picture interpretation service, a picture editing service, a text extraction service, a picture translation service, and a picture search service.

[0017] In combination with the first aspect, in one possible implementation, when the modality type of the first content includes multiple modality types, the recommendation service associated with the modality type of the first content includes a recommendation service associated with at least one of the multiple modality types.

[0018] In an embodiment of the present application, when the content displayed in the interactive area includes multiple modal types, in the second split-screen interface, recommended services associated with the multiple modal types of content displayed in the interactive area can be recommended to the user, and the user can quickly perform some comprehensive services on the content displayed in the interactive area in the second split-screen interface, such as data aggregation services, comprehensive analysis services, etc.

[0019] In combination with the first aspect, in a possible implementation, when the modal type of the first content includes a first modal type and a second modal type, the method further includes: in response to the user editing the content corresponding to the first modal type in the second split-screen interface, the electronic device synchronously updates the content corresponding to the second modal type displayed in the second split-screen interface according to the content corresponding to the edit.

[0020] In an embodiment of the present application, when the content displayed in the interactive area includes multiple modal types, the associated second content displayed in the second split-screen interface also corresponds to multiple modal types, wherein the second content can be editable. When the user edits the content corresponding to one modal type among the multiple modal types displayed on the second split-screen interface, correspondingly, the content associated with the edited content in the content corresponding to other modal types will also be adaptively updated, which can facilitate users to quickly organize data.

[0021] In combination with the first aspect, in a possible implementation manner, the second content is the content after the electronic device performs sentence completion and / or paragraph completion on the first content.

[0022] In an embodiment of the present application, when the content displayed in the interactive area is incomplete, the related content displayed by the electronic device on the second split-screen interface may be the content that completes the content displayed in the interactive area. In this way, the user does not need to spend a long time to accurately select the interactive text, which can improve the user's work efficiency in split-screen multi-tasking, and thus enhance the user's split-screen interactive experience.

[0023] In combination with the first aspect, in a possible implementation, a voice control is also displayed on the second split-screen interface, and the method further includes: in response to the user's touch operation on the voice control, the electronic device obtains a first voice instruction; the electronic device displays a first result on the second split-screen interface based on the first voice instruction and the first content.

[0024] In an embodiment of the present application, a user can ask questions and answer questions on the second split-screen interface based on the content displayed in the interactive area on the first split-screen interface. Correspondingly, the question and answer results generated based on the content displayed in the interactive area are displayed on the second split-screen interface, which can improve the user's work efficiency in handling multiple tasks in split-screen, and thus enhance the user's split-screen interactive experience.

[0025] In combination with the first aspect, in a possible implementation method, the electronic device displays the first result on the second split-screen interface based on the first voice instruction and the first content, including: the electronic device displays the first result on the second split-screen interface based on the content displayed on the first split-screen interface, the first voice instruction and the first content.

[0026] In an embodiment of the present application, users can ask questions and answer on the second split-screen interface based on the content displayed in the interactive area on the first split-screen interface. Correspondingly, the second split-screen interface displays the question and answer results generated based on the content displayed in the interactive area and the content displayed on the first split-screen interface. In this embodiment, the question and answer results obtained in the form of questions and answers can be generated based on the contextual semantics of the content displayed on the first split-screen interface, which makes the obtained question and answer results more accurate.

[0027] In combination with the first aspect, in a possible implementation, the method also includes: establishing a correspondence between the display area and the modal type in the current display interface of the first split-screen interface through the controls in the first split-screen interface, or by performing layout analysis on the first split-screen interface, or by performing document parsing on the first split-screen interface; and obtaining the modal type corresponding to the first content displayed in the interactive area according to the correspondence.

[0028] In combination with the first aspect, in a possible implementation, in response to the user's operation in the interaction area of ​​the first split-screen interface, the electronic device displays the second content associated with the first content displayed in the interaction area on the second split-screen interface, and / or the electronic device displays the recommended service associated with the first content displayed in the interaction area on the second split-screen interface, including: in response to the user's operation in the interaction area of ​​the first split-screen interface, the electronic device obtains the first content displayed in the interaction area; the electronic device identifies the first content according to the modal type corresponding to the first content to obtain the second content; the electronic device generates the recommended service associated with the first content based on the first content and / or the second content; the electronic device displays the second content on the second split-screen interface, and / or the electronic device displays the recommended service associated with the first content on the second split-screen interface.

[0029] In a second aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0030] In a third aspect, a computer-readable storage medium is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the method in the above-mentioned first aspect or any possible implementation of the first aspect is implemented.

[0031] In a fourth aspect, a chip is provided, in which instructions are stored. When the chip is run on a device, the chip executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0032] In a fifth aspect, a computer program product is provided, in which a computer program or instruction is stored. When the computer program or instruction is executed, the method in the above-mentioned first aspect or any possible implementation of the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0034] Figure 2 is a software structure block diagram of the electronic device provided in an embodiment of the present application;

[0035] Figure 3 This is a schematic flowchart of a split-screen interaction method provided by an embodiment of the present application;

[0036] Figure 4 This is a schematic diagram of an interface corresponding to a split-screen interaction method provided in an embodiment of the present application;

[0037] Figure 5 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0038] Figure 6 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0039] Figure 7 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0040] Figure 8 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0041] Figure 9 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0042] Figure 10 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0043] Figure 11 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0044] Figure 12 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0045] Figure 13 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0046] Figure 14 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0047] Figure 15 This is a schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application;

[0048] Figure 16 This is a schematic flow chart of another split-screen interaction method provided in an embodiment of the present application;

[0049] Figure 17 This is a schematic architecture diagram of a split-screen interaction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions of this application will be described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, rather than all the embodiments.

[0051] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" or "multiple" refers to two or more than two.

[0052] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0053] The terms used in the following embodiments are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of this application and the appended claims, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0054] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "one embodiment," "some embodiments," "another embodiment," and "other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically stated. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically stated.

[0055] The method provided in the embodiments of the present application can be applied to electronic devices such as mobile phones, tablet computers, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0056] For example, Figure 1: The figure shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0057] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0058] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0059] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0060] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0061] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0062] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0063] The charging management module 140 is configured to receive charging input from a charger. While charging the battery 142 , the charging management module 140 can also provide power to the electronic device through the power management module 141 .

[0064] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device. The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor.

[0065] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0066] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0067] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0068] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0069] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0070] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an App required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0071] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0072] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power level changes, messages, missed calls, notifications, etc.

[0073] The SIM card interface 195 is used to connect a SIM card. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications.

[0074] It should be understood that the phone card in the embodiments of the present application includes but is not limited to a SIM card, an eSIM card, a universal subscriber identity module (USIM), a universal integrated circuit card (UICC), and the like.

[0075] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0076] Figure 2 This is a block diagram of the software structure of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0077] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0078] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0079] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0080] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0081] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0082] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0083] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0084] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0085] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0086] Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

[0087] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0088] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0089] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0090] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0091] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0092] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0093] A 2D graphics engine is a drawing engine for 2D drawings.

[0094] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0095] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Hongmeng and other systems.

[0096] The technical solutions of the embodiments of the present application can be applied to electronic devices with display capabilities. For example, they can be applied to televisions, desktop computers, laptop computers, and in-vehicle screens. They can also be applied to portable electronic devices such as mobile phones, foldable screens, tablet computers, cameras, video cameras, video recorders, watches, and bracelets. They can also be applied to other electronic devices with display capabilities. They can also be applied to electronic devices in future networks or electronic devices in future-evolved public land mobile networks (PLMNs). The main application scenario can be split-screen interaction scenarios of electronic devices.

[0097] As smart devices become more and more popular, foldable screen devices and large-screen devices have gradually become commonly used devices for users in learning, office and other scenarios. Among them, foldable screen devices and large-screen devices have larger display space, and their split-screen display function is frequently used by users. Through the split-screen display function of foldable screen devices and large-screen devices, users can handle multiple tasks at the same time, which can significantly improve users' learning and work efficiency.

[0098] After the screen of an electronic device is split, the multiple split screens are independent operation areas. The user's operation on one split screen will only be responded to on that split screen. However, when the user uses the electronic device in split screen, the multiple processing tasks opened simultaneously through multiple split screens are often related. Therefore, the interactive experience between multiple split screens has become an issue that users are increasingly concerned about.

[0099] In some embodiments, taking the two split-screen interfaces displayed by the electronic device as the gallery application interface and the shopping application interface, the user needs to manually fill in the address information in the gallery application interface into the shopping application interface in order to save the address as the delivery address of the shopping application. For example, the user can drag the address information identified in the gallery application interface across the screen to the shopping application interface to fill in the address information identified in the gallery application interface into the new delivery address page of the shopping application. For another example, the user can also copy the identified address information through the gallery application interface and then paste the address information to the corresponding position in the new delivery address page of the shopping application; for another example, the user can manually enter the address information into the corresponding position in the new delivery address page of the shopping application by comparing the address information in the gallery application interface.

[0100] Among them, users can obtain text from the text source interface by copying or long pressing to select, and copy the text obtained in one application interface to another application interface by pasting or dragging across the screen. Moreover, they can only manually interact with the text content in the interface.

[0101] Therefore, in the current split-screen interaction method, the flow of information and data between split screens requires manual operation by the user. The device cannot perceive and synchronize the user's interactive behavior across screens. As a result, when the user handles multiple tasks in split-screen mode, it is impossible to conveniently synchronize relevant information across screens, nor is it possible to conveniently perform related operations across screens. This increases the complexity of the user's split-screen operation and affects the user experience. For example, when using a smart assistant application on one split screen to ask questions about a document displayed on another split screen, the user needs to manually enter the document information displayed on the other split screen into the interface of the smart assistant application before they can ask questions based on the document information.

[0102] In view of this, an embodiment of the present application provides a split-screen interaction method and electronic device, through which cross-screen perception between multiple split screens can be achieved, and the user's interactive content in one split screen can be perceived by another split screen, and the user can respond to the interactive content in the other split screen. For example, the interactive content can be displayed in the other split screen and / or services related to the interactive content can be actively recommended to the user in the other split screen, which can improve the user's efficiency in handling multiple tasks in the split screen and enhance the user's split-screen usage experience.

[0103] In addition, in the method provided in the embodiment of the present application, the device can obtain the modal type of display information in different areas of the interactive screen. When the user selects interactive content in one split screen, the device can perform targeted identification of the interactive content according to the modal type corresponding to the interactive content, and can recommend related services to the user on another split screen based on the interactive content and the modal type corresponding to the interactive content. For example, when the interactive content selected by the user in one split screen is a table, a recommended service related to the table can be provided to the user on another split screen; when the interactive content selected by the user in one split screen is text, a recommended service related to the text can be provided to the user on another split screen; when the interactive content selected by the user in one split screen is an image, a recommended service related to the image can be provided to the user on another split screen; when the interactive content selected by the user in one split screen includes multiple modal types, a recommended service associated with at least one of the multiple modal types can be provided to the user on another split screen. In this way, it is possible to realize interaction between multiple split screens based on content of multiple modal types, thereby providing users with more accurate and diversified services and further improving the user's split screen usage experience.

[0104] Among them, the modal type may include, for example, text type, picture type, table type, control type, link type, etc., and may also include content of other modalities, which is not limited in this application.

[0105] It should be understood that the modal type described in the embodiments of the present application may refer to the form in which information is carried, and the form in which information is carried may include, for example, sound, document, vibration, control, video, picture, photo, etc. For example, information is carried in the form of a document (for example, further divided into multiple subtypes such as .txt, .doc, .xlsx, .pdf), and information is carried in the form of a picture (for example, further divided into subtypes such as .jpg, .png, etc.). The modal type may also refer to the form in which information is expressed. For example, in the same form of expression, multiple forms of information may be included. For example, in a .doc document, links may be included, and pictures may be included. The information expressed by the links and the information expressed by the pictures in the document belong to different modal types. Information may also be carried in different forms of expression in the same form. For example, a table may be carried in a .xlsx document, a .pdf document, or a .doc document. Then, the table information carried in these different ways belongs to different modal types.

[0106] In an optional example, a split-screen interface of an electronic device displays picture 1, which includes table 1, picture 2, and link 1. This can be understood as including information of three modal types in the split-screen interface, namely, the table type corresponding to table 1, the picture type corresponding to picture 2, and the link type corresponding to link 1. In other words, when the content displayed on the split-screen interface corresponds to one carrier form, and when these contents correspond to multiple forms of expression, it can also be understood that the content displayed on the split-screen interface is display content of multiple modal types.

[0107] In another optional example, when Table 1 is carried in the form of a document on the split-screen interface, the presentation form of Table 1 is a table, and the modal type corresponding to Table 1 is a table type; when Table 1 is carried in the form of a picture on the split-screen interface, the presentation form of Table 1 is a table, and the modal type corresponding to Table 1 is a table type; when Table 1 is carried in the form of a photo on the split-screen interface, the presentation form of Table 1 is a table, and the modal type corresponding to Table 1 is a table type.

[0108] For example, Figure 3 A schematic flowchart of a split-screen interaction method 300 provided in an embodiment of the present application is shown.

[0109] The method 300 is applied to an electronic device, the display screen of the electronic device displays a first split screen interface and a second split screen interface, wherein the first split screen interface displays the content of a first application, and the second split screen interface displays the content of a second application, and the first application is different from the second application. Figure 3 As shown, the method 300 includes:

[0110] S301: In response to a user's operation in an interactive area of ​​a first split-screen interface of an electronic device, the electronic device displays second content associated with the first content displayed in the interactive area on a second split-screen interface, and / or the electronic device displays recommended services associated with the first content displayed in the interactive area on the second split-screen interface.

[0111] The electronic device displays second content on the second split-screen interface that is associated with the first content displayed in the interactive area. The relationship between the first content and the second content may include any one of the following relationships:

[0112] (1) The second content is obtained by copying the first content.

[0113] In an example, the first content may be the entire first sentence, and correspondingly, the second content may be the entire first sentence.

[0114] In some examples, the first content may be, for example, the entire first paragraph, and correspondingly, the second content may be the entire first paragraph.

[0115] In some examples, correspondingly, the user's operation in the interactive area of ​​the first split-screen interface can be, for example, an operation of triggering recognition of the content displayed on the first split-screen interface and then copying the first content in the recognized content, or an operation of selecting the first content by clicking, long pressing, voice command, circling, etc.

[0116] (2) The first content is copied and the copied first content is completed according to the content displayed in the first split-screen interface to obtain the second content.

[0117] In some examples, the first content may be, for example, a portion of a first sentence, and correspondingly, the second content may be the entire first sentence.

[0118] In some examples, the first content may be, for example, a portion of a first paragraph, and correspondingly, the second content may be the entire first paragraph.

[0119] (3) The second content is obtained by identifying the first content and converting the format of the identified first content.

[0120] In an example, the first content may be a table carried in the form of an image, and the second content may be a table in HTML format obtained by recognizing the image, identifying the table in the image, and converting the format of the table.

[0121] In one example, the first content may be a picture, for example. The picture may be identified and converted into a format that supports display on the second split screen or into an editable format. Correspondingly, the second content is the converted picture.

[0122] (4) The second content is obtained by identifying the first content and copying the identified first content.

[0123] In one example, the first content may be a picture, which is used to represent the first address information. The first content may be identified and copied. Correspondingly, the second content may be the first address information presented in editable text form.

[0124] In some embodiments, the second content associated with the first content includes: content obtained by copying the first content, content obtained by completing the first content, and / or content obtained by formatting the first content.

[0125] The first content may correspond to one or more modal types, wherein the modal type includes, for example, a text type, a picture type, a table type, or a link type.

[0126] The first content may include, for example, one or more of text, pictures, tables, and links. In addition, the first content may also include content of other modal types, which is not limited in this application.

[0127] In some embodiments, the recommendation service associated with the first content may include a recommendation service associated with a modality type of the first content and / or a recommendation service associated with a meaning of the first content, wherein the modality type is used to indicate a carrier form and / or expression form of the first content.

[0128] Exemplarily, the carrying form may include sound, document, vibration, control, video, picture, photo, etc. For example, the first content is carried in the form of a document (e.g., further divided into multiple subtypes such as .txt, .doc, .xlsx, .pdf, etc.), the first content is carried in the form of a picture (e.g., further divided into subtypes such as .jpg, .png, etc.), etc.

[0129] The same carrier form may include multiple forms of expression. For example, a .doc document may include links and pictures. The content expressed by the links and the content expressed by the pictures in the document belong to different modal types.

[0130] Content can also be carried in different forms in the same form. For example, a table can be carried in an .xlsx document, a .pdf document, and a .doc document. Then the table information carried in these different ways belongs to different modal types.

[0131] In an optional example, when the modality type of the first content includes a text type, the recommended services associated with the modality type of the first content may include one or more of a text interpretation service, a text editing service, a text translation service, and a text sharing service.

[0132] In an optional example, when the modal type of the first content includes a table type, the recommended services associated with the modal type of the first content include one or more of a table file generation service, a table editing service, a table interpretation service, and a data analysis service.

[0133] In an optional example, when the modal type of the first content includes a picture type, the recommended services associated with the modal type of the first content include one or more of a picture interpretation service, a picture editing service, a text extraction service, a picture translation service, and a picture search service.

[0134] In an optional example, when the first content includes address information, the recommended service associated with the meaning of the first content may include a navigation service, a route viewing service, and the like.

[0135] In an optional example, when the first content includes account information, the recommended services associated with the meaning of the first content may include a transfer service, a saving service, and the like.

[0136] In an optional example, when the first content includes flight information, the recommended services associated with the meaning of the first content may include a flight booking service, a flight ticket inquiry service, and the like.

[0137] In some embodiments, when the modality type of the first content includes a plurality of modality types, the recommendation service associated with the modality type of the first content includes a recommendation service associated with at least one of the plurality of modality types.

[0138] In an optional example, when the modality type of the first content includes a text type and a table type, the recommendation service associated with the modality type of the first content may include one or more of a comprehensive interpretation service, a comprehensive analysis service, and a data aggregation service.

[0139] In some embodiments, when the modal type of the first content includes a first modal type and a second modal type, the method 300 may further include: in response to the user editing the content corresponding to the first modal type in the second split-screen interface, the electronic device synchronously updates the content corresponding to the second modal type displayed in the second split-screen interface according to the content corresponding to the edit.

[0140] In an optional example, when the first content includes a first table and a first picture, the second split-screen interface of the electronic device can display the first table in an editable state. When the user modifies a certain data in the first table, correspondingly, other data associated with the data in the table are also adaptively updated, and the content associated with the data displayed in the first picture is also adaptively updated.

[0141] In some embodiments, the second content may be content obtained by the electronic device performing sentence completion and / or paragraph completion on the first content.

[0142] In some embodiments, a voice control may also be displayed on the second split-screen interface, and the method 300 may further include: in response to a user's touch operation on the voice control, the electronic device obtains a first voice instruction; and the electronic device displays a first result on the second split-screen interface based on the first voice instruction and the first content.

[0143] In some embodiments, the electronic device displays the first result on the second split-screen interface based on the first voice instruction and the first content, including: the electronic device displays the first result on the second split-screen interface based on the content displayed on the first split-screen interface, the first voice instruction and the first content.

[0144] The user's touch operation on the voice control may be a click operation, a long press operation, a sliding operation, etc.

[0145] For example, Figure 4 A schematic diagram of an interface corresponding to a split-screen interaction method provided in an embodiment of the present application is shown.

[0146] like Figure 4 As shown in (a), the display screen of the electronic device 400 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, one or more of text 401, table 402 and picture 403, and the second split screen displays, for example, a smart assistant page.

[0147] like Figure 4As shown in (b), when the user selects the entire content of the text 401 in the first split screen, the second split screen of the electronic device 400 displays the entire content of the text 401, and, in an optional example, one or more recommended services associated with the text 401 can also be displayed in the second split screen.

[0148] The one or more recommendation services associated with the text 401 may include, for example, a recommendation service associated with the modality type of the text 401 , and may also include a recommendation service associated with the semantics of the text 401 .

[0149] For example, optionally, the recommended services associated with the modality type of the text 401 may include text interpretation services, text translation services, text sharing services, etc., and may also include other recommended services, such as text preservation services, etc., which are not limited in this application.

[0150] For example, the recommended service associated with the semantics of the text 401 may include a currency conversion service, through which the user can convert the content "734,748 million yuan" in the text 401 into other monetary units, such as US dollars.

[0151] In some optional examples, when the user selects the text sharing service in the second split screen, the display interface of the second split screen can jump to the display interface of sharing text 401 with friends. After the user selects the friend to share with in the display interface, the electronic device 400 can send the content of text 401 to the friend.

[0152] In some embodiments, the user selects text 401 in the first split screen by directly performing a selection operation on the first split screen, for example, the user selects text 401 by long pressing and selecting text content in the first split screen; the user may select text 401 by circling text content in the first split screen; the user may select text 401 by directly clicking on the text content in the first split screen; the user may select text 401 through voice instructions; the user may select text 401 in other ways, for example, selecting text through a multi-finger sliding gesture, selecting text through a stylus, selecting text through floating touch, selecting text through visual scanning, etc., and this application does not limit this.

[0153] In some other embodiments, the user may first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. The trigger operation may be, for example, a long press operation, a click operation to identify a control, or other specific gesture operations; after the displayed content in the first split screen is converted into a selectable state, the user may select text 401 by clicking, sliding to select, dragging, voice instruction, stylus selection, floating touch, visual scanning, or circling.

[0154] For example, Figure 5 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0155] like Figure 5 As shown in (a), the display screen of the electronic device 400 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, one or more of text 401, table 402 and picture 403, and the second split screen displays, for example, a smart assistant page.

[0156] like Figure 5 As shown in (b), when the user selects part of the content of the text 401 in the first split screen, the second split screen of the electronic device 400 can display the content after completing the part of the content of the text 401. For example, part of the first paragraph is selected in the first split screen, and the entire first paragraph is displayed in the second split screen. For another example, part of the first sentence is selected in the first split screen, and the entire first sentence is displayed in the second split screen. In addition, in an optional example, one or more recommended services associated with the text 401 can also be displayed in the second split screen.

[0157] The one or more recommendation services associated with the text 401 may include, for example, a recommendation service associated with the modality type of the text 401 , and may also include a recommendation service associated with the semantics of the text 401 .

[0158] For example, optionally, the recommended services associated with the modal type of the text 401 may include text interpretation services, text translation services, text sharing services, etc., and may also include other recommended services, such as text preservation services, text editing services, etc., which are not limited in this application.

[0159] For example, the recommended service associated with the semantics of the text 401 may include a currency conversion service, through which the user can convert the content "734,748 million yuan" in the text 401 into other monetary units, such as US dollars.

[0160] In some optional examples, when the user selects the text sharing service in the second split screen, the display interface of the second split screen can jump to the display interface of sharing text 401 with friends. After the user selects the friend to share with in the display interface, the electronic device 400 can send the content of text 401 to the friend.

[0161] In some embodiments, the user selects text 401 in the first split screen by directly performing a selection operation on the first split screen, for example, the user selects text 401 by long pressing and selecting text content in the first split screen; the user may select text 401 by circling text content in the first split screen; the user may select text 401 by directly clicking on the text content in the first split screen; the user may select text 401 through voice instructions; the user may select text 401 in other ways, for example, selecting text through a multi-finger sliding gesture, selecting text through a stylus, selecting text through floating touch, selecting text through visual scanning, etc., and this application does not limit this.

[0162] In some other embodiments, the user may first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. The trigger operation may be, for example, a long press operation, a click operation to identify a control, or other specific gesture operations; after the displayed content in the first split screen is converted into a selectable state, the user may select text 401 by clicking, sliding to select, dragging, voice instruction, circling, stylus, floating touch or visual scanning.

[0163] In this embodiment, when the user selects part of the text content in the first split screen, the electronic device can perceive the text content selected by the user, and the electronic device can complete the text content selected by the user into complete content based on the page content, and display the complete content on the second split screen. In this way, the user does not need to spend a long time to accurately select the interactive text, which can improve the user's work efficiency in handling multiple tasks in split screen, and thus enhance the user's split-screen interactive experience.

[0164] For example, in Figure 4 or Figure 5 Based on the embodiment shown, Figure 6 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0165] like Figure 6 As shown in (a), the second split screen of the electronic device 400 displays one or more recommended services associated with the text 401 on the first split screen. The one or more recommended services may include, for example, a text interpretation service, a text translation service, and a text sharing service.

[0166] like Figure 6 As shown in (b) , when the user selects the text interpretation service, the interpretation result of the text 401 is displayed on the display interface of the second split screen of the electronic device 400 .

[0167] In some optional examples, the user may select a text interpretation service from one or more recommended services by clicking a control corresponding to the text interpretation service, or by voice instructions, etc.

[0168] In an example, text 401 may be, for example, "In 2023, the company insisted on strategic focus and gave full play to its comprehensive advantages in industrial portfolio, technological innovation and complex software and hardware platforms, achieving annual revenue of 734,748 million yuan, a year-on-year increase of 9.6%." Correspondingly, the interpretation result of text 401 displayed on the second screen may be "In 2023, the company's annual revenue was 734,748 million yuan, a year-on-year increase of 9.6%." Further, the interpretation result can also be "In 2023, the company's annual revenue was 734,748 million yuan, a year-on-year increase of 9.6%. According to public data, among companies of the same size and type, the company's annual revenue and growth rate are both in the top 20%."

[0169] For example, Figure 7 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0170] like Figure 7 As shown in (a), the display screen of the electronic device 700 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, one or more of text 701, table 702 and picture 703, and the second split screen displays, for example, a smart assistant page.

[0171] like Figure 7 As shown in (b), when the user selects table 702 in the first split screen, the second split screen of the electronic device 700 displays the content of the table 702, and in an optional example, one or more recommended services associated with the table 702 can also be displayed in the second split screen.

[0172] The one or more recommended services associated with the table 702 may include, for example, a recommended service associated with the modality type of the table 702 , and may also include a recommended service associated with the meaning expressed by the table 702 .

[0173] For example, optionally, the recommended services associated with the modal type of the table 702 may include a table file generation service, a table interpretation service, a data analysis service, a table editing service, etc., and may also include other recommended services, such as a table saving service, a table sharing service, etc., which are not limited in this application.

[0174] For example, optionally, the recommended services associated with the meaning expressed by the table 702 may include a currency conversion service, through which the user can, for example, convert the monetary unit of the amount shown in the table 702 from RMB to other monetary units, such as US dollars; the recommended services associated with the semantics of the table 702 may also include a comparison chart generation service, through which the user can, for example, obtain a revenue comparison chart for each business generated based on the revenue data of each business in each year in the table 702, so as to more intuitively see the revenue distribution.

[0175] In some embodiments, the user may select table 702 in the first split screen by directly performing a selection operation on the first split screen, for example, the user may select table 702 by long pressing table 702 in the first split screen; the user may select table 702 by circling table 702 in the first split screen; the user may select table 702 by directly clicking on table 702 in the first split screen; the user may select table 702 by voice instructions; or the user may select table 702 in other ways, for example, by copying after recognition, multi-finger sliding gesture operation, stylus, floating touch or visual scanning, etc., and this application does not limit this.

[0176] In some other embodiments, the user may first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. The trigger operation may be, for example, a long press operation, a click operation to identify a control, or other specific gesture operations; after the displayed content in the first split screen is converted into a selectable state, the user may select table 702 by clicking, sliding to select, dragging, voice instruction, circling, stylus, floating touch or visual scanning.

[0177] In some optional examples, when the user selects the interpretation form service in the second split screen, the display interface of the second split screen can jump to the following Figure 7 The display interface shown in (c) in FIG. 7 shows the interpretation result of the table 702.

[0178] In some optional examples, the user can select the table interpretation service from one or more recommended services by clicking a control corresponding to the table interpretation service, or by voice instructions, etc.

[0179] In one example, table 702 may be:

[0180] business 2023 2022 growth rate Business 1 361997 353978 2.3％ Business 2 251496 214463 17.3％ Business 3 107894 96148 12.2％ Business 4 13361 6055 120.7％ total 670644 670644 9.6％

[0181] Correspondingly, the interpretation result corresponding to Table 702 may be, for example:

[0182] 1. The revenue of Business 1 in 2023 was 361,997, and the revenue of Business 1 in 2022 was 353,978. The revenue of Business 1 in 2023 increased by 2.3% compared with 2022.

[0183] 2. The revenue of Business 2 in 2023 is 251,496, and the revenue of Business 2 in 2022 is 214,463. The revenue of Business 2 in 2023 is an increase of 17.3% over 2022.

[0184] 3. The revenue of Business 3 in 2023 is 107,894, and the revenue of Business 3 in 2022 is 96,148. The revenue of Business 3 in 2023 is an increase of 12.2% over 2022.

[0185] 4. The revenue of Business 4 in 2023 is 13361, and the revenue of Business 4 in 2022 is 6055. The revenue of Business 4 in 2023 is an increase of 120.7% over 2022.

[0186] 5. The total revenue in 2023 is 734,748, and the total revenue in 2022 is 670,644. The total revenue in 2023 increased by 9.6% compared with 2022.

[0187] For example, the interpretation result corresponding to table 702 may also be:

[0188] In 2023, among all the company's businesses, Business 1 had the highest revenue, accounting for 49.3%, and Business 4 had the least revenue, accounting for 1.8%; it is gratifying that all businesses in 2023 have grown compared with 2022, among which Business 4 has the fastest growth, with a growth rate of 120.7%; but it should be noted that Business 1, as the main source of revenue, has the slowest growth, with a growth rate of only 2.3%.

[0189] In some optional examples, when the user selects the generate table file service in the second split screen, the display interface of the second split screen can jump to the interface for making a table file. In the interface for making a table file, the content in table 702 has been automatically entered into the table file. The user can directly save the table file without manually entering the content in table 702 into the table file.

[0190] For example, Figure 8 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0191] like Figure 8 As shown in (a), the display screen of the electronic device 800 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, one or more of text 801, table 802 and picture 803, and the second split screen displays, for example, a smart assistant page.

[0192] like Figure 8 As shown in (b), when the user selects picture 803 in the first split screen, the second split screen of the electronic device 800 displays the content of the picture 803, and in an optional example, one or more recommended services associated with the picture 803 can also be displayed in the second split screen.

[0193] Among them, the one or more recommended services associated with the picture 803 may include, for example, a recommended service associated with the modality type of the picture 803, and may also include a recommended service associated with the meaning expressed by the picture 803.

[0194] For example, optionally, the recommended services associated with the modal type of the image 803 may include image interpretation services, text extraction services, image translation services, image search services, etc., and may also include other recommended services, such as image saving services, image sharing services, image editing services, etc., which are not limited in this application.

[0195] For example, optionally, the recommended service associated with the meaning expressed by the picture 803 may include a business distribution rationality analysis service. For example, the user can obtain the distribution rationality analysis results of each business in the picture 803 through the business distribution rationality analysis service, so that the user can more intuitively know the distribution rationality of each business. The distribution rationality analysis results of each business in the picture 803 can be generated by the electronic device through big data retrieval and analysis, for example, by comparing and analyzing the distribution of business 1, business 2, business 3 and business 4 shown in the picture 803 with the business distribution of similar companies.

[0196] In some embodiments, the user may select picture 803 in the first split screen by directly performing a selection operation on the first split screen, for example, the user may select picture 803 by long pressing picture 803 in the first split screen; the user may select picture 803 by circling picture 803 in the first split screen; the user may select picture 803 by directly clicking on picture 803 in the first split screen; the user may select picture 803 by voice instructions; the user may select picture 803 in other ways, for example, by copying after recognition, multi-finger sliding gesture operation, stylus, floating touch or visual scanning, etc., and this application does not limit this.

[0197] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. The trigger operation can be, for example, a long press operation, a click recognition control operation, or other specific gesture operations; after the displayed content in the first split screen is converted into a selectable state, the user can select picture 803 by clicking, sliding to select, dragging, voice instruction, circling, stylus, floating touch or visual scanning.

[0198] In some optional examples, when the user selects the image interpretation service in the second split screen, the display interface of the second split screen can jump to the following Figure 8 The display interface shown in (c) in FIG. 8 shows the interpretation result of the image 803 .

[0199] In some optional examples, the user can select a picture interpretation service from one or more recommended services by clicking a control corresponding to the picture interpretation service, or by voice instructions.

[0200] In an example, the interpretation result corresponding to picture 803 may be, for example, "the revenue share of business 1 is 49.3%; the revenue share of business 2 is 34.2%; the revenue share of business 3 is 14.7%; and the revenue share of business 4 is 1.8%."

[0201] In an example, the interpretation result corresponding to picture 803 can also be, for example, "Business 1 accounts for 49.3% of revenue; Business 2 accounts for 34.2% of revenue; Business 3 accounts for 14.7% of revenue; Business 4 accounts for 1.8% of revenue. Among them, Business 1 accounts for the highest revenue share, and Business 4 accounts for the lowest revenue share. The difference in revenue share between Business 1 and Business 4 is 47.5%. According to public data, the revenue share of Business 4 of our company is lower than that of similar companies. We can consider increasing the development layout of Business 4 in the next year."

[0202] In some optional examples, when the user selects the text extraction service in the second split screen, the display interface of the second split screen can jump to the text extraction interface corresponding to picture 803, in which the text extracted from picture 803 "Business 149.3%, Business 234.2%, Business 314.7%, Business 41.8%" can be displayed.

[0203] For example, Figure 9 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0204] like Figure 9 As shown in (a), the display screen of the electronic device 900 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application. The first application and the second application are different. The first split screen takes the gallery application interface as an example, which displays picture 901, and the second split screen displays a smart assistant page.

[0205] When the user selects picture 901 in the first split screen, the picture 901 can be displayed on the second split screen of the electronic device 900. In an optional example, one or more recommended services associated with the picture 901 can also be displayed in the second split screen.

[0206] Among them, the one or more recommended services associated with the picture 901 may include, for example, a recommended service associated with the modality type of the picture 901 , and may also include a recommended service associated with the meaning expressed by the picture 901 .

[0207] For example, optionally, the one or more recommendation services may include image interpretation services, text extraction services, image translation services, transfer services, etc., and may also include other recommendation services, such as image storage services, image sharing services, image search services, etc. This application does not limit this.

[0208] In some embodiments, the user may select picture 901 in the first split screen by directly performing a selection operation on the first split screen, for example, the user may select picture 901 by long pressing picture 901 in the first split screen; the user may select picture 901 by circling picture 901 in the first split screen; the user may select picture 901 by directly clicking on picture 901 in the first split screen; the user may select picture 901 by voice instructions; the user may select picture 901 in other ways, for example, by copying after recognition, multi-finger sliding gesture operation, stylus, floating touch or visual scanning, etc., and this application does not limit this.

[0209] In some other embodiments, the user can first perform a trigger operation on the first split screen to convert the displayed content in the first split screen into a selectable state. The trigger operation can be, for example, a long press operation, a click operation to identify a control, or other specific gesture operations; after the displayed content in the first split screen is converted into a selectable state, the user can select picture 901 by clicking, sliding to select, dragging, voice instructions, circling, stylus, floating touch or visual scanning.

[0210] In some optional examples, when the user selects the text extraction service in the second split screen, the display interface of the second split screen can jump to the following example: Figure 9 The display interface shown in (b) in the figure displays the text "Bank account number: 1234567891234567; Opening bank: xxxxxx; Payee: Zhang San" extracted from picture 901. In addition, the interface can also display one or more recommended services determined based on the extracted text information, such as a transfer service, a copy all text information service, etc.

[0211] When the user selects the transfer service in the second split screen, the display interface of the second split screen can jump to the following Figure 9 In the transfer application interface shown in (c), the text information extracted from image 901 has been automatically filled in the corresponding position. The user only needs to check the transfer information and enter the transfer amount to directly complete the transfer to the target object.

[0212] In another implementation, the user can directly Figure 9 In the second split screen shown in (a), click on the transfer service associated with the meaning expressed by the picture 901, so that the interface of the second split screen jumps directly to the Figure 9 The transfer interface shown in (c) in the figure.

[0213] For example, Figure 10 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0214] like Figure 10 As shown in (a), the display screen of the electronic device 1000 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, and the first application and the second application are different.

[0215] Taking the example of the first split screen being the interface of a video application and the second split screen being the interface of a notebook application, the first split screen displays, for example, the playback interface of a video course, in which the course content "abandon, abandon, give up, ability; talent, able; capable, abnormal, perverted" is displayed, and the second split screen displays, for example, the note-taking interface.

[0216] like Figure 10 As shown in (b), when the user wants to record the course content "able; capable, abnormal, perverted" displayed on the first split screen as a note, the user can select the course content "able; capable, abnormal, perverted" on the first split screen. In response to the user's operation of selecting the course content "able; capable, abnormal, perverted" on the first split screen, the course content "able; capable, abnormal, perverted" is automatically recorded as a note on the second split screen of the electronic device 1000.

[0217] It should be understood that this application does not limit the way in which the user selects course content on the first split screen. For example, it can be done by circling, clicking, long pressing, voice instructions, stylus operation or visual scanning.

[0218] In some embodiments, when a user selects interactive content on the interface of a first application displayed on a first split screen of an electronic device, the interface of a second application displayed on a second split screen of the electronic device may not display the selected interactive content, but may directly display a recommended service associated with the selected interactive content. Furthermore, in response to the user selecting a target recommended service, the target recommended service is executed for the selected interactive content on the second split screen.

[0219] For example, Figure 11 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0220] like Figure 11 As shown in (a), the display screen of the electronic device 1100 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, and the first application and the second application are different.

[0221] Taking the example of the first split screen being the interface of the gallery application and the second split screen being the interface of the shopping application, the first split screen displays, for example, picture 1101, which includes, for example, an image of a smart speaker 1102, and the second split screen displays, for example, a shopping application page, which may include, for example, sales information of multiple products.

[0222] like Figure 11 As shown in (b), when the user wants to obtain the sales information of the smart speaker 1102 included in the picture 1101, the user can select the picture 1101 on the first split screen. In response to the user's operation of selecting the picture 1101 on the first split screen, the sales information of smart speakers that are the same as or similar to the smart speaker 1102 included in the picture 1101 can be displayed on the second split screen of the electronic device 1000, so that the user can easily view the sales information of the smart speaker 1102 included in the picture 1101 without the need for the user to search separately.

[0223] It should be understood that the present application does not limit the way in which the user selects the picture 1101 on the first split screen. For example, the user can select the picture 1101 by copying, circling, clicking, long pressing, voice instruction, stylus operation or visual scanning.

[0224] For example, Figure 12 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0225] like Figure 12 As shown in (a), the display screen of the electronic device 1200 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, text 1201, a table 1202 and a picture 1203, and the second split screen displays, for example, a smart assistant page.

[0226] like Figure 12 As shown in (b) , when the user selects the table 1202 and the picture 1203 in the first split screen, the second split screen of the electronic device 1200 can display the content of the table 1202 and the content of the picture 1203 .

[0227] Furthermore, in the second split screen, one or more recommended services associated with the table 1202 and the picture 1203 may also be displayed.

[0228] Among them, the one or more recommendation services associated with the table 1202 and picture 1203 may include, for example, recommendation services associated with the modal type of the table 1202 and picture 1203, and may also include recommendation services associated with the meaning expressed by the table 1202 and picture 1203.

[0229] For example, optionally, the recommendation services associated with the modal types of the table 1202 and the picture 1203 may include comprehensive interpretation services, data comprehensive analysis services, data aggregation services, etc., and may also include other recommendation services, such as table and picture saving services, table and picture sharing services, etc., which are not limited in this application.

[0230] For example, the recommended services associated with the meanings expressed by the table 1202 and the picture 1203 may optionally include a comprehensive business analysis service, a currency conversion service, and / or a comparison chart generation service.

[0231] In some embodiments, the user can select table 1202 and picture 1203 in the first split screen by circling them; the user can also select table 1202 and picture 1203 by other methods, for example, by copying after recognition, multi-finger sliding gesture operation, stylus, floating touch, voice instructions or visual scanning, etc., which are not limited in this application.

[0232] In some optional examples, when the user selects the comprehensive interpretation service in the second split screen, the display interface of the second split screen can jump to the following Figure 12 The display interface shown in (c) in FIG. 1 shows the comprehensive interpretation results of the table 1202 and the picture 1203.

[0233] In some optional examples, the user can select the comprehensive interpretation of tables and pictures service from one or more recommended services by clicking a control corresponding to the comprehensive interpretation of tables and pictures service, or by voice instructions.

[0234] In one example, table 1202 may be:

[0235] business 2023 2022 growth rate Business 1 361997 353978 2.3％ Business 2 251496 214463 17.3％ Business 3 107894 96148 12.2％ Business 4 13361 6055 120.7％ total 670644 670644 9.6％

[0236] For example, picture 1203 may indicate that in 2023, business 1 accounts for 49.3%; business 2 accounts for 34.2%; business 3 accounts for 14.7%; and business 4 accounts for 1.8%.

[0237] Correspondingly, the comprehensive interpretation result corresponding to the table 1202 and the picture 1203 may be, for example:

[0238] 1. Revenue from each business segment has significantly increased compared to the previous year

[0239] Compared to 2022, revenue from each business segment increased in 2023, with total revenue increasing by 9.6%. Business 4 saw the fastest growth, at 120.7%, while Business 1 saw the slowest growth, at 2.3%.

[0240] 2. The rationality of business distribution needs to be improved

[0241] In 2023, Business 1 had the highest revenue, accounting for 49.3%, and Business 4 had the lowest revenue, accounting for 1.8%. The difference in revenue share between the two was as high as 47.5%, which was relatively large compared with similar companies.

[0242] 3. Business 4 has great room for expansion

[0243] According to public data, among companies of the same type, the revenue share of Business 4 generally increased significantly in 2023, and the growth rate of a considerable number of companies was greater than 150%. It is recommended that the company consider continuing to expand Business 4 in the next year, but a risk analysis should be conducted.

[0244] In another example, the comprehensive interpretation result corresponding to the table 1202 and the picture 1203 may be:

[0245] 1. In 2023, the revenue of Business 1 was 361,997, accounting for 49.3%. In 2022, the revenue of Business 1 was 353,978. The revenue of Business 1 in 2023 increased by 2.3% compared with 2022.

[0246] 2. In 2023, the revenue of Business 2 was 251,496, accounting for 34.2%. In 2022, the revenue of Business 2 was 214,463. The revenue of Business 2 in 2023 increased by 17.3% compared with 2022.

[0247] 3. In 2023, the revenue of Business 3 was 107,894, accounting for 14.7%. In 2022, the revenue of Business 3 was 96,148. The revenue of Business 3 in 2023 increased by 12.2% compared with 2022.

[0248] 4. In 2023, the revenue of Business 4 was 13,361, accounting for 1.8%. In 2022, the revenue of Business 4 was 6,055. The revenue of Business 4 in 2023 increased by 120.7% compared with 2022.

[0249] 5. The total revenue in 2023 is 734,748, and the total revenue in 2022 is 670,644. The total revenue in 2023 increased by 9.6% compared with 2022.

[0250] For example, Figure 13 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0251] like Figure 13 As shown in (a), the display screen of the electronic device 1200 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, text 1201, a table 1202 and a picture 1203, and the second split screen displays, for example, a smart assistant page.

[0252] like Figure 13 As shown in (b) , when the user selects the table 1202 and the picture 1203 in the first split screen, the second split screen of the electronic device 1200 can display the content of the table 1202 and the content of the picture 1203 .

[0253] Furthermore, in the second split screen, one or more recommended services associated with the table 1202 and the picture 1203 may also be displayed.

[0254] In some optional examples, table 1202 displayed in the second split screen is editable. When the user performs an editing operation on table 1202 in the second split screen, correspondingly, other content displayed in table 1202 and content displayed in picture 1203 can adapt to changes as the content of table 1202 changes.

[0255] For example, Figure 13 As shown in (c) in the figure, when the user modifies the revenue of Business 4 in 2023 from "13361" to "20157" in Table 1202 in the second split screen, correspondingly, in Table 1202, the growth rate of Business 4 in 2023 relative to 2022 changes from "120.7%" to "232.9%", and the total revenue in 2023 changes from "734748" to "741544", and the total revenue in 2023 changes from "734748" to "741544". The revenue growth rate relative to 2022 changes from "9.6%" to "10.6%"; adaptively, in Figure 1203, the revenue share of business 1 changes from "49.3%" to "48.8%", the revenue share of business 2 changes from "34.2%" to "33.9%", the revenue share of business 3 changes from "14.7%" to "14.6%", and the revenue share of business 4 changes from "1.8%" to "2.7%".

[0256] For example, Figure 14 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0257] like Figure 14 As shown in (a), the display screen of the electronic device 1400 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, text 1401, a table 1402 and a picture 1403, and the second split screen displays, for example, a smart assistant page.

[0258] like Figure 14 As shown in (b) , when the user selects the table 1402 and the picture 1403 in the first split screen, the second split screen of the electronic device 1400 can display the content of the table 1402 and the content of the picture 1403 .

[0259] Furthermore, in the second split screen, one or more recommended services associated with the table 1402 may also be displayed, and one or more recommended services associated with the picture 1403 may also be displayed.

[0260] The one or more recommended services associated with the table 1402 may include, for example, a recommended service associated with the modality type of the table 1402 , and may also include a recommended service associated with the meaning expressed by the table 1402 .

[0261] For example, optionally, the recommended services associated with the modal type of the table 1402 may include a table file generation service, a table interpretation service, a data analysis service, a table editing service, etc., and may also include other recommended services, such as a table saving service, a table sharing service, etc., which are not limited in this application.

[0262] For example, the recommended services associated with the meaning expressed by the table 1402 may include a currency conversion service or a comparison chart generation service.

[0263] Among them, the one or more recommended services associated with the picture 1403 may include, for example, a recommended service associated with the modality type of the picture 1403, and may also include a recommended service associated with the meaning expressed by the picture 1403.

[0264] For example, optionally, the recommended services associated with the modal type of the image 1403 may include image interpretation services, text extraction services, image translation services, image search services, etc., and may also include other recommended services, such as image saving services, image sharing services, image editing services, etc., which are not limited in this application.

[0265] For example, optionally, the recommended services associated with the meaning expressed by the picture 1403 may include a business distribution rationality analysis service, etc.

[0266] In some embodiments, the user can select table 1402 and picture 1403 in the first split screen by circling them; the user can also select table 1402 and picture 1403 by other methods, for example, by copying after recognition, multi-finger sliding gesture operation, stylus, floating touch, voice instructions or visual scanning, etc., which are not limited in this application.

[0267] It is understandable that Figure 12 The embodiments and Figure 14 The illustrated embodiments can be implemented in combination, that is, when the interactive content selected by the user in the first split screen includes display information of multiple modal types (for example, two or more of text, pictures, tables and links), the recommended services associated with each of the multiple display information can be displayed separately in the second split screen, and the recommended services associated with the multiple display information as a whole can also be displayed.

[0268] For example, based on Figure 12In (a), when the user selects table 1202 and picture 1203 in the first split screen, the second split screen of the electronic device 1200 can display the content of the table 1202 and the content of the picture 1203, and in the second split screen, recommended services associated with the table 1202 and the picture 1203 can also be displayed, as well as recommended services associated with the table 1202 and the picture 1203 can also be displayed.

[0269] in,

[0270] The recommended services associated with the table 1202 and the picture 1203 may include, for example, recommended services associated with the modal types of the table 1202 and the picture 1203, such as comprehensive interpretation services, comprehensive data analysis services, data aggregation services, etc.; they may also include recommended services associated with the meaning expressed by the table 1202 and the picture 1203, such as comprehensive business analysis services, currency conversion services and / or comparison chart generation services, etc.

[0271] The recommended services associated with the table 1202 may include, for example, recommended services associated with the modal type of the table 1202, such as a table file generation service, a table interpretation service, a data analysis service, a table editing service, etc.; they may also include recommended services associated with the meaning expressed by the table 1202, such as a currency conversion service or a comparison chart generation service, etc.

[0272] The recommended services associated with the image 1203 may include, for example, recommended services associated with the modal type of the image 1203, such as image interpretation service, text extraction service, image translation service, image search service, etc.; they may also include recommended services associated with the meaning expressed by the image 1203, such as business distribution rationality analysis service, etc.

[0273] It should be understood that the embodiments of the present application are illustrated using the modal type of interactive content as text, table or picture as an example, but this does not limit the modal type of interactive content of the present application. For example, the modal type of interactive content in the present application may also include web page links, etc.

[0274] For example, Figure 15 A schematic diagram of an interface corresponding to another split-screen interaction method provided in an embodiment of the present application is shown.

[0275] like Figure 15As shown in (a), the display screen of the electronic device 1500 displays a first split screen and a second split screen, the first split screen displays the interface of the first application, and the second split screen displays the interface of the second application, the first application and the second application are different, wherein the first split screen displays, for example, one or more of text 1501, table 1502 and picture 1503, and the second split screen displays, for example, a smart assistant page.

[0276] like Figure 15 As shown in (b) in FIG, when the user selects table 1502 in the first split screen, the second split screen of the electronic device 1500 displays the content of the table 1502.

[0277] In an optional example, one or more recommended services associated with the table 1502 may also be displayed in the second split screen.

[0278] Among them, the explanation of one or more recommended services associated with the table 1502 is in Figure 7 The illustrated embodiment has been described in detail and will not be repeated here for the sake of brevity.

[0279] For an explanation of how the user selects table 1502 in the first split screen, see Figure 7 In the illustrated embodiment, the explanation of how the user selects the table 702 in the first split screen will not be repeated here for the sake of brevity.

[0280] In some optional examples, when the user is in Figure 15 By clicking the voice control 1504 on the smart assistant interface displayed in the second split screen as shown in (b), a voice input box 1505 can be displayed on the display interface of the second split screen. The voice input box 1505 may include a guiding statement for guiding the user to issue a voice command, and the guiding statement may be, for example, "Please tell me your question..."

[0281] After the user issues a voice instruction based on table 1502 , the electronic device may execute the user's voice instruction based on table 1503 and output the execution result of the voice instruction to the user.

[0282] like Figure 15 (c) and Figure 15 As shown in (d), after the user triggers the voice control 1504, he issues a voice command "Help me interpret this table", and correspondingly, the second split screen displays the voice command "Help me interpret this table" and the interpretation result of the table 1502 in the form of a dialogue.

[0283] In some embodiments, the interpretation result of the table 1502 displayed on the second split screen can be generated based on the table 1502; it can also be generated based on the table 1502 and other content displayed on the first split screen (such as text 1501 and / or picture 1503); it can also be generated based on the table 1502, other content displayed on the first split screen and / or relevant information in the database.

[0284] In this embodiment, the question-and-answer results obtained in the question-and-answer format can be generated based on the contextual semantics of the content displayed on the first split screen, which makes the accuracy of the obtained question-and-answer results higher.

[0285] In some embodiments, Figure 15 Based on what is shown in (d), the user can issue further voice instructions through the voice control 1504 to enable multiple rounds of voice interaction between the user and the electronic device based on the table 1502 on the second split screen.

[0286] In some embodiments, the user can further select picture 1503 on the first split screen and issue voice commands based on the picture 1503 and the previously selected table 1502 to achieve multiple rounds of voice interaction between the user and the electronic device based on the table 1502 and picture 1503 on the second split screen.

[0287] For example, Figure 16 A schematic flowchart of another split-screen interaction method 1600 provided in an embodiment of the present application is shown.

[0288] The method 1600 is applied to an electronic device, wherein the display screen of the electronic device displays a first split-screen interface and a second split-screen interface, wherein the first split-screen interface displays the content of a first application, and the second split-screen interface displays the content of a second application, and the first application is different from the second application, such as Figure 16 As shown, the method 1600 includes:

[0289] S1601: In response to a user operation in an interactive area of ​​a first split-screen interface, the electronic device obtains first content displayed in the interactive area.

[0290] S1602: The electronic device identifies the first content according to the modality type corresponding to the first content, and obtains second content associated with the first content.

[0291] In some embodiments, the electronic device establishes a correspondence between the display area and the modal type in the current display interface of the first split-screen interface through controls in the first split-screen interface, or by performing layout analysis on the first split-screen interface, or by performing document parsing on the first split-screen interface; then the electronic device obtains the modal type corresponding to the interactive area based on the correspondence, and thus obtains the modal type corresponding to the first content displayed in the interactive area.

[0292] In some embodiments, the modality type may include one or more of a text type, a table type, a picture type, and a link type.

[0293] In some examples, the electronic device establishes a correspondence between the display area in the current display interface of the first split-screen interface and the modal type by performing layout analysis on the first split-screen interface. For example, it may refer to establishing a correspondence between the coordinate position in the first split-screen interface of each text displayed in the current display interface of the first split-screen interface and the modal type, and / or establishing a correspondence between the coordinate position in the first split-screen interface of each picture displayed in the current display interface of the first split-screen interface and the modal type, and / or establishing a correspondence between the coordinate position in the first split-screen interface of each table displayed in the current display interface of the first split-screen interface and the modal type, and / or establishing a correspondence between the coordinate position in the first split-screen interface of each link displayed in the current display interface of the first split-screen interface and the modal type, etc.

[0294] In some embodiments, different modality types correspond to different recognition models. The electronic device may determine a model for recognizing the first content based on the modality type of the first content, and then recognize the first content using the determined model.

[0295] In one example, the modality type corresponding to the first content is text. After obtaining the first content, the electronic device recognizes the first content using a recognition model corresponding to the text to obtain the second content. Alternatively, the electronic device can directly obtain the second content through layout analysis or document parsing without the need for model recognition.

[0296] In one example, the modal type corresponding to the first content is a table. After the electronic device obtains the first content, it recognizes the first content using a recognition model corresponding to the table to obtain the second content. For example, the electronic device can obtain a cropped image corresponding to the selected table from the first split-screen interface, input the cropped image into the recognition model corresponding to the table to obtain the structure of the selected table and the text content within the cells, and then represent the cropped image in HTML format to obtain the second content.

[0297] In one example, the modal type corresponding to the first content is an image. After the electronic device obtains the first content, it recognizes the first content using a recognition model corresponding to the image to obtain the second content. For example, the electronic device can call an optical character recognition (OCR) module to obtain text, and call a vision-language model (VLM) or a large multimodal model (LMM) to obtain content and description related to the image to obtain the second content, and further display the second content on the second split-screen interface.

[0298] In one example, the modality types corresponding to the first content are a table and a picture. After the electronic device obtains the first content, it recognizes the first content using a recognition model corresponding to the table and a recognition model corresponding to the picture.

[0299] In some embodiments, the electronic device obtains the first content associated with the interactive area by, for example, obtaining it through an identification control in the first split-screen interface, or by performing layout analysis on the first split-screen interface, or by performing document parsing on the first split-screen interface.

[0300] S1603: The electronic device generates a recommendation service associated with the first content based on the first content and / or the second content.

[0301] For an explanation of the recommendation service associated with the first content, see Figure 3 For the sake of brevity, the description in the illustrated embodiment will not be repeated here.

[0302] In one example, the first content includes a first text and a first picture, and correspondingly, the acquired second content includes the first text and the first picture. The modal type corresponding to the interaction area includes a text type and a picture type. After acquiring the first content, the electronic device recognizes the first text through a model corresponding to the text type and recognizes the first picture through a model corresponding to the picture type. The electronic device generates a recommendation service associated with the first text based on the recognition results of the first text and / or the first text, and generates a recommendation result associated with the first picture based on the recognition results of the first picture and / or the first picture; or the electronic device generates a recommendation service associated with the first text and the first picture based on the recognition results of the first text and the recognition results of the first picture.

[0303] S1604: The electronic device displays the second content on the second split-screen interface, and / or the electronic device displays a recommended service corresponding to the first content on the second split-screen interface.

[0304] For an explanation of this step, see Figure 3 For the sake of brevity, the explanation of S301 in the illustrated embodiment will not be repeated here.

[0305] For example, Figure 17 A schematic architecture diagram of a split-screen interaction device 1700 provided in an embodiment of the present application is shown.

[0306] like Figure 17 As shown, the device 1700 includes a perception module 1710, multiple identification modules (for example, including a first identification module 1720, a second identification module 1730, a third identification module 1740, and a fourth identification module 1750), and an intelligent analysis module 1760. The perception module 1710 can be located, for example, at a perception layer of an electronic device, and the multiple identification modules and the intelligent analysis module 1760 can be located, for example, at a processing layer of the electronic device.

[0307] The perception module 1710 is used to obtain the first content displayed in the interactive area in response to the user's operation in the interactive area of ​​the first split-screen interface.

[0308] The perception module 1710 is further configured to send the acquired first content to the corresponding recognition module.

[0309] The perception module 1710 is further configured to send the acquired first content to the intelligent analysis module 1760 .

[0310] The identification module is used to receive the first content sent by the perception module 1710, and is also used to obtain the second content associated with the first content by identifying the first content.

[0311] In one example, the recognition module may include a first recognition module 1720, a second recognition module 1730, a third recognition module 1740 and a fourth recognition module 1750, wherein the first recognition module 1720 is used to identify interactive content whose modal type is text, the second recognition module 1730 is used to identify interactive content whose modal type is table, the third recognition module 1740 is used to identify interactive content whose modal type is picture, and the fourth recognition module 1750 is used to identify interactive content whose modal type is link.

[0312] Among them, the device 1700 can also include a background processing module, the perception module 1710 sends the first content to the background processing module, and the background processing module distributes the first content to the corresponding recognition module according to the modal type corresponding to the first content; or, the perception module 1710 directly distributes the first content to the corresponding recognition module according to the modal type corresponding to the first content.

[0313] In one example, the modal type corresponding to the first content is a table, and accordingly, the first content is table content. The perception module 1710 sends the first content to the recognition module corresponding to the table through the background processing module, or the perception 1710 directly sends the first content to the recognition module corresponding to the table according to the modal type corresponding to the first content.

[0314] The intelligent analysis module 1760 is configured to receive the first content sent by the perception module 1710 .

[0315] In some embodiments, when the modality type corresponding to the first content is text, the perception module 1710 can directly send the first content to the intelligent analysis module 1760.

[0316] The intelligent analysis module 1760 is further configured to receive the second content sent by the recognition module.

[0317] The intelligent analysis module 1760 is further configured to generate a recommendation service associated with the first content based on the first content and / or the first and second contents.

[0318] The apparatus 1700 may further include a display module configured to display the second content and / or display a recommended service associated with the first content on the second split-screen interface of the electronic device.

[0319] One or more of the modules or units described herein can be implemented in software, hardware, or a combination of the two. When any of the above modules or units is implemented in software, the software exists in the form of computer program instructions and is stored in a memory, and a processor can be used to execute the program instructions and implement the above method flow. The processor may include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller (MCU), or an artificial intelligence processor, etc., a computing device that runs software, each computing device may include one or more cores for executing software instructions to perform operations or processing. The processor may be built into an SoC (system on chip) or an application specific integrated circuit (ASIC), or it may be an independent semiconductor chip. In addition to the core for executing software instructions to perform operations or processing within the processor, it may further include necessary hardware accelerators, such as field programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0320] When the modules or units described in this document are implemented in hardware, the hardware can be any one or any combination of a CPU, a microprocessor, a DSP, an MCU, an artificial intelligence processor, an ASIC, a SoC, an FPGA, a PLD, a dedicated digital circuit, a hardware accelerator, or a non-integrated discrete device, which can run the necessary software or not rely on the software to execute the above method flow.

[0321] When the modules or units described herein are implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0322] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0323] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0324] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0325] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0326] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0327] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0328] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A split-screen interaction method, characterized in that: The method is applied to an electronic device, and includes: Displaying a first split-screen interface and a second split-screen interface, wherein the first split-screen interface displays content of a first application, and the second split-screen interface displays content of a second application, where the first application is different from the second application; In response to the user's operation in the interaction area of ​​the first split-screen interface, the electronic device displays second content associated with the first content displayed in the interaction area on the second split-screen interface, and / or the electronic device displays a recommended service associated with the first content displayed in the interaction area on the second split-screen interface, wherein the second content is obtained by running the second application and the recommended service is obtained by running the second application.

2. The method according to claim 1, characterized in that The second content associated with the first content includes: The content is obtained by copying the first content, the content is obtained by completing the first content, or the content is obtained by converting the format of the first content.

3. The method according to claim 1 or 2, characterized in that The recommendation service associated with the first content includes: A recommendation service associated with a modality type of the first content and / or a recommendation service associated with a meaning of the first content, wherein the modality type is used to indicate a carrier form and / or an expression form of the first content.

4. The method according to claim 3, characterized in that When the modality type of the first content includes a text type, the recommended service associated with the modality type of the first content includes one or more of a text interpretation service, a text editing service, a text translation service, and a text sharing service; and / or When the modality type of the first content includes a table type, the recommended service associated with the modality type of the first content includes one or more of a table file generation service, a table editing service, a table interpretation service, and a data analysis service; and / or When the modal type of the first content includes a picture type, the recommended service associated with the modal type of the first content includes one or more of a picture interpretation service, a picture editing service, a text extraction service, a picture translation service, and a picture search service.

5. The method according to claim 3 or 4, characterized in that When the modality type of the first content includes a plurality of modality types, the recommendation service associated with the modality type of the first content includes a recommendation service associated with at least one of the plurality of modality types.

6. The method according to any one of claims 1 to 5, characterized in that When the modality type of the first content includes a first modality type and a second modality type, the method further includes: In response to the user's operation of editing the content corresponding to the first modal type in the second split-screen interface, the electronic device synchronously updates the content corresponding to the second modal type displayed in the second split-screen interface according to the content corresponding to the edit.

7. The method according to any one of claims 1 to 6, characterized in that The second content is the content obtained by the electronic device performing sentence completion and / or paragraph completion on the first content.

8. The method according to any one of claims 1 to 7, characterized in that The second split-screen interface also displays a voice control, and the method further includes: In response to a user's touch operation on the voice control, the electronic device acquires a first voice instruction; The electronic device displays the first result on the second split-screen interface according to the first voice instruction and the first content.

9. The method according to claim 8, characterized in that The electronic device displays the first result on the second split-screen interface according to the first voice instruction and the first content, including: The electronic device displays the first result on the second split-screen interface based on the content displayed on the first split-screen interface, the first voice instruction and the first content.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Establishing a correspondence between a display area in a current display interface of the first split-screen interface and a modal type through controls in the first split-screen interface, or through layout analysis of the first split-screen interface, or through document parsing of the first split-screen interface; According to the corresponding relationship, a modal type corresponding to the first content displayed in the interactive area is obtained.

11. An electronic device, characterized in that: include: one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that The storage medium stores a program or instruction, and when the program or instruction is executed, the method according to any one of claims 1 to 10 is implemented.

13. A chip, characterized in that: Instructions are stored in the chip, and when the instructions are executed, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product, characterized in that The computer program product stores a program or an instruction, and when the program or the instruction is executed, the method according to any one of claims 1 to 10 is implemented.

Images