Information sending method and electronic device

Abstract

The present application relates to the technical field of terminals. Provided are an information sending method and an electronic device. The method comprises: generating first information, the first information comprising second information and third information, wherein the second information is used for indicating a touch event of a first device, and the third information is used for indicating an event attribute corresponding to the touch event, the touch event and the event attribute being used for indicating a display mode of a media resource, and the media resource being a resource acquired by a second device from the first device, or the media resource being a resource acquired by the second device from a third device; and sending the first information to the second device. The method can reduce the power consumption of a terminal device and improve a screen mirroring experience.

Description

Method and electronic device for sending information

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 23, 2023, with application number 202311584721.7 and application name “Method and electronic device for sending information”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of communication technology, and in particular to a method and electronic device for sending information. Background Art

[0003] Screen projection is also called same screen, flying screen, and screen sharing. It uses screen projection technology to display the screen of a mobile device (such as a mobile phone, tablet, etc.) on the screen of another device (such as a TV, projector, etc.) in real time. The output content includes various multimedia information such as pictures, audio, video, and real-time operation pictures on the mobile device. Currently, most screen projection scenarios use mirrored screen projection. Among them, the mirrored screen projection method refers to a method of transmitting the content displayed on the screen of a source multimedia terminal (such as a mobile phone) with strong core processing capabilities to the destination multimedia terminal (such as a TV) in real time in the form of streaming media through a wireless network for display; through mirrored screen projection, the media resources (such as documents, pictures, audio and video, etc.) on terminal devices such as mobile phones or tablets can be completely copied to the projection end, thereby obtaining a cross-screen experience; however, this mirrored screen projection method will increase the power consumption of terminal devices such as mobile phones or tablets, affecting the screen projection experience.

[0004] Summary of the Invention

[0005] The present application provides a method and electronic device for sending information, which can reduce the power consumption of terminal devices and improve the screen projection experience.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a method for sending information is provided, the method comprising: generating first information, the first information comprising second information and third information, the second information being used to indicate a touch event of the first device, the third information being used to indicate an event attribute corresponding to the touch event, the touch event and the event attribute being used to indicate a display method of a media resource, the media resource being a resource obtained by the second device from the first device, or the media resource being a resource obtained by the second device from the third device; and sending the first information to the second device.

[0008] The above method can be executed by the first device, or by a module (such as a processor, chip, or chip system, etc.) applied to the first device, or by a logical node, logical module or software that can realize all or part of the functions of the first device. In the existing mirroring screen projection method, in order to ensure the synchronization of the pictures of the first device (such as a mobile phone) and the second device (such as a TV), the encoder of the first device needs to encode all the content on the display page (that is, all the content on the virtual screen) in real time (for example, encode once every ten milliseconds) and send the encoding result to the second device. However, in some cases, the content on the display page of the first device will not change in a short time. For example, the user projects the photos in the mobile phone gallery to the TV for viewing. In most cases during browsing, the user will not turn the page frequently, but will turn to the next one after a period of time (for example, 5 seconds). At this time, the TV display screen needs to be consistent with the display screen on the mobile phone; if mirroring is used, the content on the display page of the first device will not change in a short time. In the screen projection mode, when a user watches a picture for 5 seconds, the mobile phone has encoded the display page of the current photo hundreds of times. Obviously, this wastes the power consumption of the mobile phone. In this application, the first device generates corresponding control information (for example, first information) according to the user's operation requirements (for example, turning pages), and encodes the control information and sends it to the second device, instead of encoding all the contents of the current page of the first device (for example, control icons, etc.). Even if the first device sends the media resources to be displayed (for example, another picture) from the local to the second device, it only encodes the media resources without encoding the entire page of the first device, thereby reducing the power consumption of the first device and improving the user's screen projection experience.

[0009] In a possible implementation, the third information includes quantity information, index information, and coordinate information. The quantity information is used to indicate the number of pointers of the touch event, the index information is used to indicate the index of the touch event, and the coordinate information is used to indicate the coordinate data set of the touch event.

[0010] The first device indicates the event attributes of the touch event through quantity information, index information and coordinate information, which helps the second device (for example, the projection end) accurately determine the gesture touch operation corresponding to the touch event based on the quantity information, index information and coordinate information.

[0011] In one possible implementation, the coordinate information includes: identification information, first type information, first coordinate information, and second coordinate information, the identification information is used to indicate the pointer identifier of the touch event, the first type information is used to indicate the operation type corresponding to the touch event, the first coordinate information is used to indicate the horizontal coordinate of the touch event, and the second coordinate information is used to indicate the vertical coordinate of the touch event.

[0012] The first device further indicates the coordinate data set corresponding to the touch event through identification information, first type information, etc., which is beneficial for the second device (for example, the projection end) to further improve the accuracy of determining the user touch operation corresponding to the touch event based on the coordinate information.

[0013] In one possible implementation, the coordinate information also includes: second type information, when the first type information is used to indicate that the operation type corresponding to the touch event is a touch operation type, the second type information is used to indicate a floating operation type; or, when the first type information is used to indicate that the operation type corresponding to the touch event is a floating operation type, the second type information is used to indicate a touch operation type.

[0014] In some special scenarios, the coordinate information also includes second type information. The operation types indicated by the first type information and the second type information are different. Different operation types can be used to indicate the same touch event or different touch events, thereby meeting the needs of coordinate information in different scenarios to indicate different touch events of users.

[0015] In a possible implementation, the touch operation type includes a pressing action, a lifting action, and a moving action.

[0016] In a possible implementation, the touch operation type further includes an action mask.

[0017] In a possible implementation, the hover operation types include hover movement, hover entry, and hover exit.

[0018] In a possible implementation, the media resource is a picture.

[0019] In a possible implementation, the display mode is a zoom display mode, a page turning display mode, or a rotation display mode.

[0020] In a second aspect, a method for receiving information is provided, the method comprising: receiving first information from a first device, the first information comprising second information and third information, the second information being used to indicate a touch event of the first device, the third information being used to indicate an event attribute corresponding to the touch event, the touch event and the event attribute being used to indicate a display method of a media resource, the media resource being a resource obtained from the first device, or the media resource being a resource obtained from a third device; and displaying the media resource according to the display method in response to the first information.

[0021] The above method can be executed by the second device, or by a module (such as a processor, chip, or chip system, etc.) applied to the second device, or by a logical node, logical module or software that can realize all or part of the functions of the second device. In the existing mirroring screen projection method, in order to ensure the synchronization of the screen of the first device (such as a mobile phone) and the second device (such as a TV), the first device needs to encode all the content on its display page in real time (for example, encode once every ten milliseconds) and send the encoding result to the second device. Correspondingly, the second device needs to decode a lot of data; however, in some cases, the content on the display page of the first device will not change in a short time. For example, the user projects the photos in the mobile phone gallery to the TV for viewing. In most cases during browsing, the user will not turn the page frequently, but will turn to the next one after a period of time (for example, 5 seconds). At this time, the TV display screen needs to be consistent with the display screen on the mobile phone; if the mirroring screen projection method is used, the user watches a picture for 5 seconds, and the mobile phone The display page of the current photo has been encoded hundreds of times, which obviously wastes the power consumption of the mobile phone. In this application, the first device generates corresponding control information (for example, first information) according to the user's operation requirements (for example, turning pages), and encodes the control information and sends it to the second device, instead of encoding all the contents of the current page of the first device (for example, control icons, etc.). Even if the first device sends the media resources to be displayed (for example, another picture) from the local to the second device, it only encodes the media resources without encoding the entire page of the first device. The second device only needs to decode the control information and the media resources to be displayed, without decoding a lot of irrelevant data. This can reduce the power consumption of the second device, thereby improving the user's screen projection experience.

[0022] In a possible implementation, the third information includes quantity information, index information, and coordinate information. The quantity information is used to indicate the number of pointers of the touch event, the index information is used to indicate the index of the touch event, and the coordinate information is used to indicate the coordinate data set of the touch event.

[0023] The first terminal indicates the event attributes of the touch event through quantity information, index information and coordinate information, which helps the projection side (for example, the second device) accurately determine the user touch operation corresponding to the touch event based on the quantity information, index information and coordinate information.

[0024] In one possible implementation, the coordinate information includes: identification information, first type information, first coordinate information, and second coordinate information, the identification information is used to indicate the pointer identifier of the touch event, the first type information is used to indicate the operation type corresponding to the touch event, the first coordinate information is used to indicate the horizontal coordinate of the touch event, and the second coordinate information is used to indicate the vertical coordinate of the touch event.

[0025] The first terminal further indicates the coordinate data set corresponding to the touch event through identification information, first type information, etc., which is beneficial for the projection side (for example, the second device) to further improve the accuracy of determining the user touch operation corresponding to the touch event based on the coordinate information.

[0026] In one possible implementation, the coordinate information also includes: second type information, when the first type information is used to indicate that the operation type corresponding to the touch event is a touch operation type, the second type information is used to indicate a floating operation type; or, when the first type information is used to indicate that the operation type corresponding to the touch event is a floating operation type, the second type information is used to indicate a touch operation type.

[0027] In some special scenarios, the coordinate information also includes second type information. The operation types indicated by the first type information and the second type information are different. Different operation types can be used to indicate the same touch event or different touch events, thereby meeting the needs of coordinate information in different scenarios to indicate different touch events of users.

[0028] In a possible implementation, the touch operation type includes a pressing action, a lifting action, and a moving action.

[0029] In a possible implementation, the touch operation type further includes an action mask.

[0030] In a possible implementation, the hover operation types include hover movement, hover entry, and hover exit.

[0031] In a possible implementation, the media resource is a picture.

[0032] In a possible implementation, the display mode is a zoom display mode, a page turning display mode, or a rotation display mode.

[0033] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, the memory being used to store computer programs, and the processor being used to call and run the computer programs from the memory, so that the electronic device executes any one of the methods in the first aspect.

[0034] In a fourth aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, the memory being used to store computer programs, and the processor being used to call and run the computer programs from the memory, so that the electronic device executes any one of the methods in the second aspect.

[0035] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes any one of the methods in the first aspect.

[0036] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor executes any one of the methods in the second aspect.

[0037] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, which, when executed by an electronic device, enables the electronic device to execute any one of the methods in the first aspect.

[0038] In an eighth aspect, an embodiment of the present application provides a computer program product, which includes: a computer program code, which, when executed by an electronic device, enables the electronic device to execute any one of the methods in the second aspect.

[0039] In the ninth aspect, an embodiment of the present application provides a chip system, which includes a processing circuit and a storage medium, wherein the storage medium stores computer program instructions; when the computer program instructions are executed by the processing circuit, any one of the methods in the first aspect is implemented.

[0040] Optionally, the processing circuit in the above chip system can be replaced by a processor, and the storage medium can be replaced by a memory. Optionally, the chip system can also include a communication interface, which is used to realize communication between the chip system and external devices.

[0041] In the tenth aspect, an embodiment of the present application provides a chip system, which includes a processing circuit and a storage medium, wherein the storage medium stores computer program instructions; when the computer program instructions are executed by the processing circuit, any one of the methods in the second aspect is implemented.

[0042] Optionally, the processing circuit in the above chip system can be replaced by a processor, and the storage medium can be replaced by a memory. Optionally, the chip system can also include a communication interface, which is used to realize communication between the chip system and external devices.

[0043] The beneficial effects of the technical solutions in the third to tenth aspects of the present application can refer to the beneficial effects of the technical solutions in the first or second aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] FIG1 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application;

[0045] FIG2 is a schematic diagram of a software structure of an electronic device 100 provided in an embodiment of the present application;

[0046] FIG3 is a schematic diagram of an application scenario applicable to the present application provided by an embodiment of the present application;

[0047] FIG4 is a schematic diagram of another application scenario applicable to the present application provided by an embodiment of the present application;

[0048] FIG5 is a flowchart of a method 500 for sending information and a corresponding method for receiving information provided in an embodiment of the present application;

[0049] FIG6 is a schematic diagram of a touch event picture projection scenario applicable to the present application, provided in an embodiment of the present application;

[0050] FIG7 is a schematic diagram of another touch event picture projection scenario applicable to the present application provided in an embodiment of the present application;

[0051] FIG8 is a schematic diagram of another touch event picture projection scenario applicable to the present application provided in an embodiment of the present application;

[0052] FIG9 is a schematic diagram of another touch event picture projection scenario applicable to the present application, provided in an embodiment of the present application;

[0053] FIG10 is a schematic diagram of another video projection scenario for touch events applicable to the present application, provided in an embodiment of the present application;

[0054] FIG11 is a schematic diagram of a user operating a touch panel 1003 according to an embodiment of the present application;

[0055] FIG12 is a schematic diagram of a video projection scenario for a touch event applicable to the present application, provided in an embodiment of the present application;

[0056] FIG13 is a schematic diagram of an audio screen projection scenario for a touch event applicable to the present application, provided in an embodiment of the present application;

[0057] FIG14 is a schematic diagram of a user operating a touch panel 1303 according to an embodiment of the present application;

[0058] FIG15 is a schematic diagram of a video projection scenario for a touch event applicable to the present application, provided in an embodiment of the present application;

[0059] FIG16 is a schematic diagram of a screen projection process between a first device and a second device provided in an embodiment of the present application;

[0060] FIG17 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0062] FIG1 shows a schematic structural diagram of an electronic device 100 .

[0063] The electronic device 100 may include at least one of a large screen, a projector, a screen projector, a television, a smart screen, a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), and a netbook. The embodiments of the present application do not impose any particular restrictions on the specific type of the electronic device 100.

[0064] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) connector 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a button 180, an indicator 181, a subscriber identification module (SIM) card interface 182, and a display screen 183, etc.

[0065] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those in FIG. 1 , or may combine or separate certain components, or may have different component arrangements. The components in FIG. 1 may be implemented in hardware, software, or a combination of software and hardware.

[0066] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0067] The processor 110 can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and execution.

[0068] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 may be a cache memory. This memory can store instructions or data that have been used or are frequently used by processor 110. When processor 110 needs to use the instruction or data, it can directly access it from this memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0069] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface. The processor 110 may be connected to modules such as a touch sensor, an audio module, a wireless communication module, a display, and a camera through at least one of the above interfaces.

[0070] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0071] The USB connector 130 is an interface that complies with USB standard specifications and can be used to connect the electronic device 100 and peripheral devices. The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display screen 183 and the wireless communication module 160. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0072] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0073] The mobile communication module 150 can provide wireless communication solutions, including 2G / 3G / 4G / 5G, for the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), etc. In some embodiments, at least some functional modules of the mobile communication module 150 may be provided in the same device as at least some modules of the processor 110.

[0074] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs sound signals through an audio device or displays images, documents, audio or video through the display screen 183. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0075] The wireless communication module 160 can provide wireless communication solutions for the electronic device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), near field communication (NFC), etc. In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other terminal devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), etc.

[0076] Electronic device 100 can implement display functions using a GPU, display screen 183, and an application processor. A GPU is a microprocessor for image processing that connects display screen 183 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0077] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage. For example, files such as music and videos can be stored on the external memory card or transferred from the terminal device to the external memory card.

[0078] The internal memory 121 can be used to store computer executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (for example, a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (for example, audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional methods or data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0079] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170 .

[0080] The buttons 180 may include a power button, a volume button (e.g., a user may use the volume button to adjust the volume of a broadcast message), etc. The buttons 180 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0081] The indicator 181 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate information, missed calls, notifications, etc.

[0082] Optionally, the terminal device may further include a SIM card interface 182 for connecting a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 182 or removing it from the SIM card interface 182. The electronic device 100 may support one or more SIM card interfaces 182. The SIM card interface 182 may support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards may be inserted into the same SIM card interface 182 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 182 may also be compatible with different types of SIM cards. The SIM card interface 182 may also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card may be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0083] The display screen 183 is used to display data such as the projection interface and the user operation interface. For example, the display screen 183 can be used to display information such as pictures and videos that the user needs to project. The display screen 183 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or more display screens 183. In some embodiments, the display screen may be a foldable or scroll-shaped display screen.

[0084] The electronic device 100 may also adopt other architectures, which are not limited in this application. In addition, the embodiment of the present invention may also use the Harmony system as an example to illustrate the software architecture of the electronic device 100. It should be understood that the solution provided in this application may also be applied to other types of operating systems such as the Android operating system, the Apple operating system, and the Windows operating system.

[0085] FIG2 is a schematic diagram of the software structure of the electronic device 100 according to an embodiment of the present application.

[0086] In some implementations, the Harmony system includes four layers, from bottom to top: the kernel layer, the system basic service layer, the framework layer, and the application layer.

[0087] The Harmony system uses a multi-kernel design, optionally including the Linux kernel, the Hongmeng microkernel, and the lightweight IoT operating system kernel (Lite Operating System, LiteOS). Through this design, devices with different device capabilities can choose the appropriate system kernel. The kernel layer also includes the Kernel Abstraction Layer, which provides basic kernel capabilities for other Harmony layers, such as process management, thread management, memory management, file system management, network management, and peripheral management.

[0088] The system basic service layer is the core capability set of the Harmony system, which supports the Harmony system to provide services to application services through the framework layer in the scenario of multi-device deployment. This layer optionally includes the following parts:

[0089] The system's basic capability subsystems provide fundamental capabilities for running, scheduling, and migrating distributed applications across multiple devices in the Harmony system. These subsystems comprise a distributed soft bus, distributed data and file management, distributed task scheduling, the Ark runtime, and distributed security and privacy protection. The Ark runtime provides a C / C++ / JavaScript multi-language runtime and basic system class libraries. It also provides a runtime for Java programs statically compiled using the Ark compiler (i.e., applications or frameworks developed in Java).

[0090] Basic Software Service Subsystem Set: This provides common, general-purpose software services for the Harmony system. It comprises subsystems such as graphics and imaging, distributed media, distributed AI, multimodal input, Mobile Sensing Development Platform (MSDP) & Device Virtualization (DV), event notification, phone services, and Design for X (DFX). The basic software service subsystem set can be tailored to the deployment environment of different device form factors, with each subsystem capable of being tailored to the granularity of functionality.

[0091] The enhanced software service subsystem set (see the enhanced software section outlined in the dashed line in Figure 2) provides the Harmony system with differentiated, device-specific, capability-enhancing software services. This subsystem consists of subsystems such as tablet business software, smart screen business software, vehicle-mounted computer business software, and Internet of Things (IoT) business software. The enhanced software service subsystem set can be tailored to the deployment environment of different device form factors at the subsystem level, and each subsystem can be tailored to the functional level within it.

[0092] Harmony Driver Foundation (HDF) and Hardware Abstraction Layer (HAL): They are the foundation of the open hardware ecosystem of the Harmony system, providing hardware capability abstraction to the hardware upward and providing a development framework and operating environment for various peripheral drivers downward.

[0093] Hardware Service Subsystem Set: This provides common, adaptive hardware services for the Harmony system and consists of hardware service subsystems such as general sensors, location, power, USB, and biometrics. The hardware service subsystem set can be tailored to the deployment environment of different device form factors, and each subsystem can be tailored to the functional granularity.

[0094] The proprietary hardware service subsystem (see the proprietary hardware section outlined by the dashed line in Figure 2) provides the Harmony system with differentiated hardware services for different devices. This subsystem optionally includes tablet proprietary hardware services, vehicle hardware services, wearable hardware services, and IoT hardware services. The proprietary hardware service subsystem can be tailored to the subsystem granularity, and each subsystem can be tailored to the functional granularity.

[0095] The framework layer provides the Harmony system's applications with a user program framework and meta-capability framework in multiple languages, such as Java / C / C++ / JavaScript, as well as a multi-language framework application programming interface (API) that opens up various software and hardware services.

[0096] The application layer includes system applications and third-party applications (or extended applications), including camera, gallery, calendar, call, drawing, navigation, WLAN, music, video, short messaging, and other applications. Applications in the Harmony system are built based on atomic capabilities (AA) and feature capabilities (FA).

[0097] Before introducing the method for sending information provided by the present application, a brief description of the execution subject involved in the embodiment of the present application is first given. The first device, the second device, or the third device involved in the present application may also have the structure shown in Figures 1 and 2. For example, the first device may be a terminal device having the structure shown in Figures 1 and 2, such as a mobile phone, a tablet, a laptop computer, etc., or a chip, a chip system, or a processor applied to the first device, or a logical node, a logical module, or software that can realize all or part of the functions of the first device; the second device may also be a terminal device having the structure shown in Figures 1 and 2, such as a TV, a smart screen, a car-mounted central control screen, a projector, etc., or a chip, a chip system, or a processor applied to the second device, or a logical node, a logical module, or software that can realize all or part of the functions of the second device; the third device may also be a terminal device or server having the structure shown in Figures 1 and 2, such as a data service terminal, a data storage server, etc.

[0098] Below, taking a first device (e.g., a mobile phone) and a second device (e.g., a television) having the structures shown in Figures 1 and 2 as examples, combined with the method for sending information and the method for receiving information provided in this application, the workflow of the software system and hardware system of the above-mentioned electronic device 100 is exemplified.

[0099] Figure 3 shows an application scenario applicable to the present application. As shown in (a) of Figure 3, the mobile phone includes a display screen, on which various application icons, application interfaces, etc. can be displayed; for example, the current display screen 301 displays a picture display interface 302; the picture display interface 302 displays a picture A in the gallery; if the user wants to cast picture A to the TV for viewing, the user can click the shortcut icon of the screen casting application 303 in the lower right corner of the picture display interface 302, and the mobile phone will respond to the user's click operation and generate a screen casting start request; the screen casting application 303 sends the screen casting start request to the event notification management service subsystem of the framework layer through the application layer; the event notification management service subsystem generates a screen casting start event based on the received screen casting start request, and then sends the screen casting start event to the screen casting driver module of the Hongmeng driver framework layer through the hardware abstraction layer; at this time, the screen casting driver module generates a screen casting start instruction based on the screen casting start event, and sends the screen casting start instruction to the hardware driver module; the hardware driver module starts the screen casting application 303 based on the screen casting start instruction. After the screen casting application 303 is started, the mobile phone switches from the picture display interface 302 to the screen casting application interface 304, as shown in (b) of Figure 3; the screen casting application 303 can search for the TV wirelessly and establish a screen casting connection with the TV; after the connection is successfully established, the picture A on the picture display interface 302 will be displayed on the TV display screen 305, and the user can view the picture A on the TV and browse other pictures in the gallery, as shown in (c) of Figure 3. In addition, the user can also perform operations such as zooming in and out on the picture A through touch gestures, and generate the field information (for example, the first information below) defined by the protocol for the operation data (or operation instruction data) corresponding to the touch gesture operation to the TV. Correspondingly, after receiving the field information, the TV will perform operations such as zooming in and out on the picture A according to the operation instructions (for example, zoom instructions) indicated by the field information. For details, please refer to the interface embodiment below, which will not be repeated here.

[0100] It should be noted that the following embodiments of this application will take a mobile phone and a television having the structures shown in Figures 1 and 2 as examples, and in combination with the accompanying drawings and application scenarios, specifically describe the method for sending information and the corresponding method for receiving information provided in the embodiments of this application. It should be understood that the first device or the second device to which this method is applied is not limited to the hardware and software structures shown in Figures 1 and 2. In actual applications, the hardware and software system architectures shown in Figures 1 and 2 can be modified according to specific application scenarios, and this application does not limit this.

[0101] Next, still taking the mobile phone and the television with the structures shown in FIG. 1 and FIG. 2 as an example, in combination with different embodiments, the method for sending information and the corresponding method for receiving information provided by the present application are described in detail.

[0102] As can be seen from the background technology, in the screen projection scenario, the mirror projection technology can help users completely copy the media resources (such as documents, pictures, audio and video, etc.) on devices such as mobile phones or tablets to the projection end (such as TV or smart screen), thereby obtaining a cross-screen experience; however, this mirror projection method will not only increase the power consumption of terminal devices such as mobile phones or tablets, but also affect the user's screen projection experience. Therefore, this application proposes a method for sending information and a corresponding method for receiving information; this method can reduce the power consumption of terminal devices (such as the first terminal or the second terminal) and improve the screen projection experience.

[0103] Before introducing the method of sending information and the corresponding method of receiving information, a brief introduction to the possible screen projection scenarios of this application is first given; the screen projection scenarios involved in this application can be one-to-one screen projection or one-to-many screen projection, and the screen projection devices can project screens to each other; for example, as shown in Figure 3, the screen projection between a mobile phone and a TV can be regarded as a one-to-one screen projection; for another example, as shown in Figure 4, a mobile phone can simultaneously project screens to a TV, a tablet, and a notebook, which can be regarded as a one-to-many screen projection; in addition, a mobile phone can project screens to a tablet, and a tablet can also project screens to a mobile phone; a notebook can project screens to a tablet, and a tablet can also project screens to a notebook, and so on.

[0104] It should be noted that the following embodiments only take the one-to-one screen projection of a first device (e.g., a mobile phone) to a second device (e.g., a TV) as an example to illustrate the method of sending information and the corresponding method of receiving information proposed in this application, and should not be understood as a limitation on the application scenarios of this application.

[0105] As shown in FIG5 , a flowchart of a method 500 for sending information and a corresponding method for receiving information provided in an embodiment of the present application is shown. The method 500 includes step 501 and step 502 , which are described in detail below.

[0106] In step 501, the first device generates first information, which includes second information and third information. The second information is used to indicate a touch event of the first device, and the third information is used to indicate an event attribute corresponding to the touch event. The touch event and the event attribute are used to indicate a display method of a media resource. The media resource is a resource obtained by the second device from the first device, or the media resource is a resource obtained by the second device from the third device.

[0107] The manner in which the first device and the second device establish a screen projection connection includes but is not limited to a wireless manner or a wired manner; a screen projection connection can also be established through a local area network. This application does not limit the manner in which the first device and the second device establish a screen projection connection. In actual application, it can be selected according to specific circumstances. The wireless manner includes but is not limited to Bluetooth or wireless fidelity (Wi-Fi), and the wired manner includes but is not limited to a data cable. For example, the first device and the second device establish a screen projection connection via Bluetooth; for another example, the first device and the second device establish a screen projection connection via a local area network.

[0108] The first device is a device for playing media resources and sending media resources; the first device can be alternatively described as a projected end or a source end; the first device includes but is not limited to a mobile phone, a tablet or a notebook.

[0109] The second device is a device that receives media resources and displays (or broadcasts) media resources; the second device can be alternatively described as a screen projection terminal or a delivery terminal; the second device includes but is not limited to a television, a projector, or a smart screen. The second device can obtain media resources (e.g., media resource links) from the first device (e.g., a local photo library) or a third device (e.g., a data storage server), and determine the display method of the media resources based on the touch event and the corresponding event attributes included in the first information.

[0110] The third device is a data server that stores media resources. For example, the third device includes but is not limited to a data storage server or a cloud server.

[0111] The first information is a control information (or control data or operation instruction data) indicating the display method of the media resource. The first information can also be expressed as operation information, manipulation information, touch information, or touch control information, etc. The first information is a set of control operation specification fields defined by a communication protocol (for example, a stream protocol). For example, the first information includes but is not limited to the fields in Tables 1 to 3 below. The first device uses the first information to send the user's touch operation on the media resource captured by the first device to the second device, so that the second device can respond to the touch operation of the first device. Among them, the touch operation can also be described as a touch gesture operation.

[0112] The above-mentioned media resources refer to the playback resources of the first device projected on the second device; the media resources can also be described alternatively as broadcast resources, delivery resources, or projection resources, etc.

[0113] The above media resources include but are not limited to documents, pictures, audio and video.

[0114] The above-mentioned display methods are used to indicate the display method of the media resource. For example, when the media resource is an image, the above-mentioned display methods are zoom display method, page turning display method, or rotation display method. Among them, the zoom display method is used to display the image after zooming; for example, the user directly performs a two-finger pinching operation on the display screen of the first device to reduce the current image and display it; for another example, the user directly performs a two-finger pinching operation on the display screen of the first device to enlarge the current image and display it. For details, please refer to the interface examples below.

[0115] The page-turning display method is used to display the previous or next picture. For example, the user switches from the current picture to the next picture by directly performing a quick left swipe operation on the display screen of the first device; for another example, the user switches from the current picture to the previous picture by directly performing a quick right swipe operation on the display screen of the first device. For details, please refer to the interface embodiment below.

[0116] The rotation display method is used to rotate the image at different angles. For example, the user can directly perform a counterclockwise rotation on the display screen of the first device to flip the current image to a certain angle (for example, 90°) for display; the user can directly perform a clockwise rotation on the display screen of the first device to flip the current image to a certain angle (for example, 180°) for display. For details, please refer to the interface embodiment below.

[0117] The above-mentioned touch event can also be described as a touch gesture event instead, and the touch event can include a contact event and a hovering event, wherein the contact event can also be expressed as a contact gesture event, and the hovering event can also be expressed as a hovering gesture event; in some cases, the touch event can be a multi-touch motion event, for example, the user uses a two-finger pinch operation to zoom in and out of the picture, wherein the touch event corresponding to the two-finger pinch operation is a multi-touch event, that is, two fingers contacting the screen correspond to two contact points; wherein, the contact event can refer to an event generated by a user gesture directly performing a touch operation on the display screen of the first device, and the hovering event can refer to an event generated by a user gesture performing a hover operation at a preset distance on the display screen of the first device when a first condition is met, wherein the first condition is that the vertical distance between the user gesture and the display screen of the first device is greater than 0 and less than (or equal to) a preset distance, wherein the preset distance can be 1 cm or 2 cm, etc.

[0118] The user's gesture operation includes, but is not limited to, a pinch operation, a slide operation, a tap operation, a rotation operation, or a press operation. It should be noted that the gesture operation can be in direct contact with the display screen of the first device or at a preset distance (e.g., 1 cm); when the gesture operation is in direct contact with the display screen of the first device, it is a touch gesture operation, and a contact event will be generated; when the gesture operation is at a preset distance from the display screen of the first device, it is a hover gesture operation, and a hover event will be generated.

[0119] For example, taking the gesture operation of direct contact with the display screen of the first device as an example, the pinch operation may refer to the pinching of multiple fingers (for example, two fingers), for example, the user zooms in and out of the picture on the display screen of the first device by pinching two fingers; the sliding operation may refer to a slow sliding (or fast sliding) operation formed by sliding the finger left and right (or up and down), for example, the user quickly slides left on the display screen of the first device to view the next picture; the tapping operation may refer to the operation generated by tapping the display screen of the first device with the finger, for example, the user double-clicks on the display screen of the first device to pause the video playback; the rotation operation may refer to the operation generated by rotating the finger clockwise or counterclockwise on the display screen of the first device, for example, the user flips the picture by rotating the finger counterclockwise; the pressing operation may refer to the operation generated by long pressing or short pressing the finger on the display screen of the first device for a preset time (for example, 2 seconds), for example, the user can select the current projected picture by long pressing the finger (for example, index finger) on the display screen of the first device for 2 seconds.

[0120] In one example, a user performs a touch gesture (or a hover gesture) on the display screen of a first device, and the first device obtains operation data corresponding to the touch gesture; the first device may generate first information based on the operation data; the first information includes second information and third information, wherein the second information is used to indicate that the operation data obtained by the first device corresponds to a touch event, rather than other non-touch events (such as mouse events); and the third information is used to further describe the event attributes of the touch event. The event attributes may refer to a description of the touch event, such as the number of pointers and the set of coordinates included in a touch event.

[0121] In one possible implementation, the first information includes second information and third information. The second information may be an action (ACTION) field, which is used to indicate sending a touch command (sendTouchCommand); the third information includes quantity information, index information, and coordinate information. The quantity information is used to indicate the number of pointers in the touch event, for example, the quantity information may refer to the pointer number (numPointers) field in Table 1; the index information is used to indicate the index of the touch event, for example, the index information may refer to the index (index) field in Table 1; and the coordinate information is used to indicate the coordinate data set of the touch event, for example, the coordinate information may refer to the pointer coordinate (pointerCoord) field in Table 1. It should be noted that the various fields included in the first information may also be referred to as operation control specification fields.

[0122] For another example, as shown in Table 1, the third information is the data (DATA) field; the data (DATA) field is used to indicate the pointer number (numPointers) field, the index (index) field, and the pointer coordinate (pointerCoord) field, wherein the three fields of numPointers, index, and pointerCoord are used to describe the event attributes corresponding to the touch event (e.g., contact event or hover event). The meaning of each field in the first information can be referred to the description of Table 1. The type field in Table 1 is used to describe the data types corresponding to numPointers, index, and pointerCoord respectively; and the description column is an explanation of the respective meanings of numPointers, index, and pointerCoord; the optional / mandatory column is an indication of whether numPointers, index, and pointerCoord must each be present in the action (ACTION) field.

[0123] Table 1

[0124] It should be noted that Table 1 shows a data structure of the first information (or shows the various fields included in the first information). In actual applications, the content of each field in the first information can also be adjusted and expanded as needed.

[0125] In this embodiment, the first device indicates the event attributes of the touch event through quantity information, index information and coordinate information, which is beneficial for the second device (for example, the projection end) to accurately determine the gesture touch operation corresponding to the touch event based on the quantity information, index information and coordinate information.

[0126] In another possible implementation, the coordinate information includes: identification information, first type information, first coordinate information, and second coordinate information; for example, the coordinate information may refer to the pointer coordinates (PointerCoord) in Table 2; the identification information is used to indicate the pointer identification of the touch event, for example, the identification information may refer to the identification (id) field in Table 2; the coordinate information also includes action information, for example, the action information may refer to the action (action) field in Table 2, the action (action) field is used to indicate a control action (ControlAction), the control action (ControlAction) may indicate different operation types, for example, a touch operation type or a floating operation type, wherein the touch operation type may refer to pressing Action, move action, etc., the hover operation type can refer to hover movement, etc.; the action information includes: first type information, the first type information is used to indicate the operation type corresponding to the touch event, for example, the first type information can refer to the content indicated by the control action (ControlAction) field in Table 2, for example, in different situations, the first type information can indicate the touch operation type, or the first type information can indicate the hover operation type; first coordinate information is used to indicate the horizontal coordinate of the touch event, for example, the first coordinate information can refer to the X coordinate (coordinateX) field in Table 2; second coordinate information is used to indicate the vertical coordinate of the touch event, for example, the second coordinate information can refer to the Y coordinate (coordinateY) field in Table 2. The type field in Table 2 is used to describe the data types corresponding to id, action, coordinateX, and coordinateY respectively; the description column is an explanation of the meaning of id, action, coordinateX, and coordinateY respectively; the optional / mandatory column indicates whether id, action, coordinateX, and coordinateY must each be present in the pointer coordinate field.

[0127] Table 2

[0128] In this embodiment, the first device further indicates the coordinate data set corresponding to the touch event through identification information, first type information, etc., which is beneficial for the second device (for example, the projection end) to further improve the accuracy of determining the user touch operation corresponding to the touch event based on the coordinate information.

[0129] In another possible implementation, the coordinate information also includes: second type information, when the first type information is used to indicate that the operation type corresponding to the touch event is a touch operation type, the second type information is used to indicate a floating operation type; or, when the first type information is used to indicate that the operation type corresponding to the touch event is a floating operation type, the second type information is used to indicate a touch operation type.

[0130] Among them, the coordinate information also includes: second type information, which can be understood as the coordinate information including the second type information in addition to the first type information. When the operation types corresponding to the touch event are different, the contents indicated by the first type information and the second type information are different. Typically, a user's touch gesture operation directly contacts the display screen of the first device, while a hovering gesture operation is suspended at a preset distance from the display screen of the first device. Therefore, when the first device generates coordinate information based on the user's gesture operation (e.g., a touch gesture operation or a hovering gesture operation), it can generate corresponding coordinate information based on whether the gesture operation contacts the screen. For example, when the user's gesture operation contacts the display screen of the first device, it indicates that the gesture operation is a touch gesture operation, and a contact event is generated. Accordingly, the operation type corresponding to the contact event in the touch event is a touch operation type. At this time, when the first device generates coordinate information, it can indicate the user's gesture operation corresponding to the touch gesture operation through the field included in the first type of information, and the field included in the second type of information can be left blank. When the user's gesture operation is at a preset distance from the display screen of the first device, it indicates that the gesture operation is a hovering gesture operation, and a hover event is generated. Accordingly, it indicates that the operation type corresponding to the hover event in the touch event is a hover operation type. When the first device generates coordinate information, it can indicate the user's gesture operation corresponding to the hovering gesture operation through the field included in the second type of information, and the field included in the second type of information can be left blank.

[0131] For example, the second type of information may also refer to the content indicated by the control action (ControlAction) field in Table 2; for example, the second type of information may indicate a touch operation type, or the second type of information may indicate a hovering operation type.

[0132] In some special scenarios, the coordinate information also includes second type information. The operation types indicated by the first type information and the second type information are different. Different operation types can be used to indicate the same touch event or different touch events, thereby meeting the needs of coordinate information in different scenarios to indicate different touch events of users.

[0133] In a possible implementation, the touch operation type includes a pressing action, a lifting action, and a moving action.

[0134] The down action may be a touch operation when the user's gesture contacts the display screen of the first device; for example, when the user places an index finger on the display screen of the first device, it is a down action, and accordingly, a down event is generated.

[0135] An up action may refer to an operation when the user's gesture leaves the display screen of the first device; for example, when the user lifts a finger placed on the display screen of the first device, it is a up action, and accordingly, an up event is generated.

[0136] A move action can refer to a touch operation performed by a user's gesture on the display screen of the first device. For example, a user sliding a finger leftward (or rightward, or upward, or downward) on the display screen of the first device is a move action, and accordingly, a move event is generated. It should be noted that a move action can also be expressed as a slide operation or a drag operation.

[0137] Generally, the three operations of pressing (down), lifting (up) and moving (move) are a series of operations, and accordingly, a series of events will be generated; for example, when a user slides a finger on the display screen of the first device (for example, swiping left or right), the first device will trigger three events, event 1 is a down event, and the corresponding touch operation type is a press action; event 2 is a move event, and the corresponding touch operation type is a move action; event 3 is an up event, and the corresponding touch operation type is a lift action.

[0138] For example, as shown in Table 3, the press (down) action can be indicated by the remote control press action (REMOTECONTROL_ACTION_DOWN) field in the control action (ControlAction) field; the lift (up) action can be indicated by the remote control lift action (REMOTECONTROL_ACTION_UP) field in the control action (ControlAction) field; the move (move) action can be indicated by the remote control move action (REMOTECONTROL_ACTION_MOVE) field.

[0139] Table 3

[0140] In yet another possible implementation, the touch operation type further includes an action mask.

[0141] It should be noted that action masking can be alternatively described as action concealment or action disguising.

[0142] Among them, the action mask (ACTION MASK) is used for multi-touch operations. For example, when the user's multi-finger gesture (such as a two-finger pinch gesture) is pressed or lifted on the display screen of the first device, an ACTION MASK event is generated corresponding to a multi-touch operation.

[0143] For example, as shown in Table 3, the action mask (ACTION MASK) may be indicated by the remote control action mask (REMOTECONTROL_ACTION_MASK) field in the control action (ControlAction) field.

[0144] In another possible implementation, the hover operation types include hover movement, hover entry, and hover exit.

[0145] Among them, hover movement (HOVER MOVE), hover entry (HOVER ENTER) or hover exit (HOVER EXIT) refers to the operation performed when the user's hover gesture is at a preset distance from the display screen of the first device; for example, taking the user's gesture as an example of being 1 cm above the display screen of the first device, when the user's hand moves from the area outside the boundary of the display screen of the first device to the boundary of the display screen, a HOVER ENTER event will be generated; for another example, when the user's hand continues to move from the boundary of the display screen of the first device to the internal area of ​​the display screen, a HOVER MOVE event will be generated; for another example, when the user's hand moves out of the display screen area of ​​the first device to the boundary of the display screen, a HOVER EXIT event will be generated.

[0146] Normally, the three operations of hover movement, hover entry and hover exit are a series of operations, and accordingly, a series of events will be generated; for example, when the user's palm makes a slow upward swipe operation 1 cm above the display screen of the first device, the first device will detect 3 events, event 1 is a hover entry event, corresponding to the action of the user's palm hovering and moving; event 2 is a hover movement event, corresponding to the action of the user's palm hovering and moving on the display screen; event 3 is a hover exit event, corresponding to the action of the user's palm hovering and exiting. You can refer to the interface implementation examples below for understanding.

[0147] For example, as shown in Table 3, HOVER MOVE can be indicated by the remote control HOVER MOVE action (REMOTECONTROL_ACTION_HOVER_MOVE) field in the control action (ControlAction) field; HOVER ENTER can be indicated by the remote control HOVER ENTER action (REMOTECONTROL_ACTION_HOVER_ENTER) field in the control action (ControlAction) field; and HOVER EXIT can be indicated by the remote control HOVER EXIT action (REMOTECONTROL_ACTION_HOVER_EXIT) field.

[0148] In one embodiment, the first device can send the data of the floating operation type and the data of the touch operation type together to the second device; for example, the user's gesture is first in a floating state, and the gesture first floats from the area outside the display screen A of the first device to the boundary of the display screen A. At this time, a floating move event 1 is generated; the gesture then floats from the boundary of the display screen A to the area of ​​the display screen A, and a floating move event 2 is generated; the gesture (for example, a finger click) falls within the area of ​​the display screen A and contacts the display screen A, and a down event 3 is generated; the gesture falls within the area of ​​the display screen A and contacts the display screen A and moves (for example, sliding to the left), and a move event 4 is generated; the gesture moves within the area of ​​the display screen A (for example, sliding to the left) and then lifts up, and an up event 5 is generated; the gesture is within the area of ​​the display screen A. After being lifted up, it moves to the boundary of display screen A and then leaves the boundary of display screen A. At this time, a floating movement event 6 and a floating exit event 7 are generated; for down event 3, the corresponding pressing (down) action can be indicated by the remote control pressing action (REMOTECONTROL_ACTION_DOWN) field in the control action (ControlAction) field; up event 5 can indicate the corresponding lifting (up) action by the remote control lifting action (REMOTECONTROL_ACTION_UP) field in the control action (ControlAction) field; move event 4 can indicate the corresponding moving (move) action by the remote control moving action (REMOTECONTROL_ACTION_MOVE) field. Hover move event 2 (or hover move event 6) can indicate the corresponding hover move (HOVER MOVE) action through the remote control hover move action (REMOTECONTROL_ACTION_HOVER_MOVE) field in the control action (ControlAction) field; hover enter event 1 can indicate the corresponding hover enter (HOVER ENTER) action through the remote control hover enter action (REMOTECONTROL_ACTION_HOVER_ENTER) field in the control action (ControlAction) field; hover exit event 7 can indicate the corresponding hover exit (HOVER EXIT) action through the remote control hover exit action (REMOTECONTROL_ACTION_HOVER_EXIT) field.

[0149] Step 502: The first device sends the first information to the second device, and correspondingly, the second device receives the first information from the first device. After the first device generates the first information, it sends the first information to the second device. After the second device receives the first information, it switches the display mode of the media resource currently being projected according to the first information; for example, if the operation instruction indicated by the first information is to reduce the image, the second device will reduce the currently projected image and display it to the user according to the operation instruction indicated by the first information; if the operation instruction indicated by the first information is to view the next image, the second device obtains the next image from the first device (for example, a local image library) or from a third device (for example, a data storage server), and determines to replace the currently displayed image with the next image based on the first information. For details, please refer to the interface embodiment below, which will not be repeated here.

[0150] It should be noted that, after step 502, the above method 500 further includes: the second device displays the media resource according to the display method in response to the first information.

[0151] Among them, after the second device receives the first information, it injects the first information into the operating system inside the second device; the operating system switches (or updates) the display mode of the current media resource according to the operation instruction indicated by the first information, that is, through the internal system and application capabilities of the second device, the second device realizes system-level operation of the current media resource, and then completes the complex control of the remote broadcast resources. For example, the operation instruction indicated by the first information is to rotate the current picture 180° and then display it. At this time, the operating system of the second device will rotate the currently displayed picture 180° and display it to the user.

[0152] To sum up, in the existing mirroring screen projection method, in order to ensure the synchronization of the images of the first device (for example, a mobile phone) and the second device (for example, a TV), the encoder of the first device needs to encode all the content on the display page (that is, all the content on the virtual screen) in real time (for example, encode once every ten or so milliseconds) and send the encoding result to the second device. Correspondingly, the second device also needs to decode a lot of data; because in some cases the content on the display page of the first device will not change in a short time, for example, the user projects the photos in the mobile phone gallery to the TV for viewing. In most cases during browsing, the user will not turn the page frequently, but will turn to the next one after a period of time (for example, 5 seconds). At this time, the TV display screen needs to be consistent with the display screen on the mobile phone; if the mirroring screen projection method is adopted, When a user watches a picture for 5 seconds, the mobile phone has encoded the display page of the current photo hundreds of times. Obviously, this wastes the power consumption of the mobile phone. In this application, the first device generates corresponding control information (for example, first information) according to the user's operation requirements (for example, turning pages), and encodes the control information and sends it to the second device, instead of encoding all the contents of the current page of the first device (for example, control icons, etc.). Even if the first device sends the media resources to be displayed (for example, another picture) from the local to the second device, it only encodes the media resources without encoding the entire page of the first device. Accordingly, the second device does not need to decode a lot of data, thereby reducing the power consumption of the first and second devices, and improving the system performance of the terminal and the user's screen projection experience.

[0153] The above describes in detail method 500 and the corresponding method for receiving information, as well as possible implementation methods. Below, taking the first device as a mobile phone and the second device as a TV as an example, combined with interface implementation examples of different application scenarios, method 500 and possible implementation methods are further explained.

[0154] Figure 6 shows a picture projection scenario for a touch event applicable to the present application; as shown in (a) in Figure 6, the current mobile phone and the TV have successfully established a projection connection, and the mobile phone and the TV are in the projection state; at this time, the picture display interface 302 is displayed on the current display screen 301 of the mobile phone; the picture display interface 302 displays a picture A in the gallery; if the user wants to reduce the display of the currently projected picture A, the user can touch the picture display interface 302 with two fingers to perform a two-finger pinch touch operation 601. At this time, the picture A on the interface 302 changes from a normal state to a reduced state 602, as shown in (b) in Figure 6; at the same time, the mobile phone will obtain operation data corresponding to the two-finger pinch touch operation 601 from the interface 302, and the operation data includes the operation type (which can indicate a two-finger pinch touch operation 601), the number of finger contact points (for example, two-finger contact corresponds to 2 contact points), the coordinate position of the finger movement and other data; the mobile phone can process the operation data according to the data format of the first information (see Tables 1 to 3 above) to generate the first information. a message; for example, the action (ACTION) field of the first message indicates sending a touch instruction, and the sending touch instruction may indicate that the first message sent by the second device is control information of a touch event; the number of pointers (numPointers) in the data (DATA) field of the first message may indicate that the touch operation 601 is a multi-touch motion event and the number of pointers is 2; the index (index) field may indicate the order of pointers corresponding to the multi-touch motion event; the pointer coordinate (pointerCoord) field may indicate coordinate data corresponding to the multi-touch motion event; wherein the pointer coordinate field may carry information such as an identifier (id), an action (action), an X coordinate, and a Y coordinate, wherein the control action indicated by the action (action) field may be indicated by the DATA field in Table 3 (for example, fields such as an action mask); the data values ​​specifically indicated by different fields may be matched to the various fields in Tables 1 to 3 according to the operation meaning corresponding to the operation data in the actual application scenario, thereby generating the first message. After the mobile phone generates the corresponding first information (or control information) based on the data structure of the first information, it sends the first information to the television. After receiving the first information, the television parses the first information and injects it into its internal operating system. The internal operating system determines, based on the data structure of the first information, that the operation instruction (or operation instruction data or operation control data) indicated by the first information is a zoom-out instruction. Based on the zoom-out instruction, the internal operating system controls the television display screen 305 to update the currently displayed image A from the normal state 603 shown in FIG6 (c) to the zoomed-out state 604 shown in FIG6 (d). It should be noted that the data structure of the first information can also be referred to as the various protocol fields of the first information.

[0155] Figure 7 shows another picture projection scenario of the touch event applicable to the present application; as shown in (a) in Figure 7, the current mobile phone and TV are in the projection state; a picture A in the gallery is displayed on the picture display interface 302 of the mobile phone; if the user wants to enlarge the picture A, the user can hover two fingers above the picture display interface 302 at a preset distance (for example, 1 cm) to perform a two-finger outward touch operation 701. At this time, the picture A on the interface 302 changes from the normal state to the enlarged state 702, as shown in (b) in Figure 7. For example, the user's finger moves from position A to the boundary of the mobile phone, and after entering the boundary of the mobile phone, the finger is moved to the position B to perform a two-finger outward touch operation 701. After that, the finger is moved from position B to position C, and then the finger is moved from position C to exit the boundary of the mobile phone. ; At the same time, the mobile phone obtains the operation data corresponding to the two-finger outward touch operation 701 from the interface 302, and the operation data includes the operation type (which can indicate the two-finger outward touch operation 701), the number of finger contact points (for example, two-finger contact corresponds to 2 contact points), the coordinate position of the finger movement and other data; the mobile phone can process the operation data according to the data format of the first information (see Tables 1 to 3 above) to generate the first information; for details, please refer to the process of the mobile phone generating the first information according to the two-finger pinch touch operation 601 above. The only difference is that the control action indicated by the action field can be indicated by the fields such as the remote control floating move action, the remote control floating move in action and the remote control floating exit action included in the DATA field in Table 3. After the mobile phone generates the corresponding first information (or control information) according to the data structure of the first information, it sends the first information to the TV; after receiving the first information, the TV parses the first information and injects it into the internal operating system. The internal operating system determines that the operation instruction indicated by the first information is an instruction to enlarge the picture according to the data structure of the first information, and then controls the picture A currently displayed on the TV display screen 305 to be updated from the normal state 603 shown in (c) of Figure 7 to the enlarged state 703 shown in (d) of Figure 7 according to the instruction to enlarge the picture.

[0156] Figure 8 shows another picture projection scenario applicable to the touch event of the present application; as shown in (a) in Figure 8, the current mobile phone and TV are in the projection state; a picture A in the gallery is displayed on the picture display interface 302 of the mobile phone; if the user wants to view the next picture, the user can touch the picture display interface 302 with a finger and perform a quick left-swipe touch operation 801. At this time, the picture A on the interface 302 becomes the next picture 802, as shown in (b) in Figure 8; at the same time, the mobile phone obtains the operation data corresponding to the left-swipe touch operation 801 from the interface 302, and the operation data includes the operation type (which can indicate the left-swipe touch operation 801), the number of finger contact points (for example, a single-finger contact corresponds to 1 contact point), the coordinate position of the finger sliding and other data; the mobile phone generates the first information, which can refer to the process of the mobile phone generating the first information based on the two-finger pinch touch operation 601 mentioned above. The mobile phone sends a first message to the TV; after receiving the first message, the TV parses the first message and injects it into the internal operating system. The internal operating system determines that the operation instruction indicated by the first message is a left-swipe-next-page instruction based on the data structure of the first message, and then controls the display screen 305 to update the currently displayed picture A from the screen 803 shown in (c) of Figure 8 to the screen 804 of the next picture 802 shown in (d) of Figure 8 according to the left-swipe-next-page instruction.

[0157] Figure 9 shows another picture projection scenario applicable to the touch event of the present application; as shown in (a) in Figure 9, the current mobile phone and TV are in the projection state; a picture A in the gallery is displayed on the picture display interface 302 of the mobile phone; if the user wants to rotate picture A 90° clockwise to view it, the user can touch the picture display interface 302 with a finger to perform a clockwise rotation touch operation 901. At this time, picture A on the interface 302 becomes picture 902 rotated 90°, as shown in (b) in Figure 9; at the same time, the mobile phone obtains operation data corresponding to the clockwise rotation touch operation 901 from the interface 302, and the operation data includes the operation type (which can indicate the clockwise rotation touch operation 901), the number of finger contact points (for example, single-finger contact corresponds to 1 contact point), the coordinate position of the finger sliding and other data; the mobile phone generates the first information for reference, and the process of the mobile phone generating the first information according to the two-finger pinch touch operation 601. The mobile phone sends the first information to the TV; after receiving the first information, the TV parses the first information and injects it into the internal operating system. The internal operating system determines that the operation instruction indicated by the first information is a clockwise rotation instruction of 90 degrees based on the data structure of the first information, and then controls the display screen 305 to update the currently displayed picture A from the screen 903 shown in (c) of Figure 9 to the screen 904 after the picture A is rotated 90° clockwise as shown in (d) of Figure 9 according to the clockwise rotation instruction of 90 degrees.

[0158] FIG10 shows another video screen casting scenario applicable to the touch event of the present application; as shown in FIG10 (a), a video playback interface 1001 is displayed on the display screen 301 of the mobile phone; a video is playing on the interface 1001; if the user wants to cast the video to the TV for viewing, the user can click the shortcut icon of the screen casting application 1002 in the lower right corner of the interface 1001, and the mobile phone will respond to the user's single click operation and start the screen casting application 1002; at this time, the interface 1001 changes to the touch panel 1003 interface shown in FIG10 (b); the user can operate the sound, playback progress and selection of the current video through touch gestures on the touch panel 1003. Correspondingly, the TV changes from the screen casting response interface 1004 shown in FIG10 (c) to the video playback interface 1005 shown in FIG10 (d).

[0159] Figure 11 shows a schematic diagram of a user operating the touchpad 1003; as shown in (a) of Figure 11, the user can switch the currently playing video to the previous (or next) video (for example, switch from the current episode to the next episode) by performing a left (or right) quick swipe touch operation 1101 on the touchpad 1003; as shown in (b) of Figure 11, the user can turn down (or increase) the volume of the current video by performing a multi-finger pinch touch operation (or a multi-finger spread touch operation) 1102 on the touchpad 1003; as shown in (c) of Figure 11, the user can adjust the playback progress of the current video by performing an up (or down) slow swipe touch operation 1103 on the touchpad 1003; as shown in (d) of Figure 11, the user can exit the current video application by performing a four-finger quick swipe touch operation 1104 on the touchpad 1003.

[0160] It should be noted that the operation data corresponding to the user's gesture touch operation on the touchpad 1003 can also be the first information generated according to the above Tables 1 to 3, which will not be repeated here. For details, please refer to the method of generating the first information in the above-mentioned related gesture touch operation embodiment.

[0161] Figure 12 shows a video projection scenario applicable to the touch event of the present application; as shown in (c) in Figure 12, the current mobile phone and TV are in the projection state; the video projected by the mobile phone on the display screen 305 is being played on the TV display screen 305; if the user wants to turn up the volume of the video, the user can touch the touchpad 1003 with a finger on the interface of the mobile phone's touchpad 1003 to perform a multi-finger expansion touch operation 1201, as shown in (a) in Figure 12. At this time, a volume adjustment control 1202 appears on the TV display screen 305, and the volume is adjusted from the original S1 position to the S2 position, as shown in (c) in Figure 12; at the same time, the mobile phone obtains operation data corresponding to the multi-finger expansion touch operation 1201 from the interface of the touchpad 1003, and the operation data includes the operation type (which can indicate the multi-finger expansion touch operation 1201), the number of finger contact points (for example, multi-finger contact corresponds to multiple contact points), the coordinate position of the finger movement and other data; the mobile phone generates the first information based on the operation data, which can refer to the process of the mobile phone generating the first information based on the two-finger pinch touch operation 601. The mobile phone sends the first information to the TV; the TV injects the received first information into the internal operating system, the internal operating system parses the first information, and determines the volume-up instruction indicated by the first information, and then controls the display screen 305 from the current sound position S1 to the position S2 according to the volume-up instruction.

[0162] For another example, as shown in (b) in FIG12 , the video projected from the mobile phone onto the display screen 305 is currently being played on the display screen 305 of the current TV; if the user wants to watch the next video, the user can touch the touchpad 1003 of the mobile phone with a finger on the interface of the touchpad 1003 to perform a quick right swipe touch operation 1203. At this time, the next level of video playback screen 1204 appears on the display screen 305 of the TV, as shown in (d) in FIG12 ; similarly, the mobile phone generates the first information based on the operation data corresponding to the quick right swipe touch operation 1203 and the TV responds to the first information. Please refer to the process of the mobile phone and the TV processing the two-finger pinch touch operation 601, which will not be repeated here.

[0163] Figure 13 shows an audio screen projection scenario suitable for touch events of the present application; as shown in (a) in Figure 13, an audio playback interface 1301 is displayed on the display screen 301 of the mobile phone; audio is being played on the interface 1301; if the user wants to cast the audio to the TV for listening, the user can click the shortcut icon of the screen projection application 1302 in the upper right corner of the interface 1301, and the mobile phone will respond to the user's single click operation and start the screen projection application 1302; at this time, the interface 1301 becomes the touchpad 1303 interface shown in (b) in Figure 13; the touchpad 1303 is divided into three areas, namely the left area 1304, the middle area 1305 and the right area 1306; the user can control different audio display methods by touching different areas of the touchpad 1303; the user can operate the volume, playback progress and song selection of the current audio playback through touch gestures in different areas on the touchpad 1003. Correspondingly, the TV changes from the screen projection response interface 1304 shown in (c) of FIG. 13 to the video playback interface 1305 shown in (d) of FIG. 13 .

[0164] Figure 14 shows a schematic diagram of a user operating the touchpad 1303; as shown in (a) of Figure 14, the user can increase (or decrease) the playback volume of the current audio by performing a two-finger outward (or pinching) touch operation 1401 on the left area 1304 of the touchpad 1303; as shown in (b) of Figure 14, the user can fast forward (or rewind) the current audio by performing a slow up (or down) touch operation 1402 on the right area 1306 of the touchpad 1303; as shown in (c) of Figure 14, the user can switch the current audio to the next (or previous) audio by performing a quick left (or right) swipe touch operation 1403 on the middle area 1305 of the touchpad 1303; as shown in (d) of Figure 14, the user can exit the current audio application by performing a four-finger quick swipe touch operation 1404 on the middle area 1305 of the touchpad 1303.

[0165] It should be noted that the operation data corresponding to the user's gesture touch operation on the touch panel 1303 can also be the first information generated according to the above Tables 1 to 3, which will not be repeated here. For details, please refer to the method of generating the first information in the above-mentioned related gesture touch operation embodiment.

[0166] FIG15 shows a video screen casting scenario applicable to the touch event of the present application; as shown in FIG15 (c), the current mobile phone and TV are in the screen casting state; the audio projected by the mobile phone on the screen 305 is being played on the TV display 305; if the user wants to fast forward the audio, the user can touch the right area 1306 of the touch pad 1303 of the mobile phone with a finger and perform a slow upward touch operation 1501, as shown in FIG15 (a); at this time, a progress adjustment control 150 appears on the TV display 305. 2. The playback progress is adjusted from the original K1 position to the K2 position, as shown in (c) of FIG15 . At the same time, the mobile phone obtains the operation data corresponding to the slow upward sliding touch operation 1501 from the interface of the touch panel 1303. The operation data includes the operation type (which can indicate the slow upward sliding touch operation 1501), the number of finger contact points (for example, a single point of contact corresponds to one contact point), the coordinate position of the finger movement, and other data. The mobile phone generates the first information based on the operation data, which can be referred to as the process of the mobile phone generating the first information based on the two-finger pinch touch operation 601. The mobile phone sends the first information to the television. The television injects the received first information into the internal operating system, which parses the first information and determines the fast-forward instruction indicated by the first information. Then, according to the fast-forward instruction, the display screen 305 is controlled to adjust from the current playback position K1 to the position K2, as shown in (c) of FIG15 .

[0167] For another example, as shown in (b) in FIG15 , the audio projected from the mobile phone onto the display screen 305 is being played on the display screen 305; if the user wants to watch the next video, the user can touch the touchpad 1303 with a finger in the middle area 1305 of the touchpad 1303 of the mobile phone and perform a quick right swipe touch operation 1503. At this time, the next level audio playback screen 1504 appears on the display screen 305, as shown in (d) in FIG15 ; similarly, the mobile phone generates the first information based on the operation data corresponding to the quick right swipe touch operation 1503 and the TV responds to the first information. Please refer to the process of the mobile phone and the TV processing the two-finger pinch touch operation 601, which will not be repeated here.

[0168] It should be noted that although the article uses a quick (or slow) left swipe (or right swipe) operation example to switch to the next picture (or video or audio), it should be understood that in actual applications, designers can use other gesture operations (such as pinching, rotation, and other operations) to achieve the switching of the next picture (or video or audio) according to actual conditions. Due to limited space, this application only uses a quick (or slow) left swipe (or right swipe) operation to illustrate the switching of the next picture (or video or audio). Similarly, touch gesture operations such as picture zooming or rotating, audio and video playback progress, etc. are not limited to the operations given in the previous embodiments. Other touch (or floating) gesture operations (such as selecting a sliding operation) can also be used to achieve audio and video playback progress. This application does not limit this.

[0169] It should also be noted that the method for sending information provided in this application is not limited to screen projection scenarios such as pictures, videos, or audio, but can also be applied to screen projection scenarios of other resources (such as documents, etc.). This application only introduces the method for sending information using screen projection scenarios such as pictures as an example, and should not be understood as a limitation on the application scenarios of this application.

[0170] For ease of understanding, the following describes the screen projection process between a first device (e.g., a mobile phone) and a second device (e.g., a TV) in conjunction with Figure 16. The first device and the second device have established a screen projection connection.

[0171] (1) A user performs a touch gesture operation (e.g., a right swipe touch operation) on a projected media resource (e.g., a picture) on a mobile phone display, and the hardware monitors the touch gesture operation event.

[0172] (2) The hardware reports the touch gesture operation event to the system server through the underlying operating system.

[0173] (3) The system service dispatches the operation event (e.g., touch gesture operation event) to the upper-layer Harmony OS ability package (HAP), and the HAP generates first information based on the operation data corresponding to the operation event.

[0174] (4) The HAP sends the first information to the screen projection service; the screen projection service sends the first information to the second device through the soft bus or the underlying network using the screen projection protocol (e.g., the stream protocol).

[0175] (5) After receiving the first information, the casting engine of the second device parses the first information according to the data structure of the first information.

[0176] (6) The delivery engine injects the fields obtained after parsing into the multimodal input module of the operating system inside the second device; the multimodal module is used to process the fields included in the first information and obtain the operation instruction indicated by the first information (for example, the right swipe operation instruction).

[0177] (7) The multi-mode module sends the operation instruction indicated by the first information (for example, the right-swipe operation instruction) to the interface synthesizer (surface flinger).

[0178] (8) The surface synthesizer instructs the hardware to update the display mode of the media resource (e.g., picture) currently displayed on the display screen of the second device (e.g., switch to the next picture) according to the operation instruction (e.g., right swipe operation instruction).

[0179] (9) The hardware (e.g., display screen) refreshes the display mode of the current media resource and displays it to the user.

[0180] The following example illustrates the architectural flow of the method for sending information and the corresponding method for receiving information proposed in this application from a module perspective:

[0181] 1. For the first device (for example, mobile phone)

[0182] The first device may include: a manipulation monitoring module and an application verification module, wherein the manipulation monitoring module is configured to monitor and obtain operation data corresponding to a user's gesture operation.

[0183] Application verification module: used to generate first information from operation data and send it to the second device through the soft bus.

[0184] 2. For the first device (for example, the screen projection terminal)

[0185] The first device may include: an operation injection module, a multi-mode module, and an operation notification module. The operation injection module is configured to inject the operation instruction indicated by the first information sent by the first device into the multi-mode module to implement system-level operations, perform rendering, and update the current projection interface.

[0186] Multi-mode module: used for processing the first information and determining the operation instruction according to the first information.

[0187] Operation notification module: used to report the monitored operation events (such as touch events) to the application, and the application completes the corresponding gesture operation performed by the first device end according to the specific operation event and the injected operation instruction.

[0188] The above describes in detail examples of the information sending method provided by this application. It is understood that, in order to implement the above functions, the first device (or second device) includes hardware structures and / or software modules corresponding to the respective functions. Those skilled in the art should readily appreciate that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a hardware-driven hardware manner depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application. This application may divide the information sending method into functional units based on the above method examples. For example, each function may be divided into separate functional units, or two or more functions may be integrated into a single unit. The integrated units may be implemented in either hardware or software functional units. It should be noted that the division of units in this application is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used.

[0189] FIG17 is a schematic diagram of the structure of an electronic device provided by the present application. The dotted lines in FIG17 indicate that the unit or module is optional. Electronic device 1700 can be used to implement the method described in the above method embodiment. Electronic device 1700 can be a terminal device, a server, or a chip (system).

[0190] The electronic device 1700 includes one or more processors 1701, which can support the electronic device 1700 to implement the method in the method embodiment corresponding to Figure 5. The processor 1701 can be a general-purpose processor or a special-purpose processor. For example, the processor 1701 can be a central processing unit (CPU). The CPU can be used to control the electronic device 1700, execute software programs, and process data of the software programs. The electronic device 1700 can also include a communication unit 1705 to implement signal input (reception) and output (transmission).

[0191] The electronic device 1700 may be a chip (system) including a memory and a processor, wherein the processor is configured to execute a computer program stored in the memory to implement the methods shown in the above embodiments.

[0192] The communication unit 1705 may be an input and / or output circuit of the chip (system), or the communication unit 1705 may be a communication interface of the chip (system), and the chip (system) may be a component of the electronic device 1700 .

[0193] For another example, the communication unit 1705 may be a transceiver of the electronic device 1700, or the communication unit 1705 may be a transceiver circuit of the electronic device 1700. The electronic device 1700 may include one or more memories 1702, on which a program 1704 is stored. The program 1704 can be executed by the processor 1701 to generate instructions 1703, so that the processor 1701 performs the method described in the above method embodiment according to the instructions 1703. Optionally, data may also be stored in the memory 1702. Optionally, the processor 1701 may also read data stored in the memory 1702. The data may be stored at the same storage address as the program 1704, or the data may be stored at a different storage address than the program 1704.

[0194] The processor 1701 and the memory 1702 can be provided separately or integrated together, for example, integrated on a system-on-chip (SOC) of a terminal device. The specific manner in which the processor 1701 executes the method for sending information can be found in the relevant description of the method embodiment.

[0195] It should be understood that each step of the above method embodiment can be completed by hardware logic circuits or software instructions in the processor 1701. The processor 1701 can be a CPU, a digital signal processor (DSP), a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0196] The present application also provides a computer program product that, when executed by a processor 1701, implements any method embodiment of the present application. The computer program product may be stored in a memory 1702, for example, a program 1704. The program 1704 undergoes preprocessing, compilation, assembly, and linking to be converted into an executable object file that can be executed by the processor 1701.

[0197] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, implements any method embodiment of the present application. The computer program may be a high-level language program or an executable target program.

[0198] The computer-readable storage medium is, for example, memory 1702. Memory 1702 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SynchLink DRAM, SLDRAM), and direct RAM bus random access memory (DRRAM).

[0199] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment and the technical effects produced can refer to the corresponding processes and technical effects in the aforementioned method embodiments, and will not be repeated here.

[0200] In several embodiments provided in this application, the disclosed systems, devices, and methods can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not executed. The device embodiments described above are merely schematic, and the splitting of units is only a logical function splitting. There may be other splitting methods in actual implementation, and multiple units or components may be combined or integrated into another system. In addition, the coupling between the units or the coupling between the components may be direct coupling or indirect coupling, and the above coupling includes electrical, mechanical or other forms of connection.

[0201] The above embodiments are intended only to illustrate the technical solutions of the present application and are not intended to limit the same. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the above embodiments or replace some of the technical features therein with equivalents, and that such modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application and are therefore intended to be included within the scope of protection of the present application.

[0202] Finally, the above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be covered by the scope of protection of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection of the claims.

Claims

1. A method for sending information, characterized in that: The method comprises: Generate first information, where the first information includes second information and third information, where the second information is used to indicate a touch event of the first device, and the third information is used to indicate an event attribute corresponding to the touch event, where the touch event and the event attribute are used to indicate a display method of a media resource, where the media resource is a resource obtained by the second device from the first device, or the media resource is a resource obtained by the second device from a third device; The first information is sent to the second device.

2. A method for receiving information, characterized in that: The method comprises: receiving first information from a first device, the first information including second information and third information, the second information being used to indicate a touch event of the first device, the third information being used to indicate an event attribute corresponding to the touch event, the touch event and the event attribute being used to indicate a display method of a media resource, the media resource being a resource obtained from the first device, or the media resource being a resource obtained from a third device; In response to the first information, the media resource is displayed according to the display mode.

3. The method according to claim 1 or 2, characterized in that: The third information includes quantity information, index information and coordinate information, wherein the quantity information is used to indicate the number of pointers of the touch event, the index information is used to indicate the index of the touch event, and the coordinate information is used to indicate a coordinate data set of the touch event.

4. The method according to claim 3, characterized in that The coordinate information includes: identification information, first type information, first coordinate information, and second coordinate information. The identification information is used to indicate the pointer identification of the touch event, the first type information is used to indicate the operation type corresponding to the touch event, the first coordinate information is used to indicate the horizontal coordinate of the touch event, and the second coordinate information is used to indicate the vertical coordinate of the touch event.

5. The method according to claim 4, characterized in that The coordinate information also includes: second type information, when the first type information is used to indicate that the operation type corresponding to the touch event is a touch operation type, the second type information is used to indicate a floating operation type; or, when the first type information is used to indicate that the operation type corresponding to the touch event is a floating operation type, the second type information is used to indicate a touch operation type.

6. The method according to claim 5, characterized in that The touch operation types include a pressing action, a lifting action, and a moving action.

7. The method according to claim 6, characterized in that The touch operation type also includes an action mask.

8. The method according to any one of claims 5 to 7, characterized in that The types of suspension operations include suspension movement, suspension entry and suspension exit.

9. The method according to any one of claims 1 to 8, characterized in that The media resource is a picture.

10. The method according to claim 9, characterized in that The display mode is a zoom display mode, a page turning display mode or a rotation display mode.

11. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method described in any one of claims 1, 3 to 10, or the electronic device executes the method described in any one of claims 2 to 10.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the method according to any one of claims 1, 3 to 10, or the processor executes the method according to any one of claims 2 to 10.

13. A chip system, characterized in that: The chip system includes a memory and a processor, and the processor is configured to execute a computer program stored in the memory to implement the method as claimed in any one of claims 1, 3 to 10, or to implement the method as claimed in any one of claims 2 to 10.

Images