A display content method, electronic device and system

Abstract

Embodiments of the present application provide a display content method, an electronic device and a system, and relate to the technical field of terminals. A first electronic device displays a screen projection interface of a second electronic device, a first position of the screen projection interface displays a first window of a first application running on the second electronic device; the first electronic device receives a window dragging operation of a user on the first window; and the first electronic device displays a second window of the first application locally drawn at a second position on the screen projection interface, wherein the second position is any position on a dragging track generated by the window dragging operation. The first electronic device has a smaller time delay in locally drawing a window image of an application, can reduce the time delay of displaying a window of an application on the first electronic device, and improves user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a method, electronic device and system for displaying content. Background Technology

[0002] Screen mirroring is a technology that uses wired or wireless connections to synchronously display content between electronic devices, enabling multiple electronic devices to work together. It is widely used in entertainment, office, and other scenarios. Screen mirroring includes mirroring and extended mirroring. Mirroring refers to projecting the display screen of the source device onto the receiving device for display, with both the source and receiving devices displaying the same screen. Extended mirroring refers to projecting the display screen of the source device onto the receiving device for display, with the source device not displaying its own screen.

[0003] Using non-touchscreen electronic devices such as personal computers (PCs) and large screens as source devices, and touchscreen electronic devices such as tablet computers and digital displays as receiving devices, the screen of the non-touchscreen electronic device is projected onto the touchscreen electronic device through methods such as mirroring or extended projection; the touchscreen electronic device receives user input and then transmits the corresponding signals to the non-touchscreen electronic device; thus, it is possible to control the non-touchscreen electronic device through the touchscreen electronic device.

[0004] For example, PCs have strong computing power, meeting the performance requirements of professional graphics software, and many professional graphics software programs run on PCs. However, PCs generally do not have touchscreens, and users typically control the PC using input devices such as keyboards and mice. The experience of drawing on a PC using a mouse and keyboard is poor. Tablets have touchscreens, but many professional graphics software programs do not support running on tablets. On tablets running operating systems such as [insert operating system name here], the tablet can be used as an input device for a PC. Users can then use a stylus or their fingers to input on the tablet, enabling them to draw, write, and perform other tasks on their PC.

[0005] There is a time delay in the interaction between the source device and the receiving device. The time delay in the process of using a touch screen electronic device to control a non-touch screen electronic device will cause a delay in the content displayed on the touch screen electronic device, which will affect the user experience. Summary of the Invention

[0006] This application provides a method, electronic device, and system for displaying content, which can reduce the display latency of the touch screen electronic device when a user controls a non-touch screen electronic device through the touch screen electronic device, thereby improving the user experience.

[0007] To achieve the above objectives, this application adopts the following technical solution:

[0008] In a first aspect, a method for displaying an application window is provided. The method includes: a first electronic device displaying a projection interface of a second electronic device, wherein the second electronic device runs a first application, the projection interface includes a first window of the first application, the first window being located at a first position on the projection interface; the first electronic device receiving a user's window drag operation on the first window; and in response to the window drag operation, the first electronic device displaying a second window of the first application at a second position on the projection interface, the second position being any position on the drag trajectory generated by the window drag operation, the second window being drawn by the first electronic device based on the second position.

[0009] In this method, after the first electronic device receives a user's drag operation on a window of the application's projection interface on the second electronic device, it obtains the window's position after movement based on the drag operation information, and locally draws a window image based on a saved image of the application window. The locally drawn window is then displayed on the drag trajectory generated by the drag operation. The local drawing of the window image by the first electronic device has a low latency, which can reduce the latency of displaying content (the application window) on the first electronic device.

[0010] In one possible implementation, the first position is the starting point of the drag trajectory generated by the window drag operation, and the second position is the ending point of the drag trajectory generated by the window drag operation.

[0011] According to the first aspect, or any implementation of the first aspect above, in response to a window dragging operation, the first electronic device displays a third window of the first application at a third position on the projection interface. The third position is any position other than the starting and ending positions on the dragging trajectory generated by the window dragging operation, and the third window is drawn by the first electronic device based on this third position. In this way, the application window moves with the movement of the stylus or finger, providing good responsiveness.

[0012] According to the first aspect, or any implementation of the first aspect above, the first electronic device also sends information about the window drag operation to the second electronic device; the first electronic device receives a screen mirroring interface update request sent by the second electronic device, the screen mirroring interface update request including the window image content of the first application updated by the second electronic device based on the information about the window drag operation.

[0013] In this method, the second electronic device also draws a window image based on the information from the window dragging operation and sends the updated window image to the first electronic device. In this way, the first electronic device can obtain the continuously updated window image, enabling the locally drawn window image content to be updated in real time.

[0014] According to the first aspect, or any implementation of the first aspect above, the first electronic device draws the second window and / or the third window based on the image content of the first window; or, the first electronic device draws the second window and / or the third window based on the window image content updated by the first application. That is, the first electronic device performs local drawing based on the saved window image content of the first application. The window image content of the first application saved by the first electronic device can be the window image content of the first application in the projection interface when the window drag operation begins, or the window image content of the first application updated in real time by the second electronic device.

[0015] According to the first aspect, or any implementation of the first aspect above, the first electronic device receives the image content and first position of the first window drawn by the second electronic device; the first electronic device displays the projection interface of the second electronic device by: the first electronic device displaying the first window on the projection interface according to the image content and first position of the first window. In this way, the first electronic device can obtain the image of the first window and realize local drawing.

[0016] In a second aspect, a method for displaying content is provided, the method comprising: a first electronic device displaying a projection interface of a first application, receiving a first input from a stylus, the first application running on a second electronic device; the first electronic device sending the first input information to the second electronic device; the first electronic device receiving a projection image from the second electronic device, updating the projection interface according to the projection image, the projection image including a first writing trajectory drawn by the second electronic device according to the first input information; the first electronic device obtaining drawing parameters of the second electronic device according to the first writing trajectory, the drawing parameters including at least one of brush type parameters, line color parameters, line width parameters, and line texture parameters; the first electronic device receiving a second input from the stylus; the first electronic device drawing a second writing trajectory according to the drawing parameters and the second input information; and the first electronic device displaying the second writing trajectory, the second writing trajectory being displayed as a continuation of the first writing trajectory.

[0017] In this method, the first electronic device obtains the drawing parameters of the second electronic device through self-learning, and uses the drawing parameters of the second electronic device to perform local drawing, so that the writing trajectory drawn locally is consistent with the line color, line width, line texture, etc. of the writing trajectory obtained from the second electronic device; thus ensuring the consistency of user experience.

[0018] According to the second aspect, the first electronic device obtains the drawing parameters of the second electronic device based on the first writing trajectory, including: if the first electronic device does not receive pen input within a preset time threshold before receiving the first input, it performs the step of obtaining the drawing parameters of the second electronic device.

[0019] In this way, when the user starts using the stylus, the first electronic device obtains the drawing parameters of the second electronic device based on the screen projection image of the second electronic device; during the user's input using the stylus, drawing parameter learning is not enabled; thus saving computing power and power consumption of the first electronic device.

[0020] According to the second aspect, or any implementation of the second aspect above, the first electronic device uses preset drawing parameters of the first electronic device to draw a third writing trajectory based on the first input information, and the third writing trajectory has the same line trajectory as the first writing trajectory.

[0021] According to the second aspect, or any implementation of the second aspect above, the first electronic device sends second input information to the second electronic device; the first electronic device receives a second image from the second electronic device, the second image including a fourth writing trajectory drawn according to the second input information, the fourth writing trajectory having the same line trajectory as the second writing trajectory.

[0022] According to the second aspect, or any implementation of the second aspect above, the fourth writing trajectory also has at least one of the following elements that are the same as the second writing trajectory: line color, line width, and line texture.

[0023] According to the second aspect, or any of the above implementations of the second aspect, the projected image is displayed on the first layer, the second writing trajectory is displayed on the second layer, and the second layer is superimposed on the first layer.

[0024] The writing trajectory drawn locally by the first electronic device will be overwritten by the writing trajectory drawn by the second electronic device based on the same input after a certain period of time. In one implementation, the second writing trajectory stops displaying after a preset display period. In another implementation, the second writing trajectory stops displaying after the fourth writing trajectory is displayed.

[0025] Thirdly, an electronic device is provided, comprising: a display screen, one or more processors, a memory, and one or more computer programs; wherein the processor is coupled to the display screen and the memory, and the one or more computer programs are stored in the memory. When the electronic device is running, the processor can execute the one or more computer programs stored in the memory to cause the electronic device to perform the method of any of the above-mentioned implementations.

[0026] For the technical effects of the third aspect and any of its implementation methods, please refer to the technical effects of the corresponding implementation methods in the above aspects, which will not be repeated here.

[0027] Fourthly, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as instructions or code) that, when executed by an electronic device, causes the electronic device to perform a method according to any of the above-mentioned implementations.

[0028] For the technical effects of the fourth aspect and any of its implementation methods, please refer to the technical effects of the corresponding implementation methods in the above aspects, which will not be repeated here.

[0029] Fifthly, a computer program product is provided. When the computer program product is run on an electronic device, it causes the electronic device to execute any implementation method of any of the above aspects.

[0030] For the technical effects of the fifth aspect and any of its implementation methods, please refer to the technical effects of the corresponding implementation methods in the above aspects, which will not be repeated here.

[0031] Sixthly, a circuit system is provided. The circuit system includes processing circuitry configured to perform a method according to any of the implementations of any of the foregoing aspects.

[0032] For the technical effects of the sixth aspect and any of its implementation methods, please refer to the technical effects of the corresponding implementation methods in the above aspects, which will not be repeated here.

[0033] Seventhly, a chip system is provided. The chip system includes a processor and interface circuitry. The interface circuitry performs transmit / receive functions and sends instructions to the processor. When the instructions are executed by the processor, the processor performs a method according to any of the implementations described in any of the above aspects.

[0034] For the technical effects of the seventh aspect and any of its implementation methods, please refer to the technical effects of the corresponding implementation methods in the above aspects, which will not be repeated here. Attached Figure Description

[0035] Figure 1 A schematic diagram of the system architecture applicable to the display content method provided in the embodiments of this application;

[0036] Figure 2A A schematic diagram of the system architecture applicable to the display content method provided in the embodiments of this application;

[0037] Figure 2B A schematic diagram of the system architecture applicable to the display content method provided in the embodiments of this application;

[0038] Figure 3A schematic diagram illustrating a scenario in which the content display method provided in this application is applicable;

[0039] Figure 4 A schematic diagram illustrating a scenario in which the content display method provided in this application is applicable;

[0040] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0041] Figure 6 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0042] Figure 7 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0043] Figure 8A This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0044] Figure 8B This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0045] Figure 9 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0046] Figure 10 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0047] Figure 11 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0048] Figure 12 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0049] Figure 13 This is a schematic diagram of the method for obtaining drawing parameters in the method for displaying content provided in the embodiments of this application;

[0050] Figure 14 This is a schematic diagram of the method for obtaining drawing parameters in the method for displaying content provided in the embodiments of this application;

[0051] Figure 15 This is a schematic diagram of the method for obtaining drawing parameters in the method for displaying content provided in the embodiments of this application;

[0052] Figure 16 This is a schematic diagram of a method for displaying content provided in an embodiment of this application;

[0053] Figure 17 This is a schematic diagram of a method for displaying content provided in an embodiment of this application;

[0054] Figure 18 This is a schematic diagram of a method for displaying content provided in an embodiment of this application;

[0055] Figure 19 This is a schematic diagram of a method for displaying content provided in an embodiment of this application;

[0056] Figure 20 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0057] Figure 21 This is a schematic diagram of a method for displaying content provided in an embodiment of this application;

[0058] Figure 22 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0059] Figure 23 This is a scenario example diagram illustrating the display content method provided in the embodiments of this application;

[0060] Figure 24 This is a schematic diagram of the structural composition of an electronic device provided in an embodiment of this application. Detailed Implementation

[0061] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one or more (including two). The term “and / or” is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0062] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated.

[0063] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0064] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0065] The content display method provided in this application embodiment can be used for Figure 1 The system shown includes an electronic device 100 with a touchscreen and a processing device 200. The electronic device 100 and the processing device 200 are connected via a wired connection (e.g., a high-definition multimedia interface (HDMI)) or a wireless connection (e.g., wireless fidelity (Wi-Fi)). An application runs on the processing device 200. The processing device 200 projects each interface of the application to the electronic device 100 for display.

[0066] The electronic device 100 can also receive user input. For example, the system also includes a stylus 300, which the user can use to input on the screen (touchscreen) of the electronic device 100; or, for example, the user can use their finger to input on the screen (touchscreen) of the electronic device 100. This application embodiment is described using the user's input with the stylus 300 as an example.

[0067] Electronic device 100 receives input from stylus 300, acquires input information such as input coordinates, pressure sensitivity, and tilt angle of stylus 300 on the screen, and transmits the input information to processing device 200 via wired or wireless means. Processing device 200 performs corresponding operations on application 1 based on the input information (e.g., drawing a writing trajectory on the application 1 display interface, or dragging the application 1 window), renders and composites the interface of application 1, and generates a projected image. In one implementation, processing device 200 compresses the generated projected image into a video stream and sends the video stream to electronic device 100. Electronic device 100 decompresses the video stream, acquires the projected image, and displays the projected image on the screen.

[0068] The aforementioned electronic device 100 may include tablet computers, handwriting tablets, portable mobile devices (such as mobile phones), handheld computers, laptops, netbooks, personal computers (PCs), smart home devices (such as smart TVs, smart screens, large screens, etc.), personal digital assistants (PDAs), wearable devices (such as smartwatches, smart bracelets, etc.), in-vehicle computers, etc., and this application embodiment does not impose any limitations on this.

[0069] The aforementioned processing device 200 may include a personal computer (PC), portable mobile devices (such as mobile phones), handheld computers, laptops, netbooks, tablets, smart TVs, smart screens, large screens, etc., and this application embodiment does not impose any limitations on this.

[0070] The stylus 300 described above may include a stylus, a capacitive stylus, etc., and this application embodiment does not impose any limitations on it.

[0071] Taking a tablet computer as an electronic device 100 and a PC as a processing device 200 as an example. Figure 2A This illustration shows a system example to which the content display method provided in this application applies. The tablet and PC are wirelessly connected. For example, the tablet and PC are connected via Wireless Fidelity (Wi-Fi). Application 1 runs on the PC, and the tablet is set as the PC's input device; the PC also projects the interface of Application 1 onto the tablet for display. Each time the tablet receives user input, it sends the input information to the PC for processing. The PC performs corresponding operations based on the input information and updates the interface of Application 1. In this way, the user can use a stylus to input on the tablet, thereby controlling the PC and performing corresponding operations on Application 1.

[0072] The PC also projects the updated interface of Application 1 onto the tablet for display. In one implementation, the PC compresses the updated projected image into a video stream and transmits it to the tablet. The tablet decompresses the video stream, obtains the updated projected image, and displays it. In this way, both the tablet and the PC display the updated interface of Application 1.

[0073] In this way, on the one hand, the tablet computer does not need to install and run application one, but only displays the interface of application one; the tablet computer can use application one by leveraging the PC's powerful computing capabilities. On the other hand, the PC can receive user input through the tablet computer, allowing users to use a stylus to modify documents on devices without touchscreens such as PCs, input text by hand, draw in drawing software, or drag and drop application windows, etc.

[0074] It should be noted that in some embodiments, the processing device 200 may not be a terminal device, but a server. For example, Figure 2B As shown, the processing device 200 is a cloud server or a remote server. After receiving input from the stylus pen, the tablet computer transmits the input information to the server. The server performs corresponding operations based on the input information, generates a projected image, and transmits the projected image to the tablet computer. The tablet computer acquires and displays the projected image. The tablet computer utilizes the server's powerful computing capabilities to achieve high-performance stylus pen input, providing users with a better input experience.

[0075] In the above process, the delay in the transmission of input information from electronic device 100 to processing device 200, the delay in the generation of the projection image by processing device 200, and the delay in the transmission of the projection image from processing device 200 to electronic device 100 will cause delay in the display content of electronic device 100, resulting in a user experience with poor responsiveness.

[0076] Responsiveness can be represented by the latency of the displayed content; the latency of the displayed content is the time from when the electronic device receives user input (for example, the moment the stylus or the user's finger leaves the screen) to when the corresponding content is displayed on the screen; the smaller the latency of the displayed content, the better the responsiveness; the larger the latency of the displayed content, the worse the responsiveness.

[0077] In one example, such as Figure 3As shown, a stylus is used to draw lines on the screen of an electronic device. After receiving input from the stylus, the electronic device transmits the input information to a processing device via a wired or wireless connection. The processing device draws based on the input information, synthesizing an image containing the writing trajectory; the processing device then sends the generated projection image to the electronic device. The image containing the writing trajectory is displayed on the screen of the electronic device. Factors such as the latency of transmitting stylus input information from the electronic device to the processing device, the latency of the processing device generating the image containing the writing trajectory, the latency of transmitting the image containing the writing trajectory from the processing device to the electronic device, and the latency of the electronic device displaying the image containing the writing trajectory all contribute to a latency between receiving stylus input and displaying the content (the writing trajectory). During the process of drawing lines on the screen, there is a certain distance between the stylus's position on the screen and the writing trajectory. The greater the latency of the display content, the greater the distance between the stylus's position on the screen and the writing trajectory, resulting in poorer responsiveness.

[0078] In another example, such as Figure 4 As shown, a user uses a stylus to drag an application window on the screen of an electronic device, moving the window's position on the screen. After receiving input from the stylus, the electronic device transmits the input information (such as input coordinates) to a processing device via a wired or wireless connection. The processing device draws based on the input information, synthesizing an image of the application window after its movement; the processing device then sends this image back to the electronic device. The image of the moved application window is displayed on the screen of the electronic device. Factors such as the latency of transmitting stylus input information from the electronic device to the processing device, the latency of the processing device generating the application window image, the latency of transmitting the application window image from the processing device to the electronic device, and the latency of the electronic device displaying the image of the moved application window, all contribute to a latency between receiving stylus input and displaying the application window. During the process of dragging the application window on the screen, there is a certain distance between the stylus's position on the screen and the application window's position. The greater the latency of the display content, the greater the distance between the stylus's position on the screen and the application window's position, resulting in poorer responsiveness.

[0079] This application provides a method for displaying content, in which an electronic device displays a projection interface of an application on a processing device. After receiving handwriting input from a user on the projection interface of application one, the handwriting input information is transmitted to a processing device for processing. The processing device draws the application based on the input information. The electronic device also performs local drawing based on the handwriting input. Since the latency of local drawing by the electronic device is less than the latency of the electronic device acquiring the projection interface from the processing device, the distance between the handwriting and the displayed content on the electronic device screen can be reduced; this reduces the latency of displaying content on the electronic device and improves the responsiveness of the handwriting.

[0080] For example, Figure 5 A schematic diagram of an electronic device 100 provided in an embodiment of this application is shown.

[0081] Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a display screen 150, an antenna, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, and a camera 191, etc. The sensor module 180 may include a pressure sensor, a magnetic sensor, a proximity sensor, a fingerprint sensor, a touch sensor, etc.

[0082] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0083] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0084] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0085] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0086] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0087] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to a touch sensor, charger, flash, camera 191, etc., through different I2C bus interfaces. For example, the processor 110 can couple to a touch sensor through the I2C interface, enabling the processor 110 and the touch sensor to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

[0088] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to realize communication between the processor 110 and the audio module 170.

[0089] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. Both the I2S interface and the PCM interface can be used for audio communication.

[0090] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

[0091] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 150 and the camera 191. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 191 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 150 communicate via the DSI interface to enable the electronic device 100 to display images.

[0092] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to a camera 191, a display screen 150, a wireless communication module 160, an audio module 170, a sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0093] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0094] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0095] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0096] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 150, camera 191, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0097] The wireless communication function of electronic device 100 can be implemented through an antenna, wireless communication module 160, modem processor, and baseband processor.

[0098] Antennas are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antennas can be multiplexed as diversity antennas for a wireless local area network. In some other embodiments, antennas can be used in conjunction with tuning switches.

[0099] The modem processor may include a modulator and a demodulator. The modulator modulates a low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 150. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and housed within the same device as other functional modules.

[0100] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via an antenna, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via the antenna.

[0101] Electronic device 100 implements display functions through a GPU, a display screen 150, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 150 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0102] The display screen 150 is used to display images, videos, etc. The display screen 150 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 150, where N is a positive integer greater than 1.

[0103] Electronic device 100 can perform shooting functions through ISP, camera 191, video codec, GPU, display 150 and application processor.

[0104] The ISP (Image Signal Processor) is used to process data fed back from the camera 191. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 191.

[0105] Camera 191 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 191, where N is a positive integer greater than 1.

[0106] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0107] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0108] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0109] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0110] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0111] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0112] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0113] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0114] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0115] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0116] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0117] The magnetic sensor includes a Hall effect sensor. The electronic device 100 can use the magnetic sensor to detect the opening and closing of the flip cover. A distance sensor is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, during a shooting scenario, the electronic device 100 can use the distance sensor to measure distance for fast focusing. A fingerprint sensor is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, app access lock, fingerprint photography, fingerprint answering of calls, etc.

[0118] A touch sensor, also known as a "touch panel," can be located on the display screen 150. The touch sensor and the display screen 150 together form a touchscreen, also called a "touch display." The touch sensor detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 150. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 150.

[0119] A pressure sensor is used to sense pressure signals and convert them into electrical signals. In some embodiments, the pressure sensor may be located on the display screen 150. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When a force is applied to the pressure sensor, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation is applied to the display screen 150, the electronic device 100 detects the intensity of the touch operation based on the pressure sensor. The electronic device 100 may also calculate the touch position based on the detection signal from the pressure sensor. In some embodiments, the pressure sensor is used to detect the input position of a stylus, pressure sensitivity, etc.

[0120] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0121] The following describes in detail the display content method provided in the embodiments of this application, taking a tablet computer as the electronic device 100 and a PC as the processing device 200 as an example.

[0122] In one scenario, a tablet and a PC are wirelessly connected, with the tablet acting as the PC's input device. An application (Application 1) runs on the PC. The user uses a stylus to draw lines on the tablet, inputting handwriting on the display of Application 1. For example, Application 1 could be a drawing application; the user could use a stylus to draw lines on the tablet, inputting drawing strokes on the drawing application's display.

[0123] For example, such as Figure 6As shown, PC 200 displays the interface 601 of application one, and PC 200 projects the interface 601 of application one onto tablet computer 100 for display. Tablet computer 100 displays the interface 601 of application one (projected interface). The user uses stylus 300 to draw lines on the projected interface on tablet computer 100. Tablet computer 100 receives input from stylus 300 and transmits the input information to PC 200 via wireless communication (e.g., Bluetooth). The input information includes input coordinates and may also include any one or any combination of the following: pressure sensitivity, tilt angle, etc. PC 200 draws a writing trajectory one based on the input information and drawing parameters. The drawing parameters include any one or any combination of the following: brush type (e.g., pen, pencil, marker, watercolor pen, oil paintbrush, etc.), line color (e.g., black, red, green, blue, purple, etc.), line width, and line texture (e.g., no texture, water ripple, marble, wood grain, gravel, etc.). For example, on PC200, application 1 determines the position of the drawing point based on the input coordinates of the input information, determines the line width based on the pressure sensitivity of the input information, determines the size of the drawing point based on the tilt angle of the input information, and determines the line width of the writing trajectory based on the line width of the drawing parameters. PC 200 also renders and composites the current display interface and writing trajectory 1 to generate a projection image 602 containing writing trajectory 1. In one implementation, after the desktop window manager of PC 200 composites the various application layers, it calls the Windows screenshot application program interface (API) to capture the desktop image. In some examples, PC 200 encodes the desktop image (including the display interface of application 1) as a projection image and sends it to tablet computer 100 via wireless communication (such as a Wi-Fi Direct channel). Tablet computer 100 decompresses the video stream and obtains the desktop image, that is, obtains the projection image 602 including writing trajectory 1. In other examples, PC 200 divides the desktop image (including the display interface of application 1) into multiple rectangular areas and detects the rectangular areas where the current frame and the previous frame image change. If the sum of the areas of the rectangular regions where the image changes is less than a preset threshold, the image of the rectangular regions where the image changes is encoded and sent to the tablet computer 100. The tablet computer 100 decompresses the image, obtains the image of the rectangular regions where the image changes, and obtains the projection image 602 based on the saved previous frame image and the image of the rectangular regions where the image changes. The projection image 602 includes the writing trajectory. If the sum of the areas of the rectangular regions where the image changes is greater than or equal to the preset threshold, the desktop image is used as the projection image, video encoded, and sent to the tablet computer 100. The tablet computer 100 decompresses the video stream and obtains the projection image 602, which includes the writing trajectory.

[0124] The tablet computer 100 also performs local drawing based on stylus input; for example, it generates a second writing trajectory (local image 603) based on the information input by the stylus. After receiving the projected image 602 from the PC 200, the tablet computer 100 overlays the locally drawn second writing trajectory (local image 603) onto the projected image 602 for display, i.e., displays the updated projected interface 604. In one example, the tablet computer's display image (updated projected interface 604) includes two layers: a bottom layer is the projected layer (including the projected image 602), and a top layer is the local layer (including the local image 603), with the local layer floating above the projected layer. The tablet computer 100 displays an image composed of the projected layer and the local layer.

[0125] Understandably, the tablet 100 only draws the writing trajectory and does not need to generate a complete image of the application's display interface, resulting in a relatively small latency in drawing the writing trajectory. However, the process of transmitting the stylus input information from the tablet 100 to the PC 200, the PC 200 generating the projected image, and the projected image being transmitted back to the tablet 100 takes a longer time; that is, the latency for the tablet 100 to acquire the projected image is relatively long. During this latency, the user continues to input on the tablet 100's display using the stylus 300. There is a certain distance between the writing trajectory displayed on the tablet 100 and the stylus 300. Since the latency of the tablet 100 drawing the writing trajectory locally is less than the latency of acquiring the projected image from the PC 200, overlaying the writing trajectory drawn locally on the tablet 100 with the writing trajectory drawn on the PC 200 reduces the distance between the stylus and the writing trajectory on the tablet 100 screen; this reduces the latency of the content displayed on the tablet 100 and improves the stylus's responsiveness.

[0126] There is a first delay in the tablet computer 100 acquiring the projected image, and a second delay in the local drawing of the writing trajectory. The image displayed by the tablet computer 100 at time T is a composite image generated by the PC 200 based on the input at time (T - first delay) and the writing trajectory locally drawn by the tablet computer 100 based on the input at time (T - second delay). For example, Figure 7 As shown, the latency for tablet computer 100 to acquire the projected image is 100 milliseconds (ms), and the latency for tablet computer 100 to locally draw the writing trajectory is 20 ms. At time T, the writing trajectory in the projected image acquired by tablet computer 100 from PC 200 is drawn based on the stylus input at (T-100ms), while the locally drawn writing trajectory is drawn based on the stylus input at (T-20ms). The locally drawn writing trajectory reduces the distance between the writing trajectory drawn by the processing device and the stylus.

[0127] Understandably, points drawn locally on the tablet will be overwritten by points drawn on the PC based on the same input after a certain period of time. In other words, writing trajectory two is a continuation of writing trajectory one, and writing trajectory two partially overlaps with the original writing trajectory. For example, as... Figure 8A As shown, a portion of writing trajectory two completely overlaps with writing trajectory one, and the remaining portion of writing trajectory two is a continuation of writing trajectory one. In one implementation, the writing trajectory drawn locally by the tablet computer stops displaying after a preset duration. For example, this preset duration is the first delay before the tablet computer acquires the projected image. Another example is that the preset duration is (the first delay before the tablet computer acquires the projected image - the second delay before the local writing trajectory is drawn); thus, writing trajectory two is a continuation of writing trajectory one, and writing trajectory two does not overlap with writing trajectory one. For example, as shown... Figure 8B As shown, writing trajectory two does not overlap with writing trajectory one at all.

[0128] It should be noted that this application embodiment uses the example of a PC and tablet performing a drawing operation once for each input received. In other embodiments, the PC and tablet may also perform a drawing operation based on multiple inputs received. For example, a drawing operation may be performed based on handwriting input received within 10ms. This application embodiment does not limit this approach.

[0129] For example, such as Figure 9 As shown, at time T0, the tablet computer 100 displays the PC's projection interface, which does not include handwriting. The user draws lines on the tablet computer 100 using a stylus. The tablet computer 100 receives stylus input. For example, the tablet computer treats the stylus input received within the first time period as one input and sends the input information corresponding to one input to the PC 200 for drawing the writing trajectory. For example, Figure 9 In the process, the tablet computer 100 sends the input information corresponding to the stylus input between time T0 and time T1b to the PC 200. The PC 200 generates a projection image containing the writing trajectory based on the stylus input between time T0 and time T1b. The tablet computer 100 then acquires the projection image containing the writing trajectory.

[0130] The tablet computer 100 also performs local drawing based on stylus input. In one example, the tablet computer 100 performs local drawing based on stylus input between time T1b and time T2, generating writing trajectory two. The input time period of the stylus input between time T1b and time T2 is after the input time period of the stylus input between time T0 and time T1b, and does not overlap with the input time period of the stylus input between time T0 and time T1b. Accordingly, writing trajectory two is a continuation of writing trajectory one, and writing trajectory two does not coincide with writing trajectory one. In another example, the tablet computer 100 performs local drawing based on stylus input between time T1a and time T2, generating writing trajectory two. Time T1a is after time T0 and before time T1b. Writing trajectory two partially overlaps with the trajectory of writing trajectory one, and writing trajectory two includes a continuation of writing trajectory one.

[0131] The tablet 100 displays an updated screen mirroring interface. The updated interface displays a screen mirroring image containing writing trajectory one on the screen mirroring layer, and overlays writing trajectory two (a local layer) on top of this image. Writing trajectory two includes a continuation of writing trajectory one, reducing the distance between writing trajectory one and the stylus, thus decreasing the drawing latency on the tablet 100.

[0132] Figure 10 This illustration shows a scenario example of the content display method provided in the embodiments of this application.

[0133] like Figure 10 As shown, PC 200 runs application one. For example, application one is a drawing application. Tablet PC 100 is set as an input device for PC 200. PC 200 projects the interface 701 of application one onto tablet PC 100 for display. Tablet PC 100 displays the interface 701 of the drawing application. The user draws lines on interface 701 using stylus 300. For example, at time t0, tablet PC 100 receives input from stylus 300 moving from point a to point b on interface 701. Tablet PC 100 sends the input information of stylus 300 moving from point a to point b to PC 200; and performs local drawing based on the input information. For example, tablet PC 100 locally draws writing trajectory 702. At time t1, tablet PC 100 displays writing trajectory 702 on interface 701. On the other hand, PC 200 receives the input information of stylus 300 moving from point a to point b, and generates a projected image containing writing trajectory 703 based on the information. PC 200 sends the projection image containing the writing trajectory 703 to tablet PC 100. Tablet PC 100 obtains the projection image containing the writing trajectory 703, updates the projection interface at time t2, and displays the projection image containing the writing trajectory 703 on the projection layer.

[0134] Understandably, the local layer of the updated projection interface 704 includes writing trajectory 702, and the projection layer includes writing trajectory 703. The local layer is superimposed on the projection layer. Both writing trajectory 702 and writing trajectory 703 are drawn based on the input of the stylus 300 moving from point a to point b, and writing trajectory 702 and writing trajectory 703 have the same line trajectory.

[0135] Because the latency of local drawing on the tablet 100 is less than the latency of acquiring the projected image from the PC 200, time t1 is earlier than time t2. In other words, the writing trajectory drawn locally on the tablet 100 is displayed on the tablet 100 screen faster than the writing trajectory drawn on the PC 200, which can show the user the writing trajectory input by the stylus pen more quickly, shorten the stylus drawing latency, and improve the stylus responsiveness.

[0136] At time t2, tablet computer 100 acquires the projected image containing writing trajectory 703 from PC 200 and displays it on the projection layer. Writing trajectory 703 has the same line trajectory as writing trajectory 702 and will cover writing trajectory 702; from time t2 onwards, the display of writing trajectory 702 can stop. In some examples, the writing trajectory 702 drawn locally by tablet computer 100 is displayed for a period of time and then stops displaying at time t3. For example, time t3 is equal to time t2; or time t3 is later than time t2.

[0137] Understandably, in one example, between time t0 and time t2, the user continues to draw lines on the tablet computer 100 using the stylus 300, moving the stylus 300 from point b to point c. The tablet computer 100 performs local drawing based on the information input from the stylus's movement from point b to point c and displays the locally generated writing trajectory. Figure 10 The line segment between midpoint b and point c is represented by a dashed line to distinguish it from writing trajectory 702 and writing trajectory 703.

[0138] The display method provided in this application involves an electronic device generating a projection image including a writing trajectory using the computing power of a processing device, and locally drawing the writing trajectory, then overlaying the locally drawn writing trajectory onto the projection image generated by the processing device. Because the local drawing latency of the electronic device is low, the locally drawn writing trajectory reduces the distance between the stylus on the electronic device screen and the writing trajectory on the projection image; this reduces the line-drawing latency of the stylus input on the electronic device and improves the responsiveness of the stylus.

[0139] In some embodiments, for the same input from a stylus, the writing trajectory drawn on a PC may differ from that drawn on a tablet computer in terms of line color, line width, and line texture due to different drawing parameters used. For example, Figure 6The tablet computer 100 uses its own drawing parameters for local drawing, while the PC 200 uses its own drawing parameters. For example, the tablet computer 100 uses a "pen" brush to draw writing trajectory two. The writing trajectory one that the tablet computer 100 receives from the PC 200 is drawn by the PC 200 using a "brush" brush. On the updated projection interface 604, the writing trajectory two drawn locally by the tablet computer 100 is thinner than the writing trajectory one drawn by the PC 200. This creates a disconnect between the locally drawn writing trajectory and the writing trajectory on the image received from the PC on the updated projection interface displayed on the tablet computer, affecting the consistency of the user experience.

[0140] The display method provided in this application involves a tablet computer obtaining drawing parameters from a PC and using those parameters for local drawing. This ensures that the locally drawn writing trajectory has the same line color, line width, and line texture as the writing trajectory obtained from the PC, guaranteeing a consistent drawing experience for the user. For example,... Figure 11 As shown, the writing trajectory drawn locally on the tablet computer is completely consistent with the writing trajectory obtained from the PC in terms of line color, line width, and line texture, with no sense of disjointedness.

[0141] Figure 12 This illustration shows a scenario example of the content display method provided in an embodiment of this application. For example... Figure 12 As shown, application one (a drawing application) runs on PC 200. Tablet 100 is set as an input device for PC 200. PC 200 displays interface 801 of application one, and tablet 100 displays the projection interface (interface 801 of application one). The user draws lines on interface 801 using stylus 300. Tablet 100 receives input from stylus 300 moving from point a to point b on interface 801. At time Ta, tablet 100 sends information about the input from stylus 300 moving from point a to point b to PC 200.

[0142] PC 200 receives input information from the stylus 300, indicating that it has moved from point a to point b. Based on this information, it generates a first projection image containing the writing trajectory 802. PC 200 sends the first projection image containing the writing trajectory 802 to tablet PC 100. Tablet PC 100 acquires the first projection image containing the writing trajectory 802 and updates the projection interface at time Tb, displaying the updated projection interface 803. The projection layer of the updated projection interface 803 contains the first projection image. At time Tb, tablet PC 100 has not acquired the drawing parameters from PC 200. In one implementation, such as... Figure 12As shown, the tablet computer 100 does not perform local drawing before obtaining the drawing parameters from the PC 200. In another implementation, the tablet computer 100 performs local drawing using preset drawing parameters from the tablet computer 100 before obtaining the drawing parameters from the PC 200. Figure 12 (not shown in the image); it is understood that if the drawing parameters of the tablet computer 100 are different from those of the PC 200, the writing trajectory drawn by the tablet computer 100 based on the input information of the stylus 300 moving from point a to point b will have a different line color, line width, or line texture than the writing trajectory 802.

[0143] The tablet computer 100 receives the writing trajectory 802 and obtains the drawing parameters of the PC 200 based on the writing trajectory 802. In one implementation, the tablet computer 100 receives a projected image and obtains a new writing trajectory based on the change in the current frame of the projected image compared to the previous frame, or the image of the changed rectangular area. The drawing parameters of the PC 200 are then obtained based on the new writing trajectory.

[0144] In one example, the tablet 100 obtains the line color based on newly added writing traces. For example... Figure 13 As shown, the writing trajectory image is separated into three channels: R (red), G (green), and B (blue), and the extreme points (R', G', B') other than 0 in the grayscale histograms of the three channels are counted respectively; the colors corresponding to (R', G', B') are the line colors.

[0145] In one example, the tablet computer 100 binarizes the newly added writing trajectory, performs a closing operation on the binarized writing trajectory image to remove internal holes and irregular handwriting edges, and obtains the number of pixels A1 within the connected region. For example, the original image after binarizing and performing the closing operation on the newly added writing trajectory is shown below. Figure 14 As shown in (a). Then, a dilation operation is performed on the original image to obtain a dilated image, and the number of pixels A2 in the dilated image is counted. For example, a 3×3 cross-shaped structure element is used. Figure 14 Perform dilation on the original image shown in (a) to obtain the image as shown in (a). Figure 14 The dilated image shown in (b) is used to count the number of pixels A2 in the dilated image. The line width W is obtained according to the following formula ①.

[0146]

[0147] In one example, a binary image thinning algorithm (also known as an image skeleton extraction algorithm) is used to thin the connected regions of the newly added writing trajectory to a width of one pixel. For example, the original image (the connected regions of the newly added writing trajectory) is as follows: Figure 15As shown in (a), the image after thinning the original image using the binary image thinning algorithm is as follows: Figure 15 As shown in (b), the area of ​​the connected region of the newly added writing trajectory divided by the area of ​​the connected region of the thinned image is the line width.

[0148] The differences between different brush types (e.g., fountain pen, pencil, marker, watercolor brush, oil paintbrush, etc.) are mainly reflected in the texture of the main part of the writing trajectory. In one example, the Local Binary Pattern (LBP) features of the newly added writing trajectory are extracted, and the LBP histogram features are statistically analyzed as the writing trajectory texture features. A feature vector-based classification algorithm (e.g., histogram cross-validation kernel, or a machine learning approach using principal component analysis (PCA) dimensionality reduction + support vector machine (SVM)) is used to classify the writing trajectory texture features to obtain the brush type.

[0149] Of course, in other embodiments, other methods in conventional technology can also be used to obtain drawing parameters such as line color and line width based on the writing trajectory; this application does not limit this.

[0150] For example, the tablet computer 100 obtains the drawing parameters of the PC 200 based on the writing trajectory 802, which has a third time delay. The tablet computer 100 obtains the drawing parameters of the PC 200 at time Tb' (Tb' = Tb + third time delay). After obtaining the drawing parameters of the PC 200, the tablet computer 100 begins to perform local drawing based on the handwriting input using the drawing parameters of the PC 200.

[0151] For example, such as Figure 12As shown, between time Ta and time Tb', the user continues to draw lines on the tablet computer 100 using the stylus 300, moving the stylus 300 from point b to point c. The tablet computer 100 sends the input information of the stylus 300 moving from point b to point c to the PC 200. The tablet computer 100 also uses the drawing parameters of the PC 200 to perform local drawing based on the input information of the stylus 300 moving from point b to point c. At time Tc, the tablet computer 100 updates the projection interface. The local layer of the updated projection interface 804 includes a writing trajectory 805, which contains the writing trajectory locally drawn by the tablet computer 100 based on the input information of the stylus 300 moving from point b to point c. On the other hand, the PC 200 receives the input information of the stylus 300 moving from point b to point c and generates a second projection image containing the writing trajectory 806 based on this input information. The PC 200 sends the second projection image containing the writing trajectory 806 to the tablet computer 100. The tablet computer 100 acquires a second projection image containing the writing trajectory 806, updates the projection interface at time Td, and displays the second projection image containing the writing trajectory 806 on the projection layer.

[0152] Understandably, the updated projection interface 807 has a local layer including writing trajectory 805 and a projection layer including writing trajectory 806, with the local layer superimposed on the projection layer. Both writing trajectory 805 and writing trajectory 806 are drawn based on the input of the stylus 300 moving from point b to point c, and they have the same line trajectory. Furthermore, both the tablet computer 100 and the PC 200 use the same drawing parameters to draw writing trajectory 805 and writing trajectory 806, respectively, and they have the same line color, line width, and line texture. Writing trajectory 805 and writing trajectory 806 are completely identical.

[0153] Because the latency of locally drawing on the tablet 100 is less than the latency of acquiring the projected image from the PC 200, time Tc is earlier than time Td. That is, the writing trajectory drawn locally on the tablet 100 is displayed on the tablet 100 screen faster than the writing trajectory drawn on the PC 200. At time Td, the tablet 100 acquires the second projected image containing the writing trajectory 806 from the PC 200 and displays it on the projected layer. Writing trajectory 806 has the same line trajectory as writing trajectory 805 and will cover writing trajectory 805; from time Td, the display of writing trajectory 805 can stop. In some examples, the writing trajectory 805 drawn locally on the tablet 100 is displayed for a period of time and then stops displaying at time Te. For example, time Te is equal to time Td; or time Te is later than time Td.

[0154] Understandably, in one example, between time Tb' and time Td, the user continues to draw lines on the tablet 100 using the stylus 300, moving the stylus 300 from point c to point d. The tablet 100 performs local drawing based on the information input from the stylus's movement from point c to point d and displays the locally generated writing trajectory. Figure 12 (Not shown in the image).

[0155] Figure 16 This illustrates one implementation of the content display method provided in an embodiment of this application. For example... Figure 16 As shown, application one runs on the PC. For example, application one is a drawing application. The tablet computer is set up as the PC's input device. The PC also projects the interface of application one onto the tablet computer's display. Users can use a stylus to draw lines on the tablet computer, enabling handwriting input on the PC; for example, users can use a stylus to draw lines on the tablet computer screen (projected interface), enabling them to input drawing strokes on the drawing application's display interface.

[0156] For example, such as Figure 12 As shown, the tablet computer 100 receives input from the stylus 300 as it moves from point a to point b on the interface 801. In one implementation, the tablet computer's touch module receives the stylus input and obtains corresponding input information based on the input. The input information includes input coordinates and may also include any one or any combination of the following: pressure sensitivity, tilt angle, etc. Taking the input of a single point (e.g., a pixel) on the tablet computer screen as an example, the tablet computer's touch module sends the input information to the PC via wireless communication (e.g., Bluetooth). An application running on the PC receives the stylus input information and draws a writing trajectory based on the stylus input information and the PC's drawing parameters. The drawing parameters include any one or any combination of the following: brush type (e.g., pen, pencil, marker, watercolor, oil paintbrush, etc.), line color (e.g., black, red, green, blue, purple, etc.), line width, and line texture (e.g., no texture, water ripple, marble, wood grain, gravel, etc.). For example, as shown... Figure 12 As shown, application 1 running on PC 200 generates writing trace 802. Application 1 also renders and composites based on the current display interface and writing trace 1 to generate a projection image containing writing trace 1. In one implementation, after the PC's desktop window manager composites the various application layers, it calls the Windows screenshot API to capture the desktop image; the desktop image (including the display interface of application 1) is used as the projection image for video encoding to generate a video stream; the video stream is sent to the tablet computer via wireless communication (e.g., Wi-Fi). The tablet computer receives the video stream, decodes the video, and obtains the projection image; it then generates a projection layer based on the projection image.

[0157] The tablet computer obtains a writing trajectory (first type) drawn by the PC based on the projected image, and learns drawing parameters from this trajectory to acquire the PC's drawing parameters. In one implementation, the tablet receives the projected image and obtains a new writing trajectory based on the changes in the current frame compared to the previous frame. The PC's drawing parameters are then obtained based on this new writing trajectory. In one example, the tablet computer averages the values ​​of the drawing parameters obtained from the current frame's projected image multiple times to improve the accuracy of the acquired PC drawing parameters.

[0158] The tablet also performs local drawing based on stylus input. For example, application two on the tablet uses the PC's drawing parameters to draw the writing trajectory two based on the information input by the stylus. For instance, as shown... Figure 12 As shown, the tablet computer 100 obtains the drawing parameters of the PC 200 according to the writing trajectory 802, uses the drawing parameters of the PC 200, and performs local drawing based on the input information of the stylus 300 moving from point b to point c to generate the writing trajectory 805.

[0159] The tablet generates a local layer based on the writing trajectory, then composites the projected layer and the local layer. The local layer floats above the projected layer for display, showing the updated projected interface. For example,... Figure 12 As shown, the tablet computer 100 displays an updated projection interface 804; the projection layer of the updated projection interface 804 includes the writing trajectory 802, and the local layer includes the writing trajectory 805. The tablet computer draws the writing trajectory 805 and the PC draws the writing trajectory 802 using the same drawing parameters, and the line color, line width, and line texture of the writing trajectory 805 and the writing trajectory 802 are consistent.

[0160] The display method provided in this application embodiment uses the drawing parameters of the PC for local drawing on the tablet computer, so that the writing trajectory drawn locally is consistent with the line color, line width, line texture, etc. of the writing trajectory obtained from the PC; thus ensuring a consistent user experience.

[0161] Understandably, the drawing parameters do not change during the user's stylus input process, i.e., during the continuous point reporting by the stylus. If the user changes drawing parameters such as brush type, line color, line width, or line texture, the time interval between the stylus reporting points before and after the parameter change is much longer than the time interval between two strokes during input. In one implementation, if the tablet determines that the time interval between the current received stylus input and the previous received stylus input is greater than a preset time threshold, it performs drawing parameter learning after acquiring the screen projection image. If the tablet determines that the time interval between the current received stylus input and the previous received stylus input is less than or equal to the preset time threshold, it does not perform drawing parameter learning after acquiring the screen projection image. In this way, when the user starts using stylus input, the tablet obtains the PC's drawing parameters based on the screen projection image; during the user's stylus input process, drawing parameter learning is not enabled, saving the tablet's computing power and power consumption.

[0162] Figure 17 This illustrates a specific implementation of the content display method provided in an embodiment of this application. For example... Figure 17 As shown, the tablet and PC establish a Wi-Fi peer-to-peer connection via Bluetooth proximity discovery; the screen mirroring service is then initiated at the framework layer. On the tablet's input side, the tablet receives stylus input via Bluetooth, and the integrated circuit (IC) driver reports the stylus input information (e.g., input coordinates, pressure sensitivity, tilt angle, etc.). The input flinger module then transmits the information input by the stylus through two paths.

[0163] like Figure 17 As shown, path (1) sends the information input by the stylus to the PC. The PC generates a projection layer based on the information input by the stylus. Path (2) allows the tablet to perform local drawing based on the information input by the stylus, generating a local layer.

[0164] Path (1): The tablet creates a virtual USB human interface device (HID), obtains the escaped touch events based on the information input by the stylus, and sends the stylus input information to the PC via TCP / IP protocol. The PC creates a virtual USB HID to inject touch events. At the PC application layer, the Windows Paint application responds to the touch events and calls the GPU for rendering. The Windows Desktop Window Management (DWM) component manages the tiling and full-screen state of application windows, compositing the surfaces of multiple applications into a single image. The DirectX Graphics Infrastructure (DXGI) sends the composited image to the PC display for display, and simultaneously calls the Screen Capture API to obtain the projected image, which is then encoded into a video stream by the PC encoder. This video stream is packetized using the Real-Time Streaming Protocol (RTSP) and sent to the tablet via Wi-Fi using TCP / IP protocol. The tablet receives the video stream, performs RTSP decapsulation, and sends it to the tablet's decoder to obtain the projected layer (generated from the projected image). The projected image layer is sent to the surfaceflinger module for compositing. Understandably, there is an initial delay between the tablet receiving stylus input and acquiring the projected image layer.

[0165] Path (2): The tablet PC framework layer includes a drawing parameter learning service, which obtains the PC's drawing parameters based on the projection layer (the writing trajectory in the projection image). The tablet PC framework layer also includes a local drawing application, which uses the drawing parameters obtained from the drawing parameter learning service to perform local drawing based on the information input by the stylus, generating a local layer. For example, a top View (PointerLocationView) is created above all windows. PointerLocationView is disabled by default; if the projection service in the framework layer is enabled, local drawing is performed using the drawing parameters obtained from the drawing parameter learning service.

[0166] The surface flinger module of the tablet frame layer composites the local layer and the projected layer into an output buffer, and then sends the output buffer to the display compositer (HwComposer, HWC) module. The HWC module performs layer compositing and display. In this way, the tablet screen displays the composite image of the local layer and the projected layer.

[0167] The display method provided in this application embodiment has a lower latency in generating a local layer on the tablet than in acquiring a projection layer from the PC. The tablet performs local drawing and overlays the local layer onto the projection layer, reducing the distance between the stylus and the writing trajectory and improving responsiveness. Furthermore, the tablet uses the PC's drawing parameters for local drawing, ensuring that the locally drawn writing trajectory matches the writing trajectory acquired from the PC in terms of line color, line width, and line texture, thus guaranteeing a consistent user experience.

[0168] In one scenario, a tablet and a PC are wirelessly connected, with the tablet acting as the PC's input device. Application 1 runs on the PC, and its interface is projected onto the tablet's screen. The user drags and drops the window of Application 1 on the tablet screen, moving its position on both the tablet and PC screens.

[0169] For example, such as Figure 18 As shown, application one and application two are running on the PC, and the windows of application one and application two are displayed on the PC desktop. In one implementation, as... Figure 19 As shown, Application 1 and Application 2 each generate their own window images and send them to Windows DWM; DWM combines the window images of each application onto the desktop to generate the PC's display interface.

[0170] The PC will also project its interface onto the tablet screen. For example... Figure 18 As shown, the PC captures images of each application window and the desktop workspace, and sends these images to the tablet. The PC also sends window position information such as the size and coordinates of each application window to the tablet. The tablet receives the images of each application window and the desktop workspace, along with window position information such as the size (e.g., the number of pixels occupied by the window's length and width) and coordinates (e.g., the pixel position of the window's center point). Based on these images and position information, the tablet generates images of each application's window and the desktop image, performs local layer compositing, and displays each application's window on the desktop image, generating the tablet's display interface (i.e., displaying the PC's projection interface). Thus, the tablet displays the PC's projection interface, and the tablet's display interface is identical to the PC's. In one implementation, the tablet saves the window images and position information of each application's window, as well as the desktop image.

[0171] Users can input on the tablet screen. For example, they can draw lines within application windows using a stylus; or they can tap the title bar of an application window on the tablet screen using a stylus or their finger and move the stylus to drag the application window (i.e., perform a window dragging operation). For example, such as... Figure 20 As shown, the user uses the stylus 300 to tap the title bar inside the application 1 window on the screen of the tablet computer 100 and moves the stylus to move the position of the application 1 window, thus dragging the application 1 window.

[0172] In one implementation, such as Figure 21 As shown, the tablet receives input from the stylus and generates corresponding events based on the input. For example, if the stylus click is determined to be within the title bar of the application window and the stylus position moves (i.e., the user drags the application window), a drag event is generated.

[0173] If it's determined that it's not a drag-and-drop event, the tablet transmits the handwriting input to the PC via wireless communication (e.g., Bluetooth). The PC performs corresponding operations based on the input information, such as drawing the writing trajectory based on the input information and drawing parameters. Each application on the PC generates its own window image and sends it to Windows' DataWindow (DWM). The DWM composites the window images of each application onto the desktop image based on the position information of each application window (including the application window's size, coordinates, etc.), generating and displaying the PC's display interface image. For example, the display interface image includes multiple layers, with the bottom layer including the desktop image, and each layer above the bottom layer including the window image of one application. The PC performs video encoding on each layer, generating a video stream corresponding to each layer. The PC sends the video streams corresponding to each layer to the tablet via wireless communication (e.g., Wi-Fi). The tablet receives the video streams corresponding to each layer, performs video decoding, and obtains the images of each layer (i.e., the window images of each application). The PC also sends the position information of each application window to the tablet. In one implementation, the tablet saves the window images of each application, the position information of the application's window, and the desktop image. The tablet computer composites the window images of one or more applications onto the desktop image based on the position information of each application window, creating a projected interface. The tablet computer's screen then displays this projected interface. In this way, the tablet computer and the PC display the same interface.

[0174] If a drag-and-drop event is confirmed, the tablet transmits the stylus input (a drag-and-drop operation on the window of application 1) and the drag-and-drop event to the PC via wireless communication (e.g., Bluetooth). The application corresponding to the drag-and-drop event on the PC redraws the window image of application 1 and obtains the current position information of the window of application 1 based on the drag-and-drop operation information (e.g., the current input coordinates). Each application on the PC sends its respective window image and window position information to Windows' DWM (Device Manager). The DWM composites the window images of each application onto the desktop image based on the position information of each application window (including the size and coordinates of the application window), generating and displaying the PC's display interface image. The PC encodes the window images of each application and the desktop image into a video stream and sends the video stream to the tablet via wireless communication (e.g., Wi-Fi) (e.g., the PC sends a screen mirroring update request to the tablet, which includes the window images of each application and the desktop image). The tablet receives the video stream, decodes it, and obtains the window images of each application and the desktop image. The PC also sends the position information of each application window to the tablet. In one implementation, the tablet saves the window images of each application, the position information of the application's window, and the desktop image. On the other hand, the tablet computer determines the application (Application 1) corresponding to the drag event based on information from the stylus input (drag operation on the window of Application 1) (such as the input coordinates of the previous frame and the current input coordinates) and the saved position information of each application. It also redraws the window image of Application 1 based on the stylus input (drag operation on the window of Application 1) information (such as the current input coordinates) and the saved window image of Application 1. The tablet computer obtains the display position of the window image of Application 1 on the desktop image based on the current input coordinates of the stylus, and composites the window images of each application onto the desktop image based on the window images and position information of each application, thus creating a local interface. The composite local interface is displayed on the tablet computer screen. Understandably, in response to the user's drag operation on the window of Application 1, the position of the window of Application 1 on the tablet computer moves along the drag trajectory generated by the window drag operation.

[0175] For example, such as Figure 22 As shown, the tablet displays the PC's screen mirroring interface. Application 1 is running on the PC. The screen mirroring interface 901 includes a window 902 of application 1, located at position 1 within the interface. The tablet receives a user's drag operation on the window of application 1. The tablet obtains the window's position 2 at time ta, draws an image of window 903 based on the saved image of window 902, and displays application 1's window 903 at position 2 on the desktop image at time tb.

[0176] The tablet also sends information about the user's drag-and-drop operation on the application 1 window at time ta to the PC. The PC obtains the window drag-and-drop operation's position 2 at time ta, draws the image of window 903 based on the saved image of window 902, and displays application 1's window 903 at position 2 on the desktop image at time tc. The PC sends the image of application 1's window 903, its position 2 information, and the desktop image to the tablet. The tablet receives and saves the image of application 1's window 903, its position 2 information, and the desktop image at time td.

[0177] The tablet computer obtains the position 3 of the window drag operation at time td, draws the image of window 904 based on the saved image of window 903, and displays the window 904 of application 1 at position 3 of the desktop image at time te.

[0178] The application window display method provided in this embodiment involves a tablet computer receiving a user's drag operation on an application window and then locally drawing the window image based on the drag operation information and a saved image of the application window. The tablet computer's local drawing of the window image has low latency, which reduces the latency of displaying content (application windows) on the tablet computer and improves the responsiveness of the stylus.

[0179] In one implementation, the tablet performs a local drawing once for each frame of input received from the display interface; for example, such as... Figure 23 As shown, a user drags an application window on a tablet using a stylus. The tablet performs local drawing once for each frame of the display interface received input from the stylus, and the application window moves along the drag path created by the drag operation. In another implementation, the tablet performs local drawing after receiving multiple inputs. In yet another implementation, the tablet performs local drawing when it detects the end of the window drag operation, i.e., when the stylus or finger leaves the screen.

[0180] It is understood that, in order to achieve the aforementioned functions, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in a hardware-driven or software-driven manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0181] This application embodiment can divide the above-described electronic device into functional modules based on the method example described above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0182] In one example, please refer to Figure 24 The diagram illustrates a possible structural schematic of the electronic device involved in the above embodiments. The electronic device 1000 includes: a processing unit 1001, a storage unit 1002, a communication unit 1003, and a display unit 1004.

[0183] The processing unit 1001 is used to control and manage the actions of the electronic device 1000. For example, it can be used to draw a writing trajectory based on input information, and / or other processing steps in the embodiments of this application.

[0184] The storage unit 1002 is used to store the program code and data of the electronic device 1000. For example, it can be used to store drawing parameters obtained from the processing device, the position information of the application window, etc.

[0185] The communication unit 1003 is used to support communication between the electronic device 1000 and other electronic devices. For example, it can be used to send input information to the processing device, receive projected images from the processing device, and / or receive drawing parameters, application window position information, etc. from the processing device.

[0186] Display unit 1004 is used to display the interface of electronic device 1000.

[0187] Of course, the unit modules in the above-mentioned electronic device 1000 include, but are not limited to, the processing unit 1001, the storage unit 1002, the communication unit 1003, and the display unit 1004. For example, the electronic device 1000 may also include a power supply unit, etc. The power supply unit is used to supply power to the electronic device 1000.

[0188] The processing unit 1001 can be a processor or controller, such as a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The storage unit 1002 can be a memory. The communication unit 1003 can be a transceiver, transceiver circuit, etc. The display unit 1004 can be a display screen.

[0189] For example, processing unit 1001 is a processor (such as...) Figure 5 The processor 110 shown can be a memory (such as a storage unit 1002). Figure 5 The internal memory 121 shown, and the communication unit 1003, which can be called the communication interface, include a wireless communication module (such as...). Figure 5 The wireless communication module 160 shown is illustrated, and the display unit 1004 is a display screen (such as...). Figure 5 The display screen 150 shown can be a touch screen, which can integrate a display panel and a touch panel. The electronic device 1000 provided in this embodiment can be... Figure 5 The electronic device 100 shown. The aforementioned processor, memory, communication interface, display screen, etc., can be connected together, for example, via a bus.

[0190] This application also provides a computer-readable storage medium storing computer program code. When a processor executes the computer program code, the electronic device performs the method described in the above embodiments.

[0191] This application also provides a computer program product that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0192] In this application, the electronic device 1000, computer-readable storage medium, or computer program product provided in the embodiments are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0193] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0195] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or as a software functional unit.

[0196] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROMs, magnetic disks, or optical disks.

[0197] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for displaying content, characterized in that, include: The first electronic device displays the screen projection interface of the first application and receives the first input from the stylus; the first application runs on the second electronic device. The first electronic device sends the first input information to the second electronic device; The first electronic device receives a screen projection image from the second electronic device and updates the screen projection interface according to the screen projection image. The screen projection image includes a first writing trajectory drawn by the second electronic device based on the first input information. The first electronic device obtains the drawing parameters of the second electronic device based on the first writing trajectory. The drawing parameters include at least one of the following: brush type parameters, line color parameters, line width parameters, and line texture parameters. The first electronic device receives a second input from the stylus; The first electronic device draws the second writing trajectory based on the drawing parameters of the second electronic device and the information of the second input; The first electronic device displays the second writing trajectory, which is a continuation of the first writing trajectory.

2. The method according to claim 1, characterized in that, The first electronic device obtains the drawing parameters of the second electronic device based on the first writing trajectory, including: if the first electronic device does not receive pen input within a preset time threshold before receiving the first input, it performs the step of obtaining the drawing parameters of the second electronic device.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The first electronic device uses preset drawing parameters to draw a third writing trajectory based on the first input information. The third writing trajectory has the same line trajectory as the first writing trajectory.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The first electronic device sends the second input information to the second electronic device; The first electronic device receives a second image from the second electronic device. The second image includes a fourth writing trajectory drawn based on the information of the second input, the fourth writing trajectory having the same line trajectory as the second writing trajectory.

5. The method according to claim 4, characterized in that, The fourth writing trajectory also has at least one of the following elements that are the same as the second writing trajectory: line color, line width, and line texture.

6. The method according to claim 1 or 2, characterized in that, The projected image is displayed on the first layer, the second writing trajectory is displayed on the second layer, and the second layer is superimposed on the first layer.

7. The method according to claim 4, characterized in that, The second writing trajectory stops displaying after a preset duration; or, the second writing trajectory stops displaying after the fourth writing trajectory is displayed.

8. An electronic device, characterized in that, include: One or more processors; Display screen; Memory; The memory stores one or more computer programs, the one or more computer programs including instructions that, when executed by the electronic device, cause the electronic device to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium storing instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-7.

Images