Display method and related device
Through wired connection and multi-signal interface, the interaction and data transmission rate problems of terminals such as tablets and other terminals when expanding the screen are solved, and efficient display control and function expansion are achieved.
Patent Information
- Application Number
- PCT/CN2024/082672
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2024-03-20
- Publication Date
- 2024-11-21
AI Technical Summary
When terminals such as tablets are used as extended screens, users cannot interact directly with them, and the data transmission rate and frame rate of wireless screen projection technology are limited, which cannot meet the screen projection scenarios with high requirements for data transmission.
By establishing a wired connection, using interfaces (such as DP and HDMI) that support multiple signal inputs and outputs, the first device realizes displaying the content of the second device, and allows the user to control the display content of the second device through the operation of the first device, It includes image data format conversion and audio data separation through a bridge chip to ensure the speed and stability of data transmission.
It is realized that while ensuring the screen projection effect, the user can control the display content of the second device based on operation, improve the speed and stability of data transmission, and expand the functionality of the device.
Smart Images

Figure CN2024082672_21112024_PF_FP_ABST
Abstract
Description
A display method and related equipment
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 12, 2023, with application number 202310539590.4 and application name “A display method and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of terminal technology, and in particular to a display method and related equipment. Background Art
[0003] Currently, terminals such as tablet computers (PADs) can be used not only as standalone devices but also as extended screens. When used as standalone devices, tablet computers and other terminals can not only meet users' communication needs but also provide them with channels for obtaining external information and a variety of entertainment methods. When used as an extended screen for another electronic device (such as a mobile phone or laptop), they can be used to display the screen projection of the other device.
[0004] On the one hand, when a tablet or other terminal is used as an extended screen for another device, the user cannot interact directly with it and can only change the displayed image by interacting with the other device. On the other hand, if wireless screen projection technology is used, the data transmission rate and frame rate are relatively limited. This means that for some screen projection scenarios with high data transmission rate and frame rate requirements, wireless screen projection technology is not a good choice.
[0005] Therefore, how to ensure the screen projection effect so that a terminal such as a tablet computer can display the screen displayed by another electronic device and also trigger the screen displayed by the other electronic device to change based on user operations.
[0006] Summary of the Invention
[0007] This application provides a display method and related devices. The display method can be applied to a first device and a second device that establish a wired connection. According to the display method, the first device can display the display content of the second device. Furthermore, a user can control the display content of the second device to change by performing operations on the first device.
[0008] In a first aspect, the present application provides a display method. The display method can be applied to a first device including a display screen and a first interface. The first interface can be an interface that supports first-type signal input and second-type signal output. The display method may include: when the first device establishes a wired connection with a second device via the first interface, the first device may display a first interface on the display screen, the first interface may include a first display area, and the first display area may be used to display the content of the display interface of the second device; the first device may detect a first operation acting on the first display area, and in response to the first operation, the content displayed in the first display area changes.
[0009] In the solution provided in the present application, when a wired connection is established between the first device and the second device, the first device can display the content of the display interface of the second device in the first display area, and can also control the second device to perform corresponding operations based on user operations acting on the first display area. It can be understood that the first device can be used as an extended screen for the second device. Moreover, for operations that are inconvenient to perform on the second device (for example, writing, drawing, etc.), the user can control the second device to perform corresponding operations by executing on the first device. Moreover, when the operating systems of the first device and the second device are inconsistent, the first device can not only display the interface of the application supported by its own operating system, but also display the interface of the application that is not supported by its own operating system but supported by the operating system of the second device. The above method not only ensures the speed and stability of screen projection data transmission, but also realizes the functional expansion of the device.
[0010] In some embodiments of the present application, the first device may be the electronic device 100, and the second device may be the electronic device 200. The first interface may be interface 1.
[0011] In some embodiments of the present application, the first type of signal may be a DP signal. In still other embodiments of the present application, the first type of signal may be another type of signal, such as an HDMI signal.
[0012] In some embodiments of the present application, the second type of signal may be a USB signal.
[0013] In some embodiments of the present application, the first display area may be the entire display area (i.e., the entire display area) of the display screen of the first device. For example, the first display area may be the entire display area of the electronic device 100 shown in Figures 2C-2H. In still other embodiments of the present application, the first display area may be a partial display area of the display screen of the first device. For example, the display area 1a in the user interface 108 shown in Figure 2I(a). For example, the display area 1a in the user interface 109 shown in Figure 2I(b). Another example is the display area 1a shown in Figures 2J-2R.
[0014] In some embodiments of the present application, the first display area may be the display area 1a. The relevant description of the display area 1a can be referred to later and will not be repeated here.
[0015] In some embodiments of the present application, the first interface may be a user interface of the electronic device 100 as shown in FIG. 2C to FIG. 2R .
[0016] In some embodiments of the present application, the first operation may be a user operation on the pause playback control 10211 as shown in FIG. 2C . In some embodiments of the present application, the first operation may be a user operation on the play next song control 10212 as shown in FIG. 2C . In some embodiments of the present application, the first operation may be a user operation on the minimize control 1041 as shown in FIG. 2E . In some embodiments of the present application, the first operation may be a user operation on the canvas area 1061 as shown in FIG. 2G . In some embodiments of the present application, the first operation may be a user operation on the exit control 1091 as shown in FIG. 2I (b). In some embodiments of the present application, the first operation may be a user operation on the video application icon 1101 as shown in FIG. 2J .
[0017] It can be understood that the content of the second device display interface is the content included in the user interface displayed by the second device.
[0018] In combination with the first aspect, in a possible implementation, when the first device establishes a wired connection with the second device through a first interface, the first device displays a first interface on the display screen, which may specifically include: the first device can receive first data sent by the second device through the first interface, the first data may include first image data, and the first data may be a first type of signal; the first device can display the first interface on the display screen based on the first image data, and the content displayed in the first display area in the first interface corresponds to the first image data.
[0019] In the solution provided in this application, a first device can receive image data sent by a second device via a first interface and display the image data sent by the second device on a first display area of the first device's display screen. Furthermore, the first interface supports first-type signal input and second-type signal output. Thus, the first device can display the content of the second device's display interface.
[0020] In some embodiments of the present application, the first data may be a DP signal, i.e., DP signal data, also known as a DP data packet (for example, a DP data packet transmitted from interface 2 of electronic device 200 to electronic device 100 as shown in Figures 5A and 5B). The first image data may be image data in a DP data packet (as shown in Figures 5A and 5B).
[0021] In some other embodiments of the present application, the first data may be an HDMI signal, that is, HDMI signal data, which may also be referred to as HDMI data.
[0022] In conjunction with the first aspect, in one possible implementation, the format of the first image data included in the first data is in a first format. After the first device receives the first data sent by the second device via the first interface, the method may further include: the first device may convert the format of the first image data from the first format to the second format via a bridge chip. The first device may display a first interface on a display screen based on the first image data, which may specifically include: the first device may display the first interface on a display screen based on the first image data in the second format.
[0023] In the solution provided in the present application, the first data transmitted from the second device to the first device may not be directly usable, and the first device may convert the format of the first data and then transmit it to the display screen for display. For example, the first device may include a bridge chip, and convert the first image data in the first format included in the first data into the first image data in the second format through the bridge chip, and then display the corresponding picture on the display screen based on the first image data in the second format. Through this method, for the image data transmitted from the second device to the first device that cannot be directly used, the first device can first convert the format through the bridge chip, so that the first device can smoothly display the image data transmitted by the second device.
[0024] It can be understood that the bridge chip can be used to convert image data in a first format into image data in a second format.
[0025] In some embodiments of the present application, the first data may be a DP data packet (as shown in Figures 5A and 5B), the first format may be a DP data format, and the second format may be a CSI data format.
[0026] In some further embodiments of the present application, the first data may be an HDMI signal, the first format may be an HDMI data format, and the second format may be a CSI data format.
[0027] In conjunction with the first aspect, in one possible implementation, the first data may further include first audio data. Before the first device converts the format of the first image data from the first format to the second format using the bridge chip, the first device may separate the first image data and the first audio data from the first data using the bridge chip. Furthermore, after the first device separates the first image data and the first audio data from the first data using the bridge chip, the first device may play the first audio data.
[0028] In the solution provided in the present application, the second device can not only transmit image data to the first device through the first interface, but also transmit audio data to the first device through the first interface. For example, the first data may include first image data and first audio data. It is understandable that after the first device receives the first data transmitted by the second device, the first image data and the first audio data can be separated by the bridge chip, and the first image data can be format-converted by the bridge chip, and the first image data after format conversion can be transmitted to the SOC through the corresponding interface (for example, CSI interface), and the separated first audio data can be transmitted to the SOC through the corresponding interface (for example, audio interface 1). This method can solve the problem that the second device cannot play audio in some scenarios.
[0029] In some embodiments of the present application, the first data may be a DP data packet (as shown in FIG. 5A and FIG. 5B ), and the first audio data may be audio data in the DP data packet (as shown in FIG. 5A and FIG. 5B ).
[0030] In conjunction with the first aspect, in a possible implementation, the first display area may be the entire display area of the display screen. In this case, the first image data may be image data corresponding to the content displayed in the first display area.
[0031] In the solution provided in this application, the first device can display the content of the display interface of the second device in full screen.
[0032] In combination with the first aspect, in one possible implementation, the first interface may further include a second display area. The second display area may be used to display display content corresponding to the first device's own system. The first device displays the first interface on the display screen based on the first image data, which may specifically include: the first device may process the first image data and the second image data, transmit the processed image data to the display screen, and display the first display area and the second display area on the display screen. The second image data is image data generated by the first device's own system. The content displayed in the first display area corresponds to the first image data. The content displayed in the second display area corresponds to the second image data.
[0033] In the solution provided by this application, when a wired connection is established between a first device and a second device, the first device can not only display the content of the second device's display interface in the first display area, but also display the screen generated by its own system in the second display area. In this way, the first device can not only display the screen generated by its own system, but also serve as an extended screen for the second device, which not only ensures the speed and stability of screen projection data transmission, but also meets the user's demand for extended screen display.
[0034] In some embodiments of the present application, the second display area may be the display area 1b. The relevant description of the display area 1b can be referred to later and will not be repeated here.
[0035] In some embodiments of the present application, the first device may overlay the first image data and the second image data to obtain processed image data (as shown in Figures 5A and 5B), and transmit the processed image data to a display screen for display. Specifically, after overlaying the first image data and the second image data, the first device may display the image corresponding to the first image data in a first display area of the display screen, and display the image corresponding to the second image data in a second display area.
[0036] In conjunction with the first aspect, in one possible implementation, the format of the first image data included in the first data may be in a first format. The first device processing the first image data and the second image data may specifically include: the first device converting the format of the first image data from the first format to the second format via a bridge chip; and the first device performing layer overlay processing on the first image data in the second format and the second image data.
[0037] In the solution provided in this application, the first data transmitted from the second device to the first device may not be directly usable. The first device may convert the first data into a format and then transmit it to the display screen for display. The processing performed by the first device on the first image data and the second image data may be layer overlay processing. Specifically, the first device may use the budget of its internal graphics processor to perform layer overlay processing on the first image data and the second image data, superimposing them on a single screen for display.
[0038] In conjunction with the first aspect, in one possible implementation, the first type of signal may be a DP signal. The first data may be DP signal data. The first format may be a DP data format. The second format may be a CSI data format.
[0039] In combination with the first aspect, in a possible implementation, before the first device establishes a wired connection with the second device through the first interface, the method may further include: the first device may control the rest pin of the bridge chip to receive the first data based on a pipeline customized for the bridge chip.
[0040] In the solution provided in the present application, the first device can customize the usecase and / or pipeline for the bridge chip, and control the rest pin of the bridge chip to receive the first data.
[0041] In combination with the first aspect, in a possible implementation, in response to the first operation, the content displayed in the first display area changes, which may specifically include: in response to the first operation, the first device sends information related to the contact position coordinates to the second device, and receives second data sent by the second device through the first interface. The information related to the contact position coordinates is a second type signal. The second data includes third image data. The second data is a first type signal. The third image data is updated image data obtained by the second device based on the information related to the contact position coordinates. The third image data is different from the second image data. In response to the first operation, the content displayed in the first display area changes, which may specifically include: the first device displays a second interface on the display screen based on the third image data. The content displayed in the first display area in the second interface corresponds to the third image data. The content displayed in the first display area in the second interface is different from the content displayed in the first display area in the first interface.
[0042] In the solution provided in the present application, the first device can detect the corresponding user operation and send the contact position coordinate related information corresponding to the user operation to the second device. The second device can perform the corresponding operation based on the contact position coordinate related information to obtain updated image data. On the one hand, the second device can display the updated image data on its display screen. That is, the second device can display the updated display interface on the display screen. On the other hand, the second device can send the updated image data to the first device so that the first device can display the content of the updated display interface of the second device in a timely manner.
[0043] In some embodiments of the present application, the contact position coordinate related information may be the “related operation information” mentioned later.
[0044] In some embodiments of the present application, when the first device does not obtain pressure information, the information related to the contact position coordinates may be TP reporting point information. In some embodiments of the present application, when the first device obtains pressure information, the information related to the contact position coordinates may be processed TP reporting point information. The information related to the contact position coordinates may specifically include the contact position coordinates.
[0045] In conjunction with the first aspect, in one possible implementation, the first type of signal may be a DP signal. The second type of signal may be a USB signal. The first interface may be a Type-C interface that supports DP signal input and USB signal output. The first device may be provided with a second interface, which may be a Type-C interface that supports DP signal output.
[0046] In conjunction with the first aspect, in one possible implementation, the first interface further includes a second display area. Before the first device displays the first interface on the display screen, the method may further include: the first device displays a third interface on the display screen. The content of the third interface is the same as the content displayed in the second display area of the first interface.
[0047] In the solution provided by this application, when a wired connection is established between a first device and a second device, the first device can not only display the content of the second device's display interface in the first display area, but also display images generated by its own system in the second display area. This method realizes the expansion of device functionality.
[0048] In combination with the first aspect, in a possible implementation, after the first device displays the first interface on the display screen, the method may further include: in response to an operation acting on the second display area, the content displayed in the second display area changes.
[0049] In some embodiments of the present application, the second display area may be the display area 1b in the user interface 112 as shown in FIG2K . In this case, the operation on the second display area may be a user operation on the memo application icon 1011 as shown in FIG2K .
[0050] In combination with the first aspect, in a possible implementation, after the first device displays the first interface on the display screen, the method may further include: in response to an operation acting on the first interface, the first device may increase the first display area.
[0051] In some embodiments of the present application, the first interface may be the user interface 109 shown in FIG2I(b). In this case, the operation performed on the first interface may be a rightward sliding operation performed on the boundary between the display area 1a and the display area 1b as shown in FIG2I(b).
[0052] In combination with the first aspect, in one possible implementation, after the first device displays the first interface on the display screen, the method may further include: the first device detects a second operation, and in response to the second operation, the first device may enter an annotation mode; the first device detects an operation acting on the display screen, and in response to the operation acting on the display screen, displays a first batch of annotation traces on the first display area; the first device detects a third operation, and in response to the third operation, the first device may save a picture containing the first batch of annotation traces.
[0053] In some embodiments of the present application, the second operation may be a two-finger sliding operation on the user interface 112 as shown in FIG. 2L .
[0054] In some embodiments of the present application, the operation performed on the display screen may specifically be: a user operation performed on the display area 1a as shown in FIG2M .
[0055] In some embodiments of the present application, the third operation may be a user operation acting on a completion control in the toolbar 1131 as shown in FIG. 2M .
[0056] In some embodiments of the present application, the third operation may be a user operation performed on the screenshot control 11312 in the user interface 114 as shown in FIG. 2N .
[0057] In some embodiments of the present application, the third operation may be a user operation performed on a completion control in the screenshot toolbar 1182 as shown in FIG2R .
[0058] In a second aspect, the present application provides a display method. The display method can be applied to a display screen and a second device of a third interface. The third interface is an interface that supports first-type signal output and second-type signal input. The method may include: when the second device establishes a wired connection with the first device through the third interface, the second device can receive contact position coordinate related information sent by the first device, determine the target position coordinates on the display screen based on the contact position coordinate related information, and obtain updated image data based on the target position coordinates; the second device can transmit the updated image data to the display screen for display, and send the updated image data to the first device.
[0059] In the solution provided in the present application, the first device can detect the corresponding user operation and send the contact position coordinate related information corresponding to the user operation to the second device. The second device can perform the corresponding operation based on the contact position coordinate related information to obtain updated image data. On the one hand, the second device can display the updated image data on its display screen. That is, the second device can display the updated display interface on the display screen. On the other hand, the second device can send the updated image data to the first device so that the first device can display the content of the updated display interface of the second device in a timely manner.
[0060] In a third aspect, the present application provides an electronic device. The electronic device may include a first interface, a display screen, one or more memories, and one or more processors. The one or more processors are coupled to the first interface, the display screen, and the one or more memories. The first interface may be configured to receive a first type of signal and to send a second type of signal. The memory may be configured to store a computer program. The processor may be configured to invoke the computer program, causing the electronic device to perform the method described in the first aspect or any one of the implementations of the first aspect.
[0061] It can be understood that the electronic device involved in the third aspect can be the first device mentioned in the first aspect above.
[0062] In combination with the third aspect, in a possible implementation, the electronic device may further include a second interface, which may be used to output the first type of signal.
[0063] In conjunction with the third aspect, in a possible implementation, the first type signal may be a DP signal, and the second type signal may be a USB signal.
[0064] In a fourth aspect, the present application provides an electronic device. The electronic device may include a first interface, a bridge chip, a processor, and a display screen. The first interface is connected to the bridge chip, which is connected to the processor, which is connected to the display screen. The first interface can be used to connect to a second device, receive first data sent by the second device, and transmit the first data to the bridge chip. The first data is a first type of signal. The bridge chip can be used to convert first image data in a first format within the first data into first image data in a second format, and transmit the second format first image data to the processor. The processor can be used to process the second format first image data and image data generated by the electronic device's own system to obtain first processed image data. The display screen can be used to display the first processed image data. The first interface can also be used to send information related to contact position coordinates to the second device. The contact position coordinate information is a second type of signal. The first interface can also be used to receive second data sent by the second device and transmit the second data to the bridge chip. The second data is a first type of signal. The bridge chip can also be used to convert third image data in a first format within the second data into third image data in a second format, and transmit the second format third image data to the processor. The third image data is updated image data obtained by the second device based on the contact position coordinate information. The processor can also be used to process the third image data in the second format and the image data generated by the electronic device's own system to obtain second processed image data. The display screen can also be used to display the second processed image data.
[0065] It can be understood that the electronic device involved in the fourth aspect can be the first device mentioned in the first aspect above.
[0066] In some embodiments of the present application, the first processed image may be processed image data obtained by the electronic device 100 after processing input image data and image data generated by its own system (as shown in Figures 5A and 5B).
[0067] In some embodiments of the present application, the second processed image may be processed image data obtained by the electronic device 100 processing the input updated image data and the image data generated by its own system (as shown in FIG5B ).
[0068] In combination with the fourth aspect, in a possible implementation, the first processed image data is image data obtained by performing layer overlay processing on the first image data in the second format and the image data generated by the electronic device's own system.
[0069] In combination with the fourth aspect, in a possible implementation manner, the first processed image data is first image data in a second format.
[0070] In combination with the fourth aspect, in a possible implementation, the first type signal is a DP signal, and the second type signal is a USB signal.
[0071] In a fifth aspect, the present application provides an electronic device. The electronic device may include a third interface, a display screen, one or more memories, and one or more processors. The one or more processors are coupled to the third interface, the display screen, and the one or more memories. The third interface may be configured to transmit a first type of signal and receive a second type of signal. The memory may be configured to store a computer program. The processor may be configured to invoke the computer program to cause the electronic device to perform the method described in the second aspect.
[0072] In a sixth aspect, the present application provides a display system. The display system may include a first device and a second device. The first device may include a first interface. The second device may include a third interface. The first device may be connected to the third interface of the second device via the first interface. The first device is the electronic device described in the third aspect or any implementation of the third aspect, or the fourth aspect or any implementation of the fourth aspect. The second device is the electronic device described in the fifth aspect or any implementation of the fifth aspect.
[0073] In conjunction with the sixth aspect, in one possible implementation, the display system may further include a switching device. The first device may be connected to the third interface of the second device via the first interface, which may specifically include: the first device may be connected to the switching device via the first interface, and the switching device may be connected to the third interface of the second device.
[0074] In combination with the sixth aspect, in a possible implementation, the third interface may be an interface that supports DP signal output and USB signal input.
[0075] In a seventh aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on an electronic device, causes the electronic device to execute the method described in the first aspect or any one of the implementations of the first aspect.
[0076] In an eighth aspect, an embodiment of the present application provides a chip. The chip can be applied to an electronic device, and the chip includes one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to execute the method described in the first aspect or any one of the implementations of the first aspect.
[0077] In a ninth aspect, an embodiment of the present application provides a computer program product comprising instructions. When the computer program product is run on an electronic device, the electronic device executes the method described in the first aspect or any one of the implementations of the first aspect.
[0078] In a tenth aspect, the present application provides a computer storage medium comprising computer instructions, which, when executed on an electronic device, cause the electronic device to execute the method described in the second aspect or any one of the implementations of the second aspect.
[0079] In an eleventh aspect, an embodiment of the present application provides a chip. The chip can be applied to an electronic device, and the chip includes one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to execute the method described in the second aspect or any one of the implementations of the second aspect.
[0080] In a twelfth aspect, an embodiment of the present application provides a computer program product comprising instructions. When the computer program product is run on an electronic device, the electronic device executes the method described in the second aspect or any one of the implementations of the second aspect.
[0081] It can be understood that the electronic device provided in the third and fourth aspects, the display system provided in the sixth aspect, the computer storage medium provided in the seventh aspect, the chip provided in the eighth aspect, and the computer program product provided in the ninth aspect are all used to execute the method described in the first aspect or any one of the implementations of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of any possible implementation of the first aspect, and will not be repeated here. Similarly, the electronic device provided in the fifth aspect, the display system provided in the sixth aspect, the computer storage medium provided in the tenth aspect, the chip provided in the eleventh aspect, and the computer program product provided in the twelfth aspect are all used to execute the method described in the second aspect or any one of the implementations of the second aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of any possible implementation of the second aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0082] 1A-1B are schematic diagrams of a group of electronic devices connected according to an embodiment of the present application;
[0083] Figures 2A-2H are a set of user interface schematic diagrams provided in an embodiment of the present application;
[0084] Figures 2I(a)-2I(b) are another set of user interface schematic diagrams provided in an embodiment of the present application;
[0085] Figures 2J-2U are another set of user interface schematic diagrams provided in an embodiment of the present application;
[0086] FIG3 is a schematic diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application;
[0087] FIG4 is a schematic diagram of the hardware structure of an electronic device 200 provided in an embodiment of the present application;
[0088] 5A-5B are schematic diagrams of some data transmissions provided in embodiments of the present application;
[0089] FIG6A is a schematic diagram of a screen sensing area of an electronic device 100 provided in an embodiment of the present application;
[0090] FIG6B is a schematic diagram of a screen operation area of an electronic device 200 provided in an embodiment of the present application;
[0091] FIG7 is a software architecture diagram of an electronic device 100 provided in an embodiment of the present application;
[0092] FIG8A is a schematic diagram of an image data architecture provided in an embodiment of the present application;
[0093] FIG8B is a schematic diagram of an audio data architecture provided in an embodiment of the present application;
[0094] FIG8C is a schematic diagram of a USB data packet transmission architecture provided in an embodiment of the present application. DETAILED DESCRIPTION
[0095] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.
[0096] It should be understood that the terms "first," "second," and the like in the specification, claims, and drawings of this application are used to distinguish between different objects, rather than to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.
[0097] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.
[0098] The embodiments of the present application provide a display method and related devices. The display method can be applied to an electronic device 100 and an electronic device 200 that have established a wired connection (i.e., a wired communication connection). According to the display method, the display content of the electronic device 200 can be displayed on the electronic device 100, and the electronic device 100 can trigger the interface of the electronic device 200 to change in response to the corresponding user operation. In one possible implementation, the electronic device 100 can not only display the display content of the electronic device 200, but also display the display content corresponding to the electronic device 100's own system. In this case, the electronic device 100 can respond to user operations on the display content of the electronic device 200 displayed on the electronic device 100, and can also respond to user operations on the display content corresponding to the electronic device 100's own system displayed on the electronic device 100. Through this display method, a user can view and control the display content on a device with which a wired connection is established, which not only improves the speed and stability of transmitting screen projection data, but also realizes the functional expansion of the device.
[0099] The following first introduces the connection between the electronic device 100 and the electronic device 200 involved in this application.
[0100] As shown in Figure 1A, the electronic device 100 may include an interface 1. It is understandable that the interface 1 can support DP signal (i.e., data in DP data format, or DP data packet) input and USB signal (i.e., data in USB data format, or USB data packet) output. In some embodiments of the present application, the interface 1 may be a USB Type-C interface provided with an SS channel and a DP / DM channel. Among them, the interface 1 can implement the input of the DP signal based on the SS channel, and implement the output of the USB signal based on the DP / DM channel. In other words, the interface 1 can receive DP signals transmitted by other devices, and can transmit USB signals to other devices. It is understandable that the specific meanings of the SS channel and the DP / DM channel can be referred to the relevant technical documents and will not be elaborated here.
[0101] DP stands for Display Port, a digital video interface standard developed by a coalition of PC and chip manufacturers and standardized by the Video Electronics Standards Association. Interface 1 can support the DP protocol. USB, short for Universal Serial Bus, is a serial bus standard and a technical specification for input and output interfaces. There are many types of USB interfaces. For example, USB Standard Type-A, USB Mini Type-A, USB Micro Type-A, USB Standard Type-B, USB Mini Type-B, USB Micro Type-B, and USB Type-C. Among them, USB Type-C can be abbreviated as USB-C or Type-C. It is a Universal Serial Bus (USB) interface format and a USB hardware interface specification. Interface 1 can support USB-related protocols.
[0102] It is understandable that interface 1 can be not only a Type-C interface, but also other types of data transmission interfaces, and this application does not limit this.
[0103] It is understood that the electronic device 100 can be a terminal device provided with a display screen. For example, a tablet computer, a personal computer (PC), an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. It is understood that this application does not limit the specific type of the electronic device 100.
[0104] It should be noted that the electronic device 100 may further include an interface 3. Interface 3 may be a USB interface with functions such as charging, data transmission, and display output. It is understandable that interface 3 may specifically be a Type-C interface, or may be other types of USB interfaces (for example, a Type-A interface and a Type-B interface). It is worth noting that interface 3 can be set to DP OUT (i.e., DP output). This means that, unlike interface 1, interface 1 supports DP input, while interface 3 supports DP output, i.e., interface 3 supports outputting DP signals.
[0105] It is understandable that the above-mentioned DP input / output can be achieved by adding an additional DP video bus to the original USB bus in the Type-C interface.
[0106] As shown in Figure 1A, the electronic device 200 may include an interface 2. It is understandable that the interface 2 can support DP signal output and USB signal input. In some embodiments of the present application, the interface 2 may be a Type-C interface. The interface 2 may have the same function as the interface 3, that is, the interface 2 may be a Type-C interface that supports USB standard charging, data transmission, display output, and supports DP output and other functions. In some embodiments of the present application, the interface 2 may also be other types of data transmission interfaces such as HDMI, Type-A interface, and Type-B interface.
[0107] HDMI is a digital video / audio interface technology that meets the standards of the high-definition era. HDMI can be used in set-top boxes, DVD players, personal computers, and televisions. HDMI can transmit both audio and video signals simultaneously, enabling high-quality transmission of uncompressed high-definition video and multi-channel audio data.
[0108] It is understandable that the electronic device 200 can be a mobile phone, PC, wearable device, vehicle-mounted device, AR / VR device, UMPC, netbook, PDA, game console or a dedicated camera (for example, a SLR camera, a card camera) and other devices. This application does not limit the specific type of the electronic device 200.
[0109] As shown in Figure 1B, the interface 1 of the electronic device 100 can be connected to the interface 2 of the electronic device 200 via a connecting line (i.e., a data line), so that the electronic device 100 and the electronic device 200 establish a wired communication connection (referred to as a wired connection). In this case, the interface 1 and the interface 2 match, and the two ends of the connecting line match the interface 1 and the interface 2, respectively. In some embodiments of the present application, the interface 1 and the interface 2 match, and specifically may include: the interface 1 supports DP signal input and supports USB signal output, while the interface 2 supports DP signal output and supports USB signal input. For example, the interface 1 is a Type-C interface that supports DP signal input and USB signal output, and the interface 2 is a Type-C interface that supports DP signal output and USB signal input.
[0110] In some embodiments of the present application, interface 1 and interface 2 do not match. For example, interface 1 is a Type-C interface, and interface 2 is a Type-A interface. For another example, interface 1 is a Type-C interface, and interface 2 is HDMI. In this case, the electronic device 100 can be connected to the electronic device 200 based on a switching device. For example, interface 1 on the electronic device 100 can be connected to an interface (for example, a Type-A interface) on the switching device via a connecting cable, and the switching device can be connected to the electronic device 200 via a switching cable. It can be understood that the two ends of the above-mentioned connecting cable respectively match the interface 1 on the electronic device 100 and the interface on the switching device, and the two ends of the above-mentioned switching cable respectively match the interface on the switching device and the interface 2 on the electronic device 200. Exemplarily, in the case where interface 2 is HDMI, the switching cable can be an HDMI connecting cable.
[0111] The following describes the display method provided by the present application based on the connection status of the electronic devices shown in FIG. 1A and FIG. 1B and in combination with FIG. 2A to FIG. 2N .
[0112] It is understood that the terms "interface" and "user interface" in the specification, claims, and drawings of this application refer to the media interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on the display screen of an electronic device, where a control can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.
[0113] It is worth noting that the “interface” and “user interface” mentioned in this application can reflect the different contents displayed in the same window at different times, and can also reflect the different contents displayed in different windows at different times.
[0114] It can be understood that this application specifically takes the electronic device 100 in the horizontal screen state as an example to introduce the display method provided by this application, and the specific implementation method when the electronic device 100 is in the vertical screen state can refer to the specific implementation method when it is in the horizontal screen state.
[0115] The following describes the display modes of the electronic device 100 and the electronic device 200 when the electronic device 100 has not established a wired connection with the electronic device 200.
[0116] When the electronic device 100 does not establish a wired connection with the electronic device 200 (as shown in FIG. 1A ), the electronic device 100 and the electronic device 200 may respectively display corresponding user interfaces.
[0117] Exemplarily, when the electronic device 100 and the electronic device 200 do not establish a wired connection (as shown in FIG. 1A ), as shown in FIG. 2A , the electronic device 100 may display the user interface 101 , and the electronic device 200 may display the user interface 201 .
[0118] The user interface 101 displays a page with application icons. The page may include multiple application icons (e.g., a weather application icon, a calendar application icon, an email application icon, a settings application icon, an application store application icon, a memo application icon 1011, and an album application icon). These application icons can be used to trigger the electronic device 100 to launch corresponding applications. A page indicator may also be displayed below these application icons to indicate the positional relationship between the currently displayed page and other pages.
[0119] The user interface 201 displays a page with application icons, which may include multiple application icons (e.g., a video application icon 2011, a drawing application icon 2012, a gallery application icon, and a music application icon 2013). The user interface 201 may also include a taskbar. The taskbar is located below the area where the multiple application images are located. The taskbar may include multiple application icons. One or more of the multiple application icons may be permanently displayed on the taskbar, or may be displayed on the taskbar only after the corresponding application is started.
[0120] It is understandable that when the electronic device 100 does not establish a wired connection with the electronic device 200 , changes in the user interface displayed on the electronic device 100 will not affect the user interface displayed on the electronic device 200 .
[0121] For example, a user can trigger the launch of a music application via the touchpad on electronic device 200. Specifically, electronic device 200 can detect a user action on music application icon 2013. In response to the user action, electronic device 200 can launch the music application and display the corresponding interface. As shown in FIG2B , after launching the music application, electronic device 200 can play music (Song 1 as shown in FIG2B ) and display user interface 202. In this case, electronic device 100 still displays user interface 101. It can be understood that user interface 202 is a music playback interface. User interface 202 can include the title of the currently playing song (Song 1 as shown in FIG2B ) and lyrics (Lyrics 1 as shown in FIG2B ). User interface 202 can also include a playback toolbar 2021. Playback toolbar 2021 can include a play previous song control, a pause control 20211, a play next song control 20212, a playback progress bar, and a playback volume adjustment area. User interface 202 can also include a minimize control, a restore control, and an exit control 2022. The minimize control is used to minimize the current window. The restore control is used to restore the current window to its original size. The exit control is used to trigger closing the current window. For the minimize control, restore control and exit control 2022 included in the user interface 202, the current window they are directed to is the window of the music application shown in the user interface 202.
[0122] It is understandable that the user operations mentioned in this application may include but are not limited to touch (for example, click, etc.), voice control, gestures and other operations, and this application does not limit this.
[0123] The following describes the display mode of the electronic device 100 when the electronic device 100 establishes a wired connection with the electronic device 200.
[0124] In some embodiments of the present application, when the electronic device 100 and the electronic device 200 establish a wired connection (as shown in FIG1B , the interface 1 of the electronic device 100 and the interface 2 of the electronic device 200 are connected via a connection line), the electronic device 100 can display the display content of the electronic device 200. That is, the electronic device 100 can display image data that the electronic device 200 transmits to the electronic device 100 via the wired interface 2 and the interface 1.
[0125] In one possible implementation, when the electronic device 100 and the electronic device 200 establish a wired connection, the electronic device 100 may only display the display content on the electronic device 200, without displaying the display content corresponding to the system of the electronic device 100 itself. The display content corresponding to the system of the electronic device 100 itself can be understood as image data generated by the system of the electronic device 100 itself.
[0126] In another possible implementation, when the electronic device 100 establishes a wired connection with the electronic device 200, the electronic device 100 can display both the display content on the electronic device 200 and the display content corresponding to the system of the electronic device 100 itself.
[0127] In some other embodiments of the present application, when the electronic device 100 establishes a wired connection with the electronic device 200 (as shown in Figure 1B, the interface 1 of the electronic device 100 is connected to the interface 2 of the electronic device 200 via a connecting line), the electronic device 100 may not display the display content on the electronic device 200, but only display the display content corresponding to the electronic device 100's own system.
[0128] In summary, when the electronic device 100 establishes a wired connection with the electronic device 200, the three display modes of the electronic device 100 are as follows:
[0129] 1. The electronic device 100 may only display the display content on the electronic device 200, but not display the display content corresponding to the electronic device 100's own system (ie, image data generated by the electronic device 100's own system).
[0130] 2. The electronic device 100 can display both the display content on the electronic device 200 and the display content corresponding to the system of the electronic device 100 itself.
[0131] 3. The electronic device 100 may not display the display content on the electronic device 200, but only display the display content corresponding to the system of the electronic device 100 itself.
[0132] It is understood that the default display mode after the electronic device 100 and the electronic device 200 establish a wired connection can be any of the above display modes. In the subsequent process, the electronic device 100 can switch from the default display mode to other display modes.
[0133] The detailed introduction of the above three display modes is as follows:
[0134] 1. The electronic device 100 may only display the display content on the electronic device 200, but not display the display content corresponding to the system of the electronic device 100 itself.
[0135] In some embodiments of the present application, the default display mode after the electronic device 100 and the electronic device 200 establish a wired connection may be: the electronic device 100 may only display the display content on the electronic device 200, but not display the display content corresponding to the electronic device 100's own system.
[0136] It is understandable that when electronic device 100 and electronic device 200 establish a wired connection, the display content of electronic device 100 and the display content of electronic device 200 are the same, but the size of the display content of electronic device 100 and the size of the display content of electronic device 200 may be different.
[0137] It should be noted that the aspect ratios of the display areas of electronic device 100 and electronic device 200 may be different. When electronic devices 100 and 200 establish a wired connection, electronic device 100 may scale image data transmitted by electronic device 200 according to the size of the display area of electronic device 100 before displaying it. In this case, electronic device 100 may not necessarily display the display content of electronic device 200 in full screen.
[0138] For example, based on the example shown in FIG2B , the electronic device 100 can establish a wired connection with the electronic device 200. As shown in FIG2C , after the electronic device 100 and the electronic device 200 establish a wired connection, the electronic device 200 can display the user interface 202, and the electronic device 100 can display the user interface 102 and play music (such as song 1 shown in FIG2C ). For ease of description, this application refers to the display content in the user interface 102 as image 1. It is understandable that the display content in the user interface 202 is the same as the content in image 1. However, the size of the user interface 202 and the size of image 1 are not necessarily the same. That is, the user interface 202 and image 1 are not necessarily the same size. For example, as shown in FIG2C , the aspect ratio of the user interface 202 is the same as the aspect ratio of image 1, and the electronic device 100 can display the display content of the electronic device 200 in full screen. If the display areas of the electronic device 100 and the electronic device 200 are different in size, the image 1 and the user interface 202 are different in size. Image 1 can be understood as the image obtained by proportionally enlarging / reducing the user interface 202.
[0139] In one possible implementation, the ratio (ie, aspect ratio) of the display area of the electronic device 100 is different from the ratio of the display area of the electronic device 200. In this case, the electronic device 100 may not display the image 1 in full screen.
[0140] 2C , the user interface 102 may include a play toolbar 1021. As is understood, the play toolbar 1021 may include the same content as the play toolbar 2021, and may include a play previous song control, a pause control 10211, a play next song control 10212, a play progress bar, and a play volume adjustment area.
[0141] In some embodiments of the present application, a user can control electronic device 200 by operating on electronic device 100. Once the display content of electronic device 200 changes, the display content of electronic device 100 also changes accordingly. Of course, a user can also control electronic device 200 by directly operating on electronic device 200.
[0142] For example, electronic device 100 may detect a user operation on the pause control 10211 shown in FIG. 2C . In response to this user operation, electronic device 100 may transmit relevant operation information to electronic device 200. After receiving the relevant operation information, electronic device 200 may determine, based on the relevant operation information, that the specific control triggered by the user is the pause control 20211 on electronic device 200. In response to the relevant operation information, electronic device 200 may pause music playback and display user interface 203 as shown in FIG. Accordingly, electronic device 100 may display user interface 103 as shown in FIG. It is understood that the displayed content of user interface 103 is the same as that of user interface 203. It is understood that user interface 203 may include a play toolbar 2021. Unlike play toolbar 2021 in user interface 202, play toolbar 2021 in user interface 203 does not include a pause control 20211, but instead includes a start play control 20213. Similarly, user interface 103 may include play toolbar 1021. Different from the play toolbar 1021 in the user interface 102 , the play toolbar 1021 in the user interface 103 does not include a pause play control 10211 , but includes a start play control 10213 .
[0143] It is understandable that the "relevant operation information" involved in this application may refer to TP point reporting information, and may also refer to processed TP point reporting information. In some embodiments of the present application, when the electronic device 100 does not obtain pressure sensing information, the relevant operation information may be TP point reporting information. In some embodiments of the present application, when the electronic device 100 obtains pressure sensing information, the relevant operation information may be processed TP point reporting information. The relevant description of TP point reporting information can be found in the following text and will not be elaborated here.
[0144] For example, electronic device 100 may detect a user operation on the play next song control 10212 shown in FIG. 2C . In response to the user operation, electronic device 100 may transmit the relevant operation information to electronic device 200. After receiving the relevant operation information, electronic device 200 may determine, based on the relevant operation information, that the specific control triggered by the user is the play next song control 20212 on electronic device 200. In response to the relevant operation information, electronic device 200 may use electronic device 100 to play the next song (Song 2 as shown in FIG. 2E ) and display user interface 204 as shown in FIG. 2E . Accordingly, electronic device 100 may display user interface 104 as shown in FIG. 2E . It is understood that the display content of user interface 104 is the same as the display content of user interface 204. It is understood that user interface 204 may include a minimize control 2041. Accordingly, user interface 104 may include minimize control 1041. The minimize control can be used to trigger the minimization of the current window.
[0145] Exemplarily, the electronic device 100 can detect a user operation acting on the minimized control 1041 as shown in Figure 2E. In response to the user operation, the electronic device 100 can transmit the relevant operation information to the electronic device 200. After receiving the relevant operation information, the electronic device 200 can determine, based on the relevant operation information, that the specific control triggered by the user is the minimized control 2041 on the electronic device 200. In response to the relevant operation information, the electronic device 200 can minimize the window of the current music application and display it in the taskbar below, and display the user interface 201 as shown in Figure 2F. As shown in Figure 2F, the user interface 201 may include a drawing application icon 2012. Accordingly, the electronic device 100 can display the user interface 105 as shown in Figure 2F. And, as shown in Figure 2F, the electronic device 100 is still playing music. It can be understood that the display content of the user interface 105 is the same as the display content of the user interface 201 as shown in Figure 2F.
[0146] For example, electronic device 200 may detect a user operation on a drawing application icon 2012, as shown in FIG2F . In response to this user operation, electronic device 200 may launch the drawing application and display user interface 205, as shown in FIG2G . Accordingly, electronic device 100 may display user interface 106. Furthermore, as shown in FIG2G , electronic device 100 is still playing music. It is understood that the display content of user interface 106 is the same as the display content of user interface 205. User interface 205 may include canvas area 2051. Similarly, user interface 106 may include canvas area 1061.
[0147] For example, a user can use a stylus to draw on canvas area 1061 as shown in FIG2G . Accordingly, electronic device 100 can detect the user operation on canvas area 1061 and, in response to the user operation, transmit the relevant operation information to electronic device 200. After receiving the relevant operation information, electronic device 200 can determine, based on the relevant operation information, that the specific area triggered by the user is canvas area 2051 on electronic device 200. In response to the relevant operation information, electronic device 200 can display user interface 206 as shown in FIG2H . Accordingly, electronic device 100 can display user interface 107. It is understood that the display content of user interface 107 is the same as the display content of user interface 206. As shown in FIG2H , canvas area 2051 in user interface 206 can display the user's drawing content, and canvas area 1061 in user interface 107 can also display the user's drawing content.
[0148] It is understood that during the above-mentioned drawing process, the content drawn by the user on the canvas area 1061 of the electronic device 100 can be displayed in real time on the canvas area 2051 of the electronic device 200. It is understood that the "real time" mentioned here means that the time difference between the moment the user puts down the pen and the moment the electronic device 200 displays the corresponding drawing trace is less than a certain threshold (for example, 100 milliseconds).
[0149] It is understood that based on the above default display mode, the electronic device 100 may change the display mode in the subsequent process. For example, the electronic device 100 may subsequently display both the display content on the electronic device 200 and the display content corresponding to the electronic device 100's own system. For another example, the electronic device 100 may subsequently not display the display content on the electronic device 200, but only display the display content corresponding to the electronic device 100's own system.
[0150] It should be noted that in the above example, the electronic device 100 can play music through the electronic device 200, and in other embodiments provided in this application, the electronic device 100 can also play music by itself without the need for the electronic device 200 to play, and this application does not impose any restrictions on this.
[0151] 2. The electronic device 100 can display both the display content on the electronic device 200 and the display content corresponding to the system of the electronic device 100 itself.
[0152] In some embodiments of the present application, the default display mode after the electronic device 100 and the electronic device 200 establish a wired connection may be: the electronic device 100 may display both the display content on the electronic device 200 and the display content corresponding to the electronic device 100's own system.
[0153] For ease of description, this application refers to the area on the electronic device 100 used to display the display content of the electronic device 200 as the display area 1a, and the area on the electronic device 100 used to display the display content corresponding to its own system as the display area 1b.
[0154] In some embodiments of the present application, the display area of electronic device 100 is composed of display area 1a and display area 1b, and display area 1a and display area 1b have an overlapping area. For example, display area 1b is the entire display area of electronic device 100, and display area 1a is a floating window on display area 1b.
[0155] In some other embodiments of the present application, the display area of the electronic device 100 consists of a display area 1a and a display area 1b, and the display area 1a and the display area 1b have no overlapping area.
[0156] It should be noted that, in this application, the display area 1a and the display area 1b are described by taking the case where the display area 1a and the display area 1b have no overlapping area as an example, as follows:
[0157] In some embodiments of the present application, after electronic device 100 and electronic device 200 establish a wired connection, the size of display area 1a is the default size of display area 1a, and the size of display area 1b is the default size of display area 1b. It is understood that the default size of display area 1a and the default size of display area 1b can be the same or different. It is understood that the default size of display area 1a and the default size of display area 1b can be set by the user or can retain the default size setting of electronic device 100 when it leaves the factory.
[0158] In one possible implementation, each time the electronic device 100 and the electronic device 200 establish a wired connection, the size of the display area 1a displayed for the first time by the electronic device 100 is the default size of the display area 1a, and the size of the display area 1b displayed for the first time by the electronic device 100 is the default size of the display area 1b.
[0159] In one possible implementation, after electronic device 100 and electronic device 200 establish a wired connection for the first time, the size of display area 1a displayed for the first time by electronic device 100 is the default size of display area 1a, and the size of display area 1b displayed for the first time by electronic device 100 is the default size of display area 1b. In subsequent processes, after electronic device 100 and electronic device 200 establish a wired connection again, the size of display area 1a is the size of display area 1a before the wired connection was last disconnected, and the size of display area 1b is the size of display area 1b before the wired connection was last disconnected.
[0160] It can be understood that the default size of the display area 1 a and the default size of the display area 1 b are limited by the size of the entire display area of the electronic device 100 .
[0161] It should be noted that the present application does not limit the specific positions of the display area 1a and the display area 1b on the display screen of the electronic device 100.
[0162] In some embodiments of the present application, the display area 1a may be displayed above the display area 1b. For example, the display area 1a may be the upper half of the display area of the display screen of the electronic device 100, and the display area 1b may be the lower half of the display area of the display screen of the electronic device 100. Similarly, in still other embodiments of the present application, the display area 1b may be displayed above the display area 1a. For example, the display area 1b may be the upper half of the display area of the display screen of the electronic device 100, and the display area 1a may be the lower half of the display area of the display screen of the electronic device 100.
[0163] In some embodiments of the present application, the display area 1a may be displayed to the left of the display area 1b. For example, the display area 1a may be the left half of the display area of the display screen of the electronic device 100, and the display area 1b may be the right half of the display area of the display screen of the electronic device 100. Similarly, in still other embodiments of the present application, the display area 1b may be displayed to the left of the display area 1a. For example, the display area 1b may be the left half of the display area of the display screen of the electronic device 100, and the display area 1a may be the right half of the display area of the display screen of the electronic device 100.
[0164] In some embodiments of the present application, the specific positions of the display area 1a and the display area 1b are related to the current state of the electronic device 100. It is understandable that the current state of the electronic device 100 refers to whether the electronic device 100 is in a horizontal screen state or a vertical screen state. In one possible implementation, when the current state of the electronic device 100 is a vertical screen state, the default distribution positions of the display area 1a and the display area 1b are the upper half and the lower half of the display area of the electronic device 100. When the current state of the electronic device 100 is a horizontal screen state, the default distribution positions of the display area 1a and the display area 1b are the left half and the right half of the display area of the electronic device 100.
[0165] For example, based on what is shown in FIG2B , electronic device 100 can establish a wired connection with electronic device 200. When the current state of electronic device 100 is portrait mode, as shown in FIG2I(a), after electronic device 100 and electronic device 200 establish a wired connection, electronic device 200 can display user interface 202, while electronic device 100 can display user interface 108 and play music (e.g., song 1). User interface 108 can include display area 1a and display area 1b. Display area 1a is the upper half of electronic device 100, and display area 1b is the lower half of electronic device 100. The display content of display area 1a is the same as the display content of user interface 202. As shown in FIG2I(a), display area 1a in user interface 108 can display image 2. Image 2 is the same as the display content of user interface 202, and the size of image 2 is smaller than the size of user interface 202. Image 2 can be understood as an image obtained by scaling down user interface 202. The content displayed in the display area 1 b in the user interface 108 is the same as that displayed in the user interface 101 , but the size of the display area 1 b is different from that of the user interface 101 .
[0166] For example, based on the example shown in FIG2B , electronic device 100 can establish a wired connection with electronic device 200. When electronic device 100 is currently in landscape mode, as shown in FIG2I(b), after electronic device 100 and electronic device 200 establish a wired connection, electronic device 200 can display user interface 202, while electronic device 100 can display user interface 109 and play music (e.g., song 1). User interface 109 can include display area 1a and display area 1b. Display area 1a is the left half of electronic device 100, and display area 1b is the right half of electronic device 100. The display content of display area 1a in user interface 109 is the same as the display content of user interface 202. Specifically, display area 1a in user interface 109 can display image 3, which is the same as the display content of user interface 202. As shown in FIG2I(b), user interface 202 can include an exit control 2022. Similarly, display area 1a in user interface 109 can include an exit control 1091. The content displayed in the display area 1 b in the user interface 109 is the same as that displayed in the user interface 101 , but the size of the display area 1 b is different from that of the user interface 101 .
[0167] It is understood that the size of image 3 is smaller than the size of user interface 202. Image 3 can be understood as an image obtained by proportionally reducing user interface 202. It is understood that images 1, 2, and 3 have the same content but different sizes.
[0168] In some embodiments of the present application, the electronic device can adjust the size of the display area 1a and the display area 1b. The display area 1a can be at most the entire display area of the electronic device 100. In other words, the size of the display area 1a can be at most the same as the size of the entire display area of the electronic device 100. In this case, the electronic device 100 no longer displays the display area 1b. Similarly, the display area 1b can be at most the entire display area of the electronic device 100. In other words, the size of the display area 1b can be at most the same as the size of the entire display area of the electronic device 100. In this case, the electronic device 100 no longer displays the display area 1a.
[0169] In some embodiments of the present application, the electronic device 100 can detect a user operation (e.g., a slide operation, a slide and pause operation, etc.) acting on the boundary between the display area 1a and the display area 1b. In response to the user operation, the electronic device 100 can adjust the size of the display area 1a and the display area 1b. It is understood that the user operation mentioned in the present application may include but is not limited to touch (e.g., click, etc.), voice control, gestures, and other operations, and the present application does not limit this.
[0170] For example, the electronic device 100 may detect a rightward sliding operation at the boundary between the display area 1a and the display area 1b as shown in FIG2I(b). In response to the rightward sliding operation, the electronic device 100 may display the user interface 102 as shown in FIG2C. It is understood that the user interface 102 is the display content of the display area 1a. In this case, the electronic device 100 does not display the display area 1b.
[0171] In some embodiments of the present application, when electronic device 100 displays the display content corresponding to its own system, the user can control electronic device 200 by operating on electronic device 100. Once the display content of electronic device 200 changes, the display content of electronic device 100 also changes accordingly. Of course, the user can also control electronic device 200 by directly operating on electronic device 200.
[0172] For example, electronic device 100 may detect a user operation on exit control 1091, as shown in FIG2I(b). In response to this user operation, electronic device 100 may transmit relevant operation information to electronic device 200. After receiving the relevant operation information, electronic device 200 may determine, based on the relevant operation information, that the specific control triggered by the user is exit control 2022 on electronic device 200. In response to the relevant operation information, electronic device 200 may close the window of the current music application and display user interface 201, as shown in FIG2J. Accordingly, electronic device 100 may display user interface 110, as shown in FIG2J. User interface 110 may include display area 1a and display area 1b. The display content of display area 1a in user interface 110 is the same as that of user interface 201. Specifically, display area 1a in user interface 110 may display image 4. Image 4 is the same as the display content of user interface 202. As shown in FIG2J, user interface 201 may include video application icon 2011. Similarly, user interface 110 may include video application icon 1101. The content displayed in the display area 1 b in the user interface 110 is the same as that displayed in the user interface 101 , but the size of the display area 1 b is different from that of the user interface 101 .
[0173] It is understood that the size of image 4 is smaller than the size of user interface 201. Image 4 can be understood as a proportionally reduced image of user interface 202. It is understood that image 4 and image 3 have different contents but the same size.
[0174] For example, electronic device 100 may detect a user operation on video application icon 1101, as shown in FIG2J . In response to this user operation, electronic device 100 may transmit relevant operation information to electronic device 200. After receiving the relevant operation information, electronic device 200 may determine, based on the relevant operation information, that the specific control triggered by the user is video application icon 2011 on electronic device 200. In response to the relevant operation information, electronic device 200 may launch the video application, play the video, and display user interface 207, as shown in FIG2K . Accordingly, electronic device 100 may play the video, play the audio corresponding to the video, and display user interface 111, as shown in FIG2K . It will be understood that in this case, electronic device 200 does not play the audio corresponding to the video itself, but rather plays the audio corresponding to the video through electronic device 100. In other words, electronic device 200 only displays the video image without playing the audio corresponding to the video, while electronic device 100 displays both the video image and the audio corresponding to the video. User interface 111 may include display area 1a and display area 1b. The display content of display area 1a in user interface 111 is the same as that of user interface 207. Specifically, display area 1a in user interface 111 may display image 5. Image 5 is the same as the content displayed in user interface 207. Display area 1b in user interface 111 displays the same content as that displayed in user interface 101, but the size of display area 1b is different from that of user interface 101. Display area 1b in user interface 111 may include a memo application icon 1011.
[0175] It is understood that the size of image 5 is smaller than the size of user interface 207. Image 5 can be understood as an image obtained by proportionally reducing user interface 207. It is understood that image 5 and image 4 have different contents but the same size.
[0176] In some embodiments of the present application, when the electronic device 100 displays content corresponding to its own system, the user can control the electronic device 100 by performing operations on the electronic device 100. It is understood that the "controlling the electronic device 100" mentioned here specifically refers to: controlling the display content corresponding to the electronic device 100's own system to change. It is understood that changes in the display content corresponding to the electronic device 100's own system do not affect the display content of the electronic device 200.
[0177] For example, electronic device 100 may detect a user operation on memo application icon 1011 as shown in FIG2K . In response to the user operation, electronic device 100 may display user interface 112 as shown in FIG2L . At this time, electronic device 200 still displays user interface 207 . User interface 112 may include display area 1a and display area 1b . The display content of display area 1a is the same as the display content of user interface 207 . It is understood that display area 1b in user interface 112 displays the main interface of the memo application.
[0178] In some embodiments of the present application, when the electronic device 100 displays both the display content on the electronic device 200 and the display content corresponding to the electronic device 100's own system, the electronic device 100 can enter the annotation mode. In the annotation mode, the user can annotate the content displayed by the electronic device 100 (for example, the content displayed in the display area 1a and / or the display area 1b). The electronic device 100 can also save pictures containing annotation traces. It is understandable that the present application does not display the triggering method for the electronic device 100 to enter the annotation mode. For example, the user can trigger the electronic device 100 to enter the annotation mode through gestures, voice, touch, etc.
[0179] In one possible implementation, user annotations on the display content of electronic device 100 do not affect the display content of electronic device 200. In another possible implementation, user annotations on the display content displayed on electronic device 100 that is identical to the display content of electronic device 200 do affect the display content of electronic device 200. That is, annotation traces appear at the corresponding location on electronic device 200.
[0180] It is understood that the electronic device 100 saves the picture with the annotation traces, which may specifically include: the electronic device 100 can save the picture with the annotation traces to the album application and / or other applications (for example, the memo application, etc.). It is understood that the user can trigger the electronic device 100 to enter the corresponding interface of the album application and / or other applications to view the picture with the annotation traces.
[0181] It can be understood that the picture containing annotation traces saved by the electronic device 100 can be a screenshot of the display area 1a of the electronic device 100, which can be specifically a screenshot of the area in the display area 1a that is the same as the display content of the electronic device 200, or a screenshot of the entire display area of the electronic device 100, or a screenshot of a part of the display area of the electronic device 100 selected by the user.
[0182] It is understood that the annotation toolbar (for example, annotation toolbar 1131) displayed after the electronic device 100 enters the annotation mode can be displayed by default at the top center position of the entire display area of the display screen, and can also be displayed by default at the top center position of the display area 1a (as shown in Figure 2M), and this application does not limit this. It is understood that the user can drag the annotation toolbar to move it to any display area of the electronic device 100. It is understood that the annotation toolbar can also be minimized and displayed on the electronic device 100. For example, the annotation toolbar can be minimized and displayed as a floating ball.
[0183] Exemplarily, the electronic device 100 may detect a two-finger swiping operation on the user interface 112 as shown in FIG2L , and in response to the two-finger swiping operation, the electronic device 100 may enter the annotation mode. As shown in FIG2M , the electronic device 100 may display the user interface 113. The user interface 113 may include an annotation toolbar 1131. The annotation toolbar 1131 may include a graffiti control 11311, an erase control, a screenshot control 11312, a previous control, a next control, a completion control, and an exit control. When the graffiti control 11311 is turned on (as shown in FIG2M ), the user can scribble annotations on the electronic device 100. The erase control is used to erase annotation traces. The previous control and the next control are used to jump to the previous step and the next step, respectively. The completion control is used to save annotation traces. The exit control is used to exit the annotation mode.
[0184] For example, electronic device 100 may detect a user operation on display area 1a as shown in FIG2M . In response to the user operation, electronic device 100 may display user interface 114 as shown in FIG2N . It is understood that display area 1a in user interface 114 displays annotation traces. At this point, electronic device 200 may display user interface 207 as shown in FIG2N .
[0185] In some embodiments of the present application, after the annotation is completed, the electronic device 100 can exit the annotation mode and save the picture containing the annotation traces.
[0186] For example, the electronic device 100 may detect a user operation on the completion control in the toolbar 1131 shown in FIG2M . In response to the user operation, the electronic device 100 may exit the annotation mode and save the image containing the annotation traces to the electronic device 100. It is understood that regardless of whether the electronic device 200 displays the corresponding annotation traces before the electronic device 100 exits the annotation mode, the electronic device 200 no longer displays the annotation traces after the electronic device 100 exits the annotation mode.
[0187] In some embodiments of the present application, in the annotation mode, the electronic device 100 can obtain a picture containing annotation traces and save the picture containing the annotation traces.
[0188] According to the above, in a possible implementation, in the annotation mode, the electronic device 100 can obtain a picture containing annotation traces, and save the picture containing annotation traces to an album application of the electronic device 100 .
[0189] For example, electronic device 100 may detect a user action on screenshot control 11312 in user interface 114 shown in FIG2N . In response to this action, electronic device 100 may retrieve screenshot 1, save it to the photo album application, and display user interface 115 shown in FIG2O . User interface 115 may include a thumbnail of screenshot 1. It is understood that screenshot 1 may be a screenshot of the area where image 5 is located, included in display area 1a of user interface 115. Electronic device 100 may detect a user action on the thumbnail of screenshot 1 shown in FIG2O . In response to this action, electronic device 100 may redirect to the photo album application to display screenshot 1. As shown in FIG2P , electronic device 100 may display user interface 116. At this point, electronic device 200 still displays user interface 207. Display area 1a of user interface 116 displays the same content as user interface 207. Display area 1b of user interface 112 displays the image viewing interface of the photo album application. This image viewing interface includes screenshot 1. The user can share, collect, edit, delete, and perform other operations on the screenshot 1.
[0190] It can be understood that when the picture containing annotation traces that the electronic device 100 needs to obtain is a screenshot of the display area 1a, if the annotation toolbar 1131 is located in the display area 1a, then when the electronic device 100 obtains the picture containing annotation traces, it can automatically hide the annotation toolbar 1131 and then take a screenshot of the display area 1a, and then restore the display of the annotation toolbar 1131 after the screenshot is completed.
[0191] Of course, the user can not only trigger the electronic device 100 to display the picture containing annotation traces (for example, the above screenshot 1) by taking a screenshot thumbnail, but can also directly trigger the electronic device 100 to start the album application to display the picture containing annotation traces.
[0192] According to the above, in one possible implementation, in annotation mode, the electronic device 100 can obtain a picture containing annotation traces and save the picture containing annotation traces to other applications of the electronic device 100 (for example, a memo application) other than the album application.
[0193] For example, the electronic device 100 may detect a user operation on the screenshot 1 thumbnail as shown in FIG. 2O . In response to the user operation, the electronic device 100 may jump to the memo application to display the screenshot 1 .
[0194] Exemplarily, as shown in FIG2O , the electronic device 100 displays a corresponding interface of the memo application in the display area 1b. In this case, once the electronic device 100 can detect a user operation acting on the thumbnail of screenshot 1 as shown in FIG2O , in response to the user operation, the electronic device 100 can save the picture containing the annotation traces and directly display it in the corresponding interface of the memo application included in the display area 1b (the display area 1b in the user interface 117 as shown in FIG2Q ). In this way, the picture containing the annotation traces can be saved in the memo application. As shown in FIG2Q , the content displayed in the display area 1a in the user interface 117 is the same as the content displayed in the user interface 207 as shown in FIG2Q . The display area 1b in the user interface 117 displays a new note interface in the memo application. The new note interface includes the above-mentioned screenshot 1. The user can continue to input in the display area 1b in the user interface 117.
[0195] In some embodiments of the present application, the user can select the screenshot area based on the screenshot control 11312.
[0196] In a possible implementation, the user may select a screenshot area in the entire display area of the electronic device 100 .
[0197] In yet another possible implementation, the user may select a screenshot area in the display area 1 a of the electronic device 100 .
[0198] Exemplarily, the electronic device 100 may detect a user operation acting on the screenshot control 11312 as shown in FIG2N , and in response to the user operation, the electronic device 100 may display an interface to be screenshotted. The user may select any area in the display area 1a of the interface to be screenshotted for screenshotting. The electronic device 100 may detect a user operation acting on the interface to be screenshotted, and in response to the user operation, the electronic device 100 may display a user interface 118 as shown in FIG2R . The user interface 118 may include a screenshot area 1181 and a screenshot toolbar 1182. The screenshot area 1181 is a screenshot area selected by the user in the display area 1a. For ease of description, the screenshot corresponding to the screenshot area 1181 is referred to as screenshot 2 in this application. The screenshot toolbar 1182 may include a completion control and an exit control. The completion control may be used to save the image corresponding to the screenshot area 1181 (i.e., screenshot 2). The exit control is used to exit the screenshot interface.
[0199] Similar to the above, the electronic device 100 can not only save the screenshot 2 to the album application, but also save the screenshot 2 to other applications such as the memo application and the note application.
[0200] It is understood that based on the above default display mode, the electronic device 100 may change the display mode in the subsequent process. For example, the electronic device 100 may subsequently only display the display content on the electronic device 200, and not display the display content corresponding to the electronic device 100's own system. For another example, the electronic device 100 may subsequently not display the display content on the electronic device 200, and only display the display content corresponding to the electronic device 100's own system.
[0201] 3. The electronic device 100 may not display the display content on the electronic device 200, but only display the display content corresponding to the system of the electronic device 100 itself.
[0202] In some embodiments of the present application, the default display mode after the electronic device 100 and the electronic device 200 establish a wired connection may specifically be: the electronic device 100 may not display the display content on the electronic device 200, but only display the display content corresponding to the electronic device 100's own system.
[0203] For example, based on the example shown in FIG2B , electronic device 100 can establish a wired connection with electronic device 200. As shown in FIG2S , after electronic devices 100 and 200 establish a wired connection, electronic device 200 can continue to display user interface 202 but pause music playback, while electronic device 100 can continue to display user interface 101 but start playing music. In this case, electronic device 200 plays music through electronic device 100, but does not display the same display content as that of electronic device 200 on electronic device 100.
[0204] It is understood that based on the above default display mode, the electronic device 100 may change the display mode in the subsequent process. For example, the electronic device 100 may subsequently display both the display content on the electronic device 200 and the display content corresponding to the electronic device 100's own system. For another example, the electronic device 100 may subsequently display only the display content on the electronic device 200, without displaying the display content corresponding to the electronic device 100's own system.
[0205] The following describes how the electronic device 100 and the electronic device 200 play music when the electronic device 100 and the electronic device 200 establish a wired connection.
[0206] In some embodiments of the present application, when the electronic device 100 and the electronic device 200 establish a wired connection (as shown in FIG1B , the interface 1 of the electronic device 100 and the interface 2 of the electronic device 200 are connected via a connection cable), the electronic device 100 can play the audio transmitted by the electronic device 200 to the electronic device 100. That is, the electronic device 100 can play the audio data transmitted by the electronic device 200 to the electronic device 100 via the wired interface 2 and the interface 1. In this case, the electronic device 200 does not need to play the audio itself.
[0207] In some embodiments of the present application, when the electronic device 100 and the electronic device 200 establish a wired connection (as shown in FIG1B , the interface 1 of the electronic device 100 and the interface 2 of the electronic device 200 are connected via a connecting cable), the electronic device 200 can play the audio by itself. In one possible implementation, the electronic device 200 does not need to transmit the audio data to the electronic device 100, but the electronic device 200 can play the audio by itself. In another possible implementation, the electronic device 200 transmits the audio data to the electronic device 100, but the electronic device 100 does not need to play the audio data. In this case, the electronic device 200 plays the audio by itself. It is understood that the “electronic device 100 may not play the audio data” mentioned here may specifically include: the user chooses not to use the electronic device 100 to play the audio data of the electronic device 200. The “electronic device 100 may not play the audio data” mentioned here may also specifically include: because there is audio that needs to be played in the electronic device 100's own system, the electronic device 100 may play the audio that needs to be played in its own system instead of playing the audio data transmitted by the electronic device 200.
[0208] In some embodiments of the present application, there is audio that needs to be played in both electronic device 100 and electronic device 200. In this case, if the user selects electronic device 100 to play the audio in electronic device 200, electronic device 100 may not play the audio in its own system, but play the audio transmitted by electronic device 200.
[0209] In summary, when the electronic device 100 and the electronic device 200 establish a wired connection, the two playback modes of the electronic device 100 and the electronic device 200 are as follows:
[0210] 1. The electronic device 100 can play the audio of the electronic device 200, and the electronic device 200 does not need to play the audio itself.
[0211] 2. The electronic device 100 may not play the audio of the electronic device 200, while the electronic device 200 plays the audio on its own.
[0212] It is understood that the playback modes shown in Figures 2C, 2E-2H, 2I(a)-2I(b), and 2K-2S are all the above-mentioned playback mode 1. The specific implementation of the second playback mode can refer to the introduction of the first playback mode in this application.
[0213] In some embodiments of the present application, users can select the display mode and playback mode after electronic devices 100 and 200 establish a wired connection. For example, after electronic devices 100 and 200 establish a wired connection, electronic device 100 may display a corresponding prompt box. This prompt box may include several display options. The user can select the display mode and playback mode.
[0214] For example, based on the configuration shown in FIG2B , electronic device 100 can establish a wired connection with electronic device 200. As shown in FIG2T , after electronic device 100 and electronic device 200 establish a wired connection, electronic device 200 can continue to display user interface 202, and electronic device 100 can display user interface 119. User interface 119 may include a prompt box 1191. Prompt box 1191 may include a display mode option area 11911, a play mode option area 11912, default options 11913, and a confirmation control 11914. Display mode option area 11911 may include several display mode options. Specifically, display mode option area 11911 may include: "Only display the projected screen," "Only display your own screen," and "Show both the projected screen and your own screen." "Only display the projected screen" refers to the first display mode described above. That is, electronic device 100 can only display the display content on electronic device 200, without displaying the display content corresponding to the electronic device 100's own system. "Only display your own screen" refers to the third display mode described above. That is, the electronic device 100 may not display the display content on the electronic device 200, but only display the display content corresponding to the electronic device 100's own system. "Displaying the projection screen and the own screen at the same time" refers to the second display mode mentioned above. That is, the electronic device 100 can display both the display content on the electronic device 200 and the display content corresponding to the electronic device 100's own system. The playback mode option area 11912 may include several playback mode options. Specifically, the playback mode option area 11912 may include: "Play received audio" and "Do not play received audio." "Play received audio" means: the electronic device 100 plays the audio transmitted by the electronic device 200, and the electronic device 200 does not need to play the audio by itself. In other words, the electronic device 200 can use the electronic device 100 as an audio player. "Do not play received audio" means: the electronic device 100 does not play the audio transmitted by the electronic device 200, and the electronic device 200 plays the audio by itself. The default option 11913 refers to the default display mode and the default playback mode. When the user selects the default option 11913, the electronic device 100 can display the video based on the preset default display mode and play the video according to the preset default play mode. The confirmation control 11914 is used to confirm the selection of the currently set display mode and play mode.
[0215] For example, the electronic device 100 may detect a user operation on "Only display the projection screen" in the display mode option area 11911 as shown in FIG2T , and "Play received audio" in the playback mode option area 11912. In response to the user operation, the electronic device 100 may display the user interface 120 as shown in FIG2U . At this time, the electronic device 200 may still display the user interface 202. The electronic device 100 may detect a user operation on the confirmation control 11914 as shown in FIG2U . In response to the user operation, the electronic device 100 may display the user interface 102 as shown in FIG2C .
[0216] The hardware structure of the electronic device involved in this application is introduced below.
[0217] FIG3 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of the present application.
[0218] As shown in Figure 3, the electronic device 100 may include a system on chip (SOC), a camera serial interface (CSI), a display serial interface (DSI), an audio interface 1, an audio interface 2, a Type-C interface, a bridge chip, a display screen, a touch sensor, an internal memory, an external memory interface, an audio module, a speaker, a receiver, a microphone and a headphone interface.
[0219] It is understandable that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. It is understandable that the illustrated components can be implemented in hardware, software, or a combination of software and hardware. In some embodiments of the present application, the electronic device 100 may include more components than shown in the diagram. Exemplarily, the electronic device 100 may include a wireless communication module, a mobile communication module, etc. Exemplarily, the electronic device 100 may also include other types of sensors such as an acceleration sensor, a gyroscope sensor, and a temperature sensor. In some other embodiments of the present application, the electronic device 100 may include fewer components than shown in the diagram, or combine certain components, or split certain components, or arrange the components differently. The interface connection relationship between the modules illustrated in the embodiments of the present application is merely a schematic illustration and does not constitute a structural limitation on the electronic device 100.
[0220] A SoC is a system or product composed of multiple integrated circuits with specific functions combined on a single chip, encompassing a complete hardware system and the embedded software it carries. For example, an SoC may integrate a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), memory, a modem, a navigation and positioning module, and a multimedia module.
[0221] CSI and DSI belong to the same family and are both interface specifications developed by the Mobile Industry Processor Interface (MIPI) Alliance. That is, CSI and DSI are both MIPI interfaces. The MIPI interface can be used to connect the SOC with peripheral devices such as display screens and cameras. In some embodiments of the present application, the SOC and the camera (not shown in the figure) communicate via the CSI interface to realize the shooting function of the electronic device 100. In some embodiments of the present application, the SOC and the display screen communicate via the DSI interface to realize the display function of the electronic device 100.
[0222] It is understandable that the electronic device 100 may further include other MIPI interfaces. For example, the electronic device 100 may further include a radio frequency interface DigRF and a microphone / speaker interface SLIMbus.
[0223] Audio interface 1 and audio interface 2 are audio interfaces in electronic device 100. In some embodiments of the present application, audio interface 1 and audio interface 2 may be an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Pulse Density Modulation (PDM) interface, etc. In one possible implementation, audio interface 1 and audio interface 2 are different types of audio interfaces. In another possible implementation, audio interface 1 and audio interface 2 are the same type of audio interfaces.
[0224] The I2S interface can be used for audio communication. In some embodiments, the SOC can include multiple I2S buses. The SOC can be coupled to the audio module via the I2S bus to enable communication between the SOC and the audio module. In some embodiments of the present application, the audio module can transmit audio signals to the wireless communication module (not shown) in the electronic device 100 via the I2S interface to enable the function of answering calls via a Bluetooth headset.
[0225] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments of the present application, the audio module can be coupled to a wireless communication module (not shown) in the electronic device 100 via a PCM bus interface. In some embodiments of the present application, the audio module can also transmit audio signals to the wireless communication module via the PCM interface, enabling the function of answering calls through a Bluetooth headset.
[0226] The PDM interface can also be used for audio communication to sample, quantize and encode analog signals. PDM uses a clock sampling modulated analog component at a much higher sampling rate than PCM, and has only one bit output, either 0 or 1. In some embodiments of the present application, the sound signal collected by the microphone can be transmitted to the SOC via the PDM interface for processing. In some embodiments of the present application, the audio module can also transmit audio signals to the wireless communication module through the PDM interface to realize the function of answering calls through Bluetooth headphones.
[0227] It is understood that the I2S interface, PCM interface, and PDM interface can all be used for audio communication. Detailed descriptions of the I2S interface, PCM interface, and PDM interface can be found in relevant technical documents and will not be elaborated here.
[0228] The Type-C interface is a type of USB interface. The Type-C interface in the electronic device 100 may include a Type-C interface that supports DP output (e.g., interface 1). The relevant description of interface 1 can be referred to above.
[0229] It is understandable that the electronic device 100 may further include other USB interfaces. For example, a Mini USB interface, a Micro USB interface, and a USB Type-C interface (referred to as a Type-C interface). In some embodiments of the present application, the electronic device 100 may further include a Type-C interface that supports DP output. For example, interface 3 shown in Figures 1A and 1B. It is understandable that interface 3 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices (for example, a keyboard, a mouse, etc.), and can also be used to connect headphones to play audio through the headphones.
[0230] It is understandable that the electronic device 100 may also include other interfaces. For example, a General-Purpose Input / Output (GPIO) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, etc. The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments of the present application, the GPIO interface can be used to connect the SOC with other modules (for example, an audio module, a display screen, etc.). The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc. The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the SOC to a wireless communication module (not shown in the figure). For example, the SOC communicates with the Bluetooth module in the wireless communication module via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module can transmit audio signals to the wireless communication module via the UART interface to implement the function of playing music through a Bluetooth headset.
[0231] It is understood that the interface connection relationship between the modules illustrated in the embodiment of the present invention is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.
[0232] The Bridge chip can be used to convert data formats. In this application, the Bridge chip can convert the audio and video data format input from other electronic devices (for example, the electronic device 200) to the electronic device 100 into an audio and video data format that can be recognized by the SOC of the electronic device 100.
[0233] In some embodiments of the present application, the Bridge chip in the electronic device 100 may convert image data in the DP data format input from the electronic device 200 to the electronic device 100 into image data in the CSI data format.
[0234] The display screen is used to display images, videos, etc. In some embodiments of the present application, the electronic device 100 may include one or more display screens. The electronic device 100 may implement the display function through a graphics processor, a display screen, and an application processor.
[0235] In the present application, the ability of the electronic device 100 to display the user interface shown in FIG. 2A to FIG. 2U depends on the display functions provided by the aforementioned GPU, display screen, and application processor.
[0236] A touch sensor is also called a "touch control device". The touch sensor is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operation to the SOC to determine the type of touch event, and can also provide visual output related to the touch operation through the display. In some embodiments of the present application, the touch sensor can be specifically a touch screen (Touch Pad, TP). In this case, the screen of the electronic device 100 can be formed by combining a touch sensor and a display.
[0237] In the present application, the electronic device 100 may utilize a touch sensor to detect operations such as clicks and slides performed by a user on the display screen to implement the display method shown in FIG. 2A to FIG. 2U .
[0238] Internal memory can include one or more random access memories (RAM) and one or more non-volatile memories (NVM). RAM can be directly read and written by the processor and can be used to store executable programs (e.g., machine instructions) for the operating system or other running programs, as well as user and application data. NVM can also store executable programs and user and application data and can be pre-loaded into RAM for direct reading and writing by the processor.
[0239] In the present application, the code for implementing the display method described in the embodiment of the present application can be stored in a non-volatile memory. When running the screen projection application, the electronic device 100 can load the executable code stored in the non-volatile memory into the random access memory.
[0240] The external memory interface can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100.
[0241] The electronic device 100 can implement audio functions through an audio module, a speaker, a receiver, a microphone, a headphone jack, and an application processor.
[0242] The audio module converts digital audio information into analog audio signals for output, and also converts analog audio input into digital audio signals. A speaker, also known as a "horn," converts electrical audio signals into sound signals. A receiver, also known as a "handset," converts electrical audio signals into sound signals. A microphone, also known as a "microphone" or "microphone," converts sound signals into electrical signals. The headphone jack is used to connect wired headphones.
[0243] In some embodiments of the present application, the interface 1 in the electronic device 100 may not be a Type-C interface, but another type of interface. For example, HDMI. In this case, in some embodiments of the present application, the Bridge chip in the electronic device 100 can convert the image data in the HDMI data format input from the electronic device 200 to the electronic device 100 into image data in the CSI data format. It is understood that the electronic device 100 may also include a Type-C interface (for example, interface 3) that supports DP output.
[0244] FIG4 is a schematic diagram of the hardware structure of an electronic device 200 provided in an embodiment of the present application.
[0245] As shown in FIG4 , the electronic device 200 may include a processor, an external memory interface, an internal memory, an audio module, a speaker, a receiver, a microphone, an earphone interface, a display, and a Type-C interface.
[0246] The structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 200. In some embodiments of the present application, the electronic device 200 may include more components than shown in the figure. For example, the electronic device 200 may include a wireless communication module, a mobile communication module, etc., and may also include other types of sensors such as an accelerometer. In some embodiments of the present application, the electronic device 200 may include fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. The interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 200.
[0247] The processor may include one or more processing units. For example, the processor may include an application processor (AP), a modem processor, a graphics processor, an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The processor may also include a memory for storing instructions and data.
[0248] Electronic device 200 can implement display functions through a GPU, a display screen, and an application processor. The GPU is a microprocessor for image processing that connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs, which execute program instructions to generate or change display information. The display screen is used to display images, videos, etc. In some embodiments of the present application, electronic device 200 may include one or more display screens.
[0249] In the present application, the ability of the electronic device 200 to display the user interface shown in FIG. 2A to FIG. 2U depends on the display functions provided by the aforementioned GPU, display screen, and application processor.
[0250] The relevant descriptions of the internal memory and external memory interfaces can be found above and will not be repeated here.
[0251] The electronic device 200 can implement audio functions through an audio module, a speaker, a receiver, a microphone, an earphone interface, and an application processor, etc. The relevant descriptions of the audio module, the speaker, the receiver, the microphone, and the earphone interface can be referred to above, and this application will not repeat them here.
[0252] The Type-C interface in the electronic device 200 may include an interface 2. For the related description of the Type-C interface and the interface 2, reference may be made to the above.
[0253] In some embodiments of the present application, the interface 2 in the electronic device 200 may be another type of data transmission interface instead of a Type-C interface.
[0254] According to the above, when interface 1 and interface 2 are matching data transmission interfaces, interface 1 in electronic device 100 can be directly connected to interface 2 of electronic device 200 via a connecting cable. Interface 1 and interface 2 are both matching data transmission interfaces, which can specifically refer to: interface 1 is an interface that supports DP signal input (for example, a Type-C interface that supports DP signal input as shown in Figure 3), and interface 2 is an interface that supports DP signal output (for example, a Type-C interface that supports DP signal output as shown in Figure 4).
[0255] According to the above, in the case where interface 1 and interface 2 are unmatched data transmission interfaces, interface 1 in electronic device 100 can be connected to interface 2 on electronic device 200 via a switching device.
[0256] Exemplarily, interface 1 and interface 2 are unmatched data transmission interfaces, specifically: interface 1 is an interface supporting DP signal input (for example, the Type-C interface supporting DP signal input as shown in FIG3 ), and interface 2 is an interface supporting HDMI signal output.
[0257] Exemplarily, interface 1 and interface 2 are incompatible interfaces, which may specifically mean that interface 1 is an interface supporting HDMI signal input, and interface 2 is an interface supporting DP signal output (for example, a Type-C interface supporting DP signal output as shown in FIG4 ).
[0258] The following describes the specific implementation of the aforementioned embodiment from a hardware perspective based on the hardware structure of the electronic device shown in FIG3 and FIG4 .
[0259] As shown in FIG1B , the interface 1 of the electronic device 100 can be connected to the interface 2 of the electronic device 200, so that the electronic devices 100 and 200 establish a wired communication connection. In this case, as shown in FIG5A , the electronic device 200 transmits the DP data packet to the interface 1 of the electronic device 100 via the interface 2. After the electronic device 100 receives the DP data packet transmitted by the electronic device 200 via the interface 1, it can transmit the DP data packet to the Bridge chip. The Bridge chip can separate the image data and audio data in the DP data packet.
[0260] For the image data in the DP data packet, the Bridge chip in the electronic device 100 can convert the image data from the DP data format to the CSI data format and transmit it to the SOC via the CSI. The SOC in the electronic device 100 can process the image data after the format conversion (i.e., the input image data) and the image data generated by the electronic device 100's own system (for example, layer overlay) to obtain the processed image data, and send the processed image data to the display screen via the DSI for display. It can be understood that the processed image data is image data in the DSI format.
[0261] It can be understood that the above-mentioned input image data refers to the image data input from the electronic device 200 to the interface 1 of the electronic device 100 .
[0262] In some embodiments of the present application, the processed image data may specifically include: input image data. In this case, the electronic device 100 may only display the display content of the electronic device 200, without displaying the display content corresponding to its own system (as shown in Figures 2C-2H).
[0263] In some embodiments of the present application, the processed image data may specifically include: image data obtained after the SOC in the electronic device 100 performs layer overlay processing on the input image data and the image data generated by the electronic device 100's own system. It is worth noting that before the above-mentioned layer overlay, the electronic device 100 only displays the image data generated by its own system. However, after the above-mentioned layer overlay processing, the image data finally displayed on the display screen of the electronic device 100 includes not only the image data of its own system, but also the image data transmitted by the electronic device 200. In other words, the electronic device 100 can display both the display content of the electronic device 200 and the display content corresponding to its own system (as shown in Figures 2I(a), 2I(b)-2R).
[0264] In some embodiments of the present application, the processed image data may specifically include image data generated by the electronic device's own system. In this case, the electronic device 100 may only display the display content corresponding to its own system, and not display the display content of the electronic device 200.
[0265] It should be noted that CSI is an exemplary camera interface provided in the embodiments of the present application, and DSI is an exemplary display interface provided in the embodiments of the present application. In other embodiments of the present application, other types of camera interfaces and display interfaces may be used in the electronic device 100, and the present application does not limit this.
[0266] For the audio data in the DP data packet, the Bridge chip in the electronic device 100 can transmit the audio data to the SOC via the audio interface 1. The SOC in the electronic device 100 can process the audio data and transmit the processed audio data to the speaker via the audio interface 2 for playback.
[0267] It is understandable that the present application does not limit the specific types of the audio interface 1 and the audio interface 2. For the relevant description of the audio interface 1 and the audio interface 2, please refer to the above and will not be elaborated here.
[0268] For example, when the audio interface is an I2S interface, the Bridge chip in the electronic device 100 can transmit the audio data in the DP data packet to the SOC via the I2S interface, and the SOC can also transmit the processed audio data to the speaker via the I2S interface.
[0269] In some embodiments of the present application, the DP data packet does not include actual audio data. It can also be understood that the audio data in the DP data packet is empty. In this case, the electronic device 100 still needs to execute the audio data processing flow. Specifically, the Bridge chip in the electronic device 100 can transmit the empty format corresponding to the audio data to the SOC via the audio interface 1, and the SOC can process the empty format and then transmit the empty format to the speaker via the audio interface 2. It can be understood that in this case, since there is no actual audio data, the speaker has no audio to play.
[0270] In some embodiments of the present application, when the electronic device 100 and the electronic device 200 establish a wired communication connection, the electronic device 100 can enter the annotation mode. At this time, the user can annotate the content displayed by the electronic device 100 (as shown in Figures 2L-2M). Specifically, as shown in Figure 5A, on the one hand, the electronic device 100 can receive the DP data packet transmitted by the electronic device 200 through the interface 1, and then transmit the image data in the DP data packet to the Bridge chip. After the Bridge chip converts the image data from the DP data format to the CSI data format, the format-converted image data is transmitted to the SOC via the CSI. It can be understood that the SOC in the electronic device 100 can process the format-converted image data (i.e., the input image data) and the image data generated by its own system (for example, layer overlay) to obtain the processed image data. On the other hand, the touch sensor in the electronic device 100 can detect the user's operation on the electronic device 100 through a finger, a stylus, etc., obtain the touch screen (Touch Pad, TP) point information, and transmit the TP point information to the SOC. It is understood that the TP point information is the relevant information of the contact position (for example, the contact position coordinates, etc.). After receiving the TP point information, the SOC can draw a layer based on the TP point information and superimpose the layer with the processed image data to obtain superimposed image data. It is understood that the superimposed image data is image data in DSI format. The SOC can transmit the superimposed image data to the display screen via DSI for display. At this time, the annotation traces can be displayed on the display screen (as shown in Figures 2N-2R, etc.).
[0271] It should be noted that before the user uses the stylus pen to annotate on the electronic device 100 , the stylus pen and the electronic device 100 need to establish a communication connection (eg, a Bluetooth connection).
[0272] In some embodiments of the present application, the pressure sensor in the stylus can send pressure-related information (i.e., pressure information) when the stylus falls to the electronic device 100 via Bluetooth or other means, and the electronic device 100 can adjust the thickness of the strokes corresponding to the stylus based on the received pressure-related information.
[0273] Based on the display method shown in FIG5A , the electronic device 100 may further transmit TP point information to the electronic device 200. After receiving the TP point information, the electronic device 200 may perform corresponding operations based on the TP point information (e.g., writing, drawing, operating a control at a target position coordinate in the electronic device 200, etc.). In other words, the user may control the electronic device 200 through the electronic device 100.
[0274] Specifically, when the electronic device 100 and the electronic device 200 establish a wired communication connection (as shown in FIG1B ), as shown in FIG5B , on the one hand, after the electronic device 100 receives the DP data packet transmitted by the electronic device 200, the DP data packet can be transmitted to the Bridge chip, and the image data and audio data in the DP data packet are separated by the Bridge chip, and the image data and audio data in the DP data packet are processed accordingly, and then the corresponding picture is displayed, and the corresponding audio is played (that is, the audio is played when the audio data in the DP data packet is not empty). For details, please refer to the above and will not be repeated here. On the other hand, the touch sensor in the electronic device 100 can detect the user's operation on the electronic device 100 through fingers, styluses, etc., obtain TP point information, and transmit the TP point information to the SOC.
[0275] In some embodiments of the present application, the SOC in the electronic device 100 can determine whether the above-mentioned operation acting on the electronic device 100 is aimed at the display content of the electronic device 200 or the display content corresponding to the electronic device 100's own system based on the TP reporting point information and the current display mode.
[0276] If the current display mode is that the electronic device 100 only displays the display content on the electronic device 200 , the electronic device 100 may determine that the above-mentioned operation acting on the electronic device 100 is for the display content of the electronic device 200 .
[0277] If the current display mode is that the electronic device 100 displays both the display content on the electronic device 200 and the display content corresponding to its own system, the electronic device 100 can determine whether the contact position coordinates are located in the display area 1a or the display area 1b based on the TP reporting information. In the case where the contact position coordinates are located in the display area 1a, the electronic device 100 can determine that the above-mentioned operation acting on the electronic device 100 is for the display content of the electronic device 200. In the case where the contact position coordinates are located in the display area 1b, the electronic device 100 can determine that the above-mentioned operation acting on the electronic device 100 is for the display content corresponding to the electronic device 100's own system.
[0278] If the current display mode is that the electronic device 100 only displays the display content corresponding to its own system, the electronic device 100 may determine that the above operation acting on the electronic device 100 is for the display content corresponding to the electronic device 100's own system.
[0279] When the electronic device 100 determines that the above-mentioned operation acting on the electronic device 100 is directed to the display content of the electronic device 200, as shown in Figure 5B, the SOC in the electronic device 100 can encapsulate the TP point information into a USB data packet and transmit the USB data packet to the interface 1. The electronic device 100 transmits the USB data packet to the interface 2 of the electronic device 200 via the interface 1. It is understood that the USB data packet may include the TP point information. After the electronic device 200 receives the USB data packet transmitted by the electronic device 100, it can determine the target position coordinates in the electronic device 200 based on the TP point information in the USB data packet, and perform corresponding operations based on the target position coordinates in the electronic device 200 to obtain updated image data, and transmit the updated image data to the display screen of the electronic device 200 for display. It is understood that the electronic device 200 determines the target position coordinates in the electronic device 200 based on the TP point information in the USB data packet, which can specifically include: the electronic device 200 maps the contact position coordinates on the electronic device 100 included in the TP point information in the USB data packet to the target position coordinates in the electronic device 200.
[0280] It can be understood that the corresponding operations performed by the electronic device 200 based on the target position coordinates in the electronic device 200 may specifically include writing, drawing, etc. (as shown in Figure 2H), and may also include operating the controls at the target position coordinates in the electronic device 200 (as shown in Figures 2E-2F, 2J-2K). This application does not impose specific restrictions on this.
[0281] In some embodiments of the present application, the SOC in the electronic device 100 sends the USB data packet to the electronic device 200 via a USB 2.0 interface.
[0282] In some embodiments of the present application, the electronic device 200 maps the contact position coordinates on the electronic device 100 to the target position coordinates in the electronic device 200. Specifically, the electronic device 200 may map the contact position coordinates to the target position coordinates in the screen operation area of the electronic device 200 based on the maximum coordinate value of the screen sensing area of the electronic device 100 and the maximum coordinate value of the screen operation area of the electronic device 200. It is understandable that the contact position coordinates in the electronic device 100 are the coordinates of the user acting on the screen sensing area of the electronic device 100.
[0283] In some embodiments of the present application, the maximum coordinate value of the screen sensing area of the electronic device 100 may be sent by the electronic device 100 to the electronic device 200 .
[0284] In some embodiments of the present application, before the electronic device 100 sends a USB data packet to the electronic device 200, the electronic device 100 may send the maximum coordinate value of the screen sensing area of the electronic device 100 (for example, the maximum horizontal coordinate value of the screen sensing area and the maximum vertical coordinate value of the screen sensing area) to the electronic device 200.
[0285] In some embodiments of the present application, the USB data packet sent by the electronic device 100 to the electronic device 200 may include the maximum coordinate value of the screen sensing area of the electronic device 100 .
[0286] It should be noted that, when the current display mode of the electronic device 100 is that the electronic device 100 only displays the display content of the electronic device 200, the screen sensing area of the electronic device 100 can be understood as the area on the display screen of the electronic device 100 where the display content of the electronic device 200 is displayed. That is, if the electronic device 100 displays the display content of the electronic device 200 in full screen, the screen sensing area of the electronic device 100 is the display area of the entire display screen of the electronic device 100. For example, as shown in FIG2C , the electronic device 100 can display the display content of the electronic device 200 in full screen. In this case, the screen sensing area of the electronic device 100 is the display area of the entire display screen of the electronic device 100. However, if the electronic device 100 does not display the display content of the electronic device 200 in full screen, the screen sensing area of the electronic device 100 is not the display area of the entire display screen of the electronic device 100, but is only the corresponding area on the electronic device 100 where the display content of the electronic device 200 is displayed.
[0287] It should be noted that when the current display mode of the electronic device 100 is such that the electronic device 100 displays both the display content of the electronic device 200 and the display content corresponding to its own system, the screen sensing area of the electronic device 100 can be understood as the area within the display area 1a where the display content of the electronic device 200 is displayed. In other words, if the display area 1a of the electronic device 100 is entirely used to display the display content of the electronic device 200, then the screen sensing area of the electronic device 100 is the entire display area 1a of the electronic device 100. For example, as shown in FIG2I(a), the content displayed in the display area 1a (i.e., image 2) is the display content of the electronic device 200, and the screen sensing area of the electronic device 100 is the display area 1a. However, if the display area 1a of the electronic device 100 is not entirely used to display the display content of the electronic device 200, then the screen sensing area of the electronic device 100 is not the entire display area 1a of the electronic device 100, but rather is only the corresponding area within the display area 1a where the display content of the electronic device 200 is displayed. For example, as shown in FIG2I(b), the display area 1a can display other content in addition to the display content of the electronic device 200 (i.e., image 3). That is, the display area 1a is not entirely used to display the display content of the electronic device 200. In this case, the screen sensing area of the electronic device 100 is the area in the display area 1a where the image 3 is displayed.
[0288] In some embodiments of the present application, the screen operation area of the electronic device 200 may be the entire display screen area of the electronic device 200 .
[0289] It can be understood that the maximum coordinate value of the screen sensing area of the electronic device 100 can reflect the coordinate range of the screen sensing area of the electronic device 100, and the maximum coordinate value of the screen operation area of the electronic device 200 can reflect the coordinate range of the screen operation area of the electronic device 200. Therefore, according to the maximum coordinate value of the screen sensing area of the electronic device 100 and the maximum coordinate value of the screen operation area of the electronic device 200, the mapping relationship between the position coordinates in the screen sensing area of the electronic device 100 and the position coordinates in the screen operation area of the electronic device 200 can be determined. According to the mapping relationship, the contact position coordinates on the electronic device 100 can be mapped to the target position coordinates in the screen operation area of the electronic device 200.
[0290] For example, as shown in FIG6A , the maximum coordinate value of the screen sensing area 601 of the electronic device 100 is (10, 6), and as shown in FIG6B , the maximum coordinate value of the screen operation area 602 of the electronic device 200 is (20, 12). Then, the mapping relationship between the position coordinates (Xa, Ya) in the screen sensing area 601 of the electronic device 100 and the position coordinates (Xb, Yb) in the screen operation area 602 of the electronic device 200 is: Xb=2Xa, Yb=2Ya. In this case, as shown in FIG6A and FIG6B , assuming that the contact position coordinates (2, 3) in the screen sensing area 601 of the electronic device 100 sent by the electronic device 100 are received by the electronic device 200, the electronic device 200 can map the contact position coordinates (2, 3) to the target position coordinates (4, 6) in the screen operation area 602 of the electronic device 200 according to the mapping relationship.
[0291] In some embodiments of the present application, based on the operation of the user acting on the electronic device 100 through a finger, a stylus, etc., the electronic device 100 can not only obtain TP point reporting information, but also obtain pressure information. The pressure information can be used to characterize the granularity of the operation of the user acting on the electronic device 100 through a finger, a stylus, etc. In this case, the SOC in the electronic device 100 can process the TP point reporting information and the pressure information to obtain the processed TP point reporting information. It can be understood that the processed TP point reporting information can not only include the contact position coordinates, but also the pressure information corresponding to the corresponding contact position coordinates. The SOC can encapsulate the processed TP point reporting information into a USB data packet and transmit the USB data packet to interface 1. The electronic device 100 transmits the USB data packet to interface 2 of the electronic device 200 via interface 1. The processing of the USB data packet by the electronic device 200 can refer to the above and will not be repeated here.
[0292] It is worth noting that the display method shown in Figures 5A and 5B is based on the premise that interface 1 and interface 2 match. When interface 1 and interface 2 do not match, it is necessary to add a switching device between interface 1 and interface 2 to implement the above display method. For details, please refer to the above and relevant technical documents, which will not be explained in detail here.
[0293] The software architecture of the electronic device 100 involved in this application is introduced below.
[0294] As shown in FIG. 7 , the software framework of the electronic device 100 may include an application layer, an application framework layer (framework, FWK), a system library, a runtime, a hardware abstraction layer (HAL), and a kernel layer (kernel).
[0295] Among them, the application layer may include a series of application packages. For example, screen projection applications, gallery, Bluetooth and WLAN applications (also referred to as applications). Among them, the screen projection application can be used to display image data input from other electronic devices (for example, electronic device 200) to the electronic device 100 on the display screen of the electronic device 100, and can also be used to play audio data input from other electronic devices (for example, electronic device 200) to the electronic device 100 through the speakers of the electronic device 100. In some embodiments of the present application, the screen projection application can be a system application rather than a third-party application.
[0296] It can be understood that the screen projection application can correspond to a corresponding life cycle management module, a window and control management module, a process animation module, a volume control module, a display control module, a foreground and background control module, an audio playback thread, an audio recording thread and an audio path management module. Among them, the life cycle management module is used to manage the life cycle of the screen projection application. The window and control management module is used to manage the display window and related controls of the screen projection application. The process animation module is used to manage the related animations of the screen projection application. The volume control module can be used to control the volume of the above-mentioned audio data during playback. The display control module can be used to control the display area of the above-mentioned image data on the display screen of the electronic device 100. The foreground and background control modules are used to control the operation of the screen projection application in the foreground / background. The audio path management module is used to create a recording path and a playback path. The audio playback thread is the thread corresponding to the playback path. The audio recording thread is the thread corresponding to the recording path. The relevant functions of the screen projection application can also refer to the relevant descriptions of Figures 8A-8B below.
[0297] The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.
[0298] The application framework layer can include a series of system services. System services are modular components that focus on specific functions. The functions provided by the application framework API can communicate with system services to access the underlying hardware.
[0299] The application framework layer may include a camera FWK, an audio service, an audio manager, a wired access manager, and a USB service module.
[0300] The Camera FWK provides an API for applications (for example, the camera app) to call, receives requests from them, maintains the business logic for request processing, and ultimately sends the request to the Camera Service for processing by calling the camera's AIDL cross-process API. It then waits for the Camera Service to return the result and sends the final result to the camera app. AIDL stands for Android Interface Definition Language.
[0301] The audio service primarily controls audio input and output. For example, you can adjust the volume, set the volume mode, and configure the Bluetooth audio mode. It can be understood that the audio service includes the PlaybackThread and AudioRecord. The PlaybackThread is the thread within the audio service that handles audio processing algorithms such as mixing and sound effects. The AudioRecord is the component within the audio service that is related to recording.
[0302] In some embodiments of the present application, the audio service may include an audio policy management module. In the case where there are audio data input by other electronic devices (for example, electronic device 200) and audio data generated by its own system in the electronic device 100, the audio policy management module can determine the playback order of the audio data based on relevant policies. For example, the audio policy management module can determine the audio data to be played first based on the priority of different external devices. Exemplarily, the electronic device 200 is connected to the interface that supports DP signal input in the electronic device 100, and the priority of the electronic device 200 is higher than the priority of the electronic device 100. The audio policy management module can determine that the audio data input from the electronic device 200 to the electronic device 100 is played first.
[0303] The audio manager has a Binder Native client for the audio service, acting as a proxy for the audio service on the client side. Generally speaking, client requests to the audio manager are ultimately handled by the audio service. The Binder Native client is the client side of the Binder mechanism in the C++ layer.
[0304] The wired access manager is used to report that other devices (eg, DP IN devices) are connected to the electronic device 100 via a wired connection. It is understood that a DP IN device refers to a device that establishes a wired connection with the electronic device 100 via an interface on the electronic device 100 that supports DP input.
[0305] The USB service module is the framework part of the USB software architecture. The USB service module can be used to process the TP point information uploaded by the TP driver and obtain the processed coordinate information.
[0306] The application framework layer may also include a window manager and a view system, etc. The specific meanings of these can be found in the relevant technical documentation and will not be elaborated here.
[0307] The system library can include multiple functional modules, such as the Surface Manager, Media Libraries, 3D graphics processing library (e.g., OpenGL ES), and 2D graphics engine (e.g., SGL). The specific meanings and functions of these functional modules can be found in the relevant technical documentation and will not be explained in detail here.
[0308] The runtime is responsible for system scheduling and management. The runtime includes the core library and the virtual machine.
[0309] The Hardware Abstraction Layer (HAL) is an interface layer located between the operating system kernel and upper-level software. Its purpose is to abstract the hardware. The Hardware Abstraction Layer is an abstract interface driven by the device kernel, which is used to implement the application programming interface that provides access to the underlying device to the higher-level Java API framework. The HAL can provide a standard interface to display the device hardware functions to the higher-level Java API framework. The HAL contains multiple library modules, such as the camera HAL, audio HAL, etc. Each library module implements an interface for a specific type of hardware component. When the system framework layer API requires access to the hardware of a portable device, the operating system will load the library module for the hardware component. It can be understood that the camera HAL can provide the camera FWK with an interface to access hardware components such as the camera. Similarly, the audio HAL can provide the audio service, audio manager, etc. with an interface to access hardware components such as speakers.
[0310] In some embodiments of the present application, the audio HAL may include an audio device management module and an audio data processing module. The audio device management module is used to manage audio devices connected to the electronic device 100. The audio data processing module is used to establish playback and recording paths, and transmit and process audio data.
[0311] It is understood that the HAL may also include a DP control module. The DP control module may be used to interact with the bridge chip (e.g., notifying the bridge chip to power on, receiving insertion events obtained by the bridge chip), and may also be used to obtain the resolution and frame rate of other electronic devices (e.g., electronic device 200) that have a wired connection with the electronic device 100.
[0312] It is understood that the HAL may also include a USB configuration module. The USB configuration module may be used to perform USB device controller (UDC) enumeration, i.e., establish a USB communication connection between electronic device 100 and electronic device 200, and may also be used to transmit the USB device type of electronic device 100 to electronic device 200.
[0313] The kernel layer is the foundation of the Android operating system. It is responsible for hardware drivers, networking, power, system security, and memory management. The kernel layer acts as an intermediate layer between hardware and software, relaying application requests to the hardware. The kernel layer includes drivers for bridge chipsets, Bluetooth, TP, USB, display, sensor, and audio. Audio drivers specifically include drivers for speakers, wired headphones, and Bluetooth headsets.
[0314] It should be noted that the software architecture diagram of the electronic device 100 shown in FIG7 provided in this application is merely an example and does not limit the specific module divisions within the different layers of the Android operating system. For details, please refer to the introduction of the Android operating system software architecture in conventional technology. In addition, the display method provided in this application can also be implemented based on other operating systems, and this application will not cite examples one by one.
[0315] It is understood that when the interface 1 in the electronic device 100 is HDMI, the electronic device 100 may include an HDMI control module. Similar to the DP control module, the HDMI control module can be used to interact with the bridge chip. Its specific functions can refer to the above description of the DP control module.
[0316] The following describes the specific implementation of the aforementioned embodiment from a software perspective based on the software architecture of the electronic device 100 shown in FIG. 7 .
[0317] 1. Image Data Transmission Architecture
[0318] When the electronic device 100 and the electronic device 200 establish a wired connection (as shown in Figure 1B), as shown in Figure 8A, the screen projection application in the electronic device 100 can first power on the electronic device 200 and obtain the resolution and frame rate of the electronic device 200. Specifically, the screen projection application can send a power-on notification message_1 to the DP control module. After the DP control module receives the power-on notification message_1, it can send a power-on notification message_2 to the bridge chip driver to notify the Bridge chip to power on. After the Bridge chip is powered on, the bridge chip driver can call the interface for obtaining resolution and frame rate to obtain resolution-related information and frame rate-related information of the electronic device 200, and send it to the DP control module. After the DP control module receives the resolution-related information and frame rate-related information of the electronic device 200 sent by the bridge chip driver, it can determine the resolution and frame rate of the electronic device 200 based on the relevant information, and send the resolution and frame rate of the electronic device 200 to the screen projection application. In some embodiments of the present application, the frame rate-related information is the bit rate. In this case, based on the bit rate, the DP control module can use a frame rate calculation algorithm to determine the frame rate.
[0319] As shown in Figure 8A, after the electronic device 100 powers on the electronic device 200, the screen projection application, camera FWK and camera HAL in the electronic device 100 can create a camera (camera) path. For example, the screen projection application can send a camera path creation instruction to the camera FWK. Similarly, after the camera FWK receives the camera path creation instruction sent by the screen projection application, it can also send a corresponding camera path creation instruction to the camera HAL. It can be understood that the camera path creation instruction sent by the screen projection application to the camera FWK may be different from the camera path creation instruction sent by the camera FWK to the camera HAL. For details, please refer to the relevant technical documents, which will not be explained in detail here.
[0320] It is worth noting that in the process of creating the camera path, the camera HAL is customized for the Bridge chip. For example, the camera HAL can customize the use case and / or pipeline for the Bridge chip. When requesting frame data, the electronic device 100 can control whether the electronic device 100 receives the DP signal by controlling the rest pin of the Bridge chip. As shown in Figure 8A, the camera HAL can send a rest pin control instruction to the bridge chip driver to control the bridge chip to receive the DP signal (for example, image data in DP data format) input by the electronic device 200.
[0321] In software engineering or systems engineering, a use case (or use case) is a description of how a system responds to external requests. It is a technique for capturing requirements through user scenarios. A pipeline represents a linear communication model of pipeline segments that exchange data between an add-in and its host.
[0322] Based on the above content, after the electronic device 100 receives the image data transmitted by the electronic device 200, the bridge chip driver in the electronic device 100 can convert the data format of the image data to obtain the image data after format conversion, and upload the image data after format conversion to the screen projection application via the camera HAL and camera FWK. After the screen projection application receives the image data after format conversion, it can overlay the image data with the image data of the electronic device 100's own system to obtain processed image data, and transmit the processed image data to the display screen for display.
[0323] 2. Audio Data Transmission Architecture
[0324] When the electronic device 100 and the electronic device 200 establish a wired connection (as shown in Figure 1B), as shown in Figure 8B, the bridge chip driver in the electronic device 100 can report an insertion event. Specifically, the bridge chip driver can report the insertion event to the DP control module, and the DP control module can then report the insertion event to the wired access manager. On the one hand, the wired access manager can report the insertion event to the screen projection application. On the other hand, the wired access manager can pass the insertion event to the audio manager, and the audio manager passes the insertion event to the audio device management module in the audio HAL to notify the audio HAL that a new device has been inserted (ie, connected to the electronic device 100). It can be understood that after the audio HAL receives the insertion event, parameters such as sampling rate, bit width, and number of channels can be read from the new device node.
[0325] As shown in Figure 8B, after the bridge chip driver in the electronic device 100 reports the insertion event, the audio path management module, audio recording thread, audio service, and audio HAL in the electronic device 100 can establish a recording path, and the audio path management module, audio playback thread, audio service, and audio HAL in the electronic device 100 can establish a playback path. Specifically, the audio path management module can send a recording path creation instruction to the audio recording thread, the audio recording thread then sends a corresponding instruction to the audio service, and the audio service then sends a corresponding instruction to the audio HAL, thereby achieving the creation of the recording path. Similarly, the audio path management module can send a playback path creation instruction to the audio playback thread, the audio playback thread then sends a corresponding instruction to the audio service, and the audio service then sends a corresponding instruction to the audio HAL, thereby achieving the creation of the playback path.
[0326] In some embodiments of the present application, the audio recording thread, the PlaybackThread in the audio service, and the audio data processing module in the audio HAL can establish a recording path, while the audio playback thread, the AudioRecord in the audio service, and the audio data processing module in the audio HAL can establish a playback path.
[0327] Based on the above content, after the electronic device 100 receives the audio data transmitted by the electronic device 200, the bridge chip driver in the electronic device 100 can upload the audio data to the screen projection application via the recording path (including audio HAL and audio service). The audio recording thread in the screen projection application can transfer the audio data to the audio playback thread. The audio playback circuit can download the audio data to the audio driver via the playback path (including audio service and audio HAL), and finally be played by the relevant hardware device.
[0328] 3. USB Data Packet Transmission Architecture
[0329] When the electronic device 100 and the electronic device 200 establish a wired connection (as shown in Figure 1B), as shown in Figure 8C, the USB driver, USB configuration module and USB service module in the electronic device 100 can perform UDC enumeration. That is, the electronic device 100 can establish a USB communication connection with the electronic device 200 and identify the USB device type of the electronic device 100. In some embodiments of the present application, the electronic device 200 can identify the electronic device 100 as a human interface device (Human Interface Device, HID) through UDC enumeration. In some embodiments of the present application, during the UDC enumeration process of the electronic device 100 and the electronic device 200, the electronic device 100 can act as a slave device, and the electronic device 200 can act as a host device.
[0330] The USB data packet transmission architecture is described below by taking the corresponding operation performed by a stylus on the electronic device 100 as an example.
[0331] First, the stylus can send Bluetooth pairing information to the Bluetooth driver in the electronic device 100. The Bluetooth driver can upload the Bluetooth pairing information sent by the stylus and display the corresponding pairing options through the stylus pop-up window. The user can choose whether to pair the stylus with the electronic device 100 through Bluetooth (i.e., establish a Bluetooth connection) based on the corresponding pairing options displayed in the stylus pop-up window on the electronic device 100. When the user triggers the stylus to establish a Bluetooth connection with the electronic device 100, the user can use the stylus to write or draw on the electronic device 100, as well as trigger corresponding controls. In this case, the stylus can generate pressure information, and the electronic device 100 can obtain TP point information through the touch sensor. It is understood that the TP point information may include the coordinates of the contact position of the stylus on the electronic device 100. The stylus can transmit the pressure information generated by the stylus to the Bluetooth driver via the Bluetooth connection between the stylus and the electronic device 100. The Bluetooth driver then transmits the pressure information to the USB service module. The TP point information obtained by the electronic device 100 can be transmitted by the TP driver in the electronic device 100 to the USB service module. After the USB service module receives the pressure data transmitted by the Bluetooth driver and the TP point information transmitted by the TP driver, it can process the pressure data and TP point information to obtain processed TP point information. It is understandable that the processed TP point information can include not only the contact position coordinates, but also the pressure information corresponding to the corresponding contact position coordinates. It is understandable that if the user writes or draws on the electronic device 100 with a stylus, the electronic device 100 can determine the thickness and position of the stylus line based on the processed TP point information. The USB service module can transmit the processed TP point information to the USB configuration module, and the USB configuration module can transmit the processed TP point information to the USB driver, which can send the processed TP point information to the electronic device 200 in the form of a USB data packet. It is understandable that, in conjunction with Figure 5B, the USB driver can send the processed TP point information to the electronic device 200 via interface 1 in the form of a USB data packet.
[0332] It can be understood that according to the above, after the electronic device 200 receives the USB data packet transmitted by the electronic device 100, it can determine the target position coordinates in the electronic device 200 based on the TP reporting point information in the USB data packet, and perform corresponding operations based on the target position coordinates in the electronic device 200 (as shown in Figure 5B), obtain updated image data, and transmit the updated image data to the display screen of the electronic device 200 for display.
[0333] It is understandable that after the electronic device 200 obtains the updated image data, it can transmit the updated image data to the electronic device 100 for display. For details, please refer to the image data transmission architecture shown in Figure 8A, which will not be elaborated here.
[0334] In some embodiments of the present application, when a user touches the electronic device 100 with a finger or the like, the electronic device 100 may only obtain TP point information without obtaining pressure information. Subsequent operations may refer to the above and will not be repeated here.
[0335] In some other embodiments of the present application, when a user touches the electronic device 100 with a finger or the like, the electronic device 100 can obtain both TP point information and pressure information. For subsequent operations, please refer to the above and will not be described in detail here. In one possible implementation, the electronic device 100 can obtain pressure information when a finger or the like touches the electronic device 100 through a pressure sensor.
[0336] It is understood that when the electronic device 100 includes an HDMI control module, the electronic device 100 can establish a corresponding image data transmission architecture and an audio data transmission architecture. The specific implementation of the image data transmission architecture and the audio data transmission architecture of the electronic device 100 can be referred to the relevant descriptions of Figures 8A and 8B respectively, and will not be repeated here.
[0337] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A display method, characterized in that: Applied to a first device including a display screen and a first interface, the first interface being an interface supporting a first type of signal input and a second type of signal output, the method comprising: When the first device establishes a wired connection with the second device through the first interface, the first device displays a first interface on the display screen, where the first interface includes a first display area, and the first display area is used to display content of the display interface of the second device; The first device detects a first operation performed on the first display area; In response to the first operation, content displayed in the first display area changes.
2. The method according to claim 1, characterized in that In a case where the first device establishes a wired connection with the second device through the first interface, the first device displays a first interface on the display screen, specifically including: The first device receives first data sent by the second device through the first interface; the first data includes first image data; the first data is the first type signal; The first device displays the first interface on the display screen based on the first image data; content displayed in the first display area in the first interface corresponds to the first image data.
3. The method according to claim 2, characterized in that The format of the first image data included in the first data is a first format; after the first device receives the first data sent by the second device through the first interface, the method further includes: the first device converts the format of the first image data from the first format to a second format through a bridge chip; The first device displays the first interface on the display screen based on the first image data, specifically including: the first device displays the first interface on the display screen based on the first image data in the second format.
4. The method according to claim 3, characterized in that The first data further includes first audio data; before the first device converts the format of the first image data from the first format to a second format through the bridge chip, the method further includes: The first device separates the first image data and the first audio data in the first data through the bridge chip; After the first device separates the first image data and the first audio data in the first data through the bridge chip, the method further includes: the first device plays the first audio data.
5. The method according to any one of claims 2 to 4, characterized in that: The first display area is the entire display area of the display screen.
6. The method according to any one of claims 2 to 4, characterized in that: The first interface also includes a second display area; the second display area is used to display the display content corresponding to the system of the first device itself; The first device displays the first interface on the display screen based on the first image data, specifically including: The first device processes the first image data and the second image data, and transmits the processed image data to the display screen; the second image data is image data generated by the system of the first device itself; The first device displays the first display area and the second display area on the display screen; the content displayed in the first display area corresponds to the first image data; and the content displayed in the second display area corresponds to the second image data.
7. The method according to claim 6, characterized in that The format of the first image data included in the first data is a first format; and the first device processes the first image data and the second image data, specifically including: The first device converts the format of the first image data from the first format to a second format through a bridge chip; The first device performs layer overlay processing on the first image data in the second format and the second image data.
8. The method according to any one of claims 3, 4 and 7, characterized in that: The first type signal is a DP signal; the first data is DP signal data; the first format is a DP data format, and the second format is a CSI data format.
9. The method according to claim 3 or 4, characterized in that Before the first device establishes a wired connection with the second device through the first interface, the method further includes: The first device controls a rest pin of the bridge chip to receive the first data based on a pipeline customized for the bridge chip.
10. The method according to any one of claims 2 to 9, characterized in that In response to the first operation, the content displayed in the first display area changes, specifically including: In response to the first operation, the first device sends information related to contact position coordinates to the second device, and receives second data sent by the second device through the first interface; the information related to contact position coordinates is the second type signal; the second data is the first type signal; the second data includes third image data; the third image data is updated image data obtained by the second device based on the information related to contact position coordinates; the third image data is different from the second image data; The first device displays a second interface on the display screen based on the third image data; the content displayed in the first display area in the second interface corresponds to the third image data; the content displayed in the first display area in the second interface is different from the content displayed in the first display area in the first interface.
11. The method according to any one of claims 1 to 10, characterized in that The first type of signal is a DP signal; the second type of signal is a USB signal; the first interface is a Type-C interface that supports DP signal input and USB signal output; the first device is provided with a second interface, and the second interface is a Type-C interface that supports DP signal output.
12. The method according to any one of claims 1 to 4 or any one of claims 6 to 7, characterized in that: The first interface also includes a second display area; before the first device displays the first interface on the display screen, the method also includes: the first device displays a third interface on the display screen; the content of the third interface is the same as the content displayed in the second display area in the first interface.
13. The method according to claim 12, characterized in that After the first device displays the first interface on the display screen, the method further includes: In response to an operation performed on the second display area, content displayed in the second display area changes.
14. The method according to claim 12, characterized in that After the first device displays the first interface on the display screen, the method further includes: In response to an operation performed on the first interface, the first device increases the first display area.
15. The method according to any one of claims 1 to 14, characterized in that After the first device displays the first interface on the display screen, the method further includes: The first device detects a second operation, and in response to the second operation, the first device enters an annotation mode; The first device detects an operation performed on the display screen, and displays a first batch of injection traces on the first display area in response to the operation performed on the display screen; The first device detects a third operation, and in response to the third operation, the first device saves the picture containing the first batch of annotation traces.
16. A display method, characterized in that: Applied to a second device including a display screen and a third interface, the third interface being an interface supporting a first type of signal output and a second type of signal input, the method comprising: When the second device establishes a wired connection with the first device through the third interface, the second device receives the contact position coordinate related information sent by the first device, determines the target position coordinates on the display screen based on the contact position coordinate related information, and obtains updated image data based on the target position coordinates; The second device transmits the updated image data to the display screen for display, and sends the updated image data to the first device.
17. An electronic device, characterized in that: include: A first interface, a display screen, one or more memories, and one or more processors; The one or more processors are coupled to the first interface, the display screen, and the one or more memories; the first interface is used to receive a first type of signal and to send a second type of signal; the memory is used to store a computer program; the processor is used to call the computer program so that the electronic device executes the method described in any one of claims 1-15.
18. The electronic device according to claim 17, characterized in that: The electronic device further includes a second interface, and the second interface is used to output the first type of signal.
19. The electronic device according to claim 17 or 18, characterized in that: The first type signal is a DP signal, and the second type signal is a USB signal.
20. An electronic device, characterized in that: The electronic device comprises a first interface, a bridge chip, a processor and a display screen; the first interface is connected to the bridge chip, the bridge chip is connected to the processor, and the processor is connected to the display screen; The first interface is used to connect to a second device, receive first data sent by the second device, and transmit the first data to the bridge chip; the first data is a first type signal; The bridge chip is used for converting the first image data in the first format in the first data into the first image data in the second format, and transmitting the first image data in the second format to the processor; The processor is used for processing the first image data in the second format and the image data generated by the system of the electronic device itself to obtain first processed image data; The display screen is used to display the first processed image data; The first interface is further used to send contact position coordinate related information to the second device; the contact position coordinate related information is a second type of signal; The first interface is also used to receive second data sent by the second device, and transmit the second data to the bridge chip; the second data is the first type signal; The bridge chip is further used for converting the third image data in the first format in the second data into the third image data in the second format, and transmitting the third image data in the second format to the processor; The third The image data is updated image data obtained by the second device based on the contact position coordinate related information; The processor is further used to process the third image data in the second format and the image data generated by the system of the electronic device itself to obtain second processed image data; The display screen is also used to display the second processed image data.
21. The electronic device according to claim 20, characterized in that: The first processed image data is image data obtained by layer overlay processing of the first image data in the second format and the image data generated by the system of the electronic device itself.
22. The electronic device according to claim 20, characterized in that: The first processed image data is first image data in the second format.
23. The electronic device according to any one of claims 20 to 22, characterized in that: The first type signal is a DP signal, and the second type signal is a USB signal.
24. An electronic device, characterized in that: include: a third interface, a display screen, one or more memories, and one or more processors; The one or more processors are coupled to the third interface, the display screen, and the one or more memories; the third interface is used to send a first type of signal and receive a second type of signal; the memory is used to store a computer program; the processor is used to call the computer program so that the electronic device executes the method described in claim 16.
25. A display system, characterized in that: The display system includes a first device and a second device; the first device includes a first interface; the second device includes a third interface; the first device is connected to the third interface of the second device via the first interface; the first device is an electronic device as described in any one of claims 17-23; the second device is an electronic device as described in claim 24.
26. The display system according to claim 25, characterized in that The display system also includes a switching device, and the first device is connected to the third interface of the second device through the first interface, specifically including: the first device is connected to the switching device through the first interface, and the switching device is connected to the third interface of the second device.
27. The display system according to claim 25 or 26, characterized in that: The third interface is an interface that supports DP signal output and USB signal input.
28. A computer storage medium, characterized in that include: Computer instructions; when the computer instructions are executed on a first device, the first device is caused to execute the method according to any one of claims 1 to 15.
29. A computer storage medium, characterized in that include: Computer instructions; when the computer instructions are executed on a second device, the second device is caused to execute the method of claim 16.
Citation Information
Patent Citations
Data transmission method and related equipment
CN112527221A
Display control method and related device
CN114115770A
Screen projection method and related device
CN115145515A
Display method and related equipment
CN117692694A
Systems and methods for implementing a dynamic and / or interactive media campaign
US9137557B1
