Display device, display device terminalization method, apparatus, and storage medium

Abstract

Embodiments of the present application provide a display device, a display device terminalization method, an apparatus and a storage medium. The display device comprises a display screen configured to display images; and a processor connected to the display screen. The method comprises: displaying, by the display screen, an interface of a terminal device when the display device is in a terminalization mode; in response to a first touch operation of a user on the display screen, transmitting a control instruction corresponding to the first touch operation to the terminal device, so that the terminal device executes the control instruction and transmits an execution result generated by the terminal device to the display device; and receiving the execution result transmitted by the terminal device and displaying, by the display screen, an interface corresponding to the execution result. In this way, the effect of the interaction between the user and the display device is the same as the effect of the interaction between the user and the terminal device, so that the user can achieve the same touch interaction experience with the terminal device on the large screen of the display device.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device, a display device terminalization method, an apparatus, and a storage medium. Background Technology

[0002] With the increasing prevalence of display devices (such as smart TVs) and terminal devices (such as smartphones), the demand for home entertainment and information interaction is becoming increasingly diversified. Users want an immersive experience on a large screen when watching movies and TV shows, but they also need the portability and convenience of touch interaction on their phones for social, office, and gaming scenarios.

[0003] In existing technologies, display devices cannot be directly integrated with the systems of terminal devices, resulting in users not being able to achieve the same touch interaction experience on the large screen of the display device as they do on the terminal device. Therefore, how to enable users to achieve the same touch interaction experience on the large screen of the display device as they do on the terminal device is an urgent problem to be solved. Summary of the Invention

[0004] This application provides a display device, a display device terminalization method, an apparatus, and a storage medium, enabling users to achieve the same touch interaction experience on the large screen of the display device as on a terminal device.

[0005] In a first aspect, embodiments of this application provide a display device, the display device comprising:

[0006] A display screen used for displaying images;

[0007] The processor connected to the display screen is configured to:

[0008] When the display device is in terminal mode, the interface of the terminal device is displayed on the display screen;

[0009] In response to a user's first touch operation on the display screen, a control command corresponding to the first touch operation is transmitted to the terminal device, so that the terminal device executes the control command and transmits the generated execution result to the display device;

[0010] Upon receiving the execution result transmitted by the terminal device, the interface corresponding to the execution result is displayed on the display screen.

[0011] In one possible implementation, displaying the interface of the terminal device via the display screen includes:

[0012] Obtain the current display parameters of the display device and the display content of the interface of the terminal device;

[0013] The interface of the terminal device is displayed on the display screen according to the display parameters and the display content.

[0014] In one possible implementation, the display device enters a terminal mode, including:

[0015] In response to the operation of entering terminal mode, the operating system of the terminal device is launched; wherein the operating system is an operating system obtained from the terminal device, or a pre-stored image operating system of the terminal device;

[0016] The initial interface of the terminal device is displayed on the screen.

[0017] In one possible implementation, displaying the initial interface of the terminal device via the display screen includes:

[0018] The display mode selection interface includes a landscape display mode and a portrait display mode.

[0019] In response to a second touch operation by the user selecting a target display mode, the target display parameters under the target display mode and the initial display content of the initial interface are obtained;

[0020] According to the target display parameters, the layout of the display content is adjusted so that the initial interface is displayed on the display screen in accordance with the target display mode; wherein, the target display mode is the landscape display mode or the portrait display mode.

[0021] In one possible implementation, the processor is further configured to:

[0022] In response to a third touch operation that adjusts the display mode, the new display mode corresponding to the third touch operation is obtained;

[0023] The interface of the terminal device is displayed using the new display method.

[0024] In one possible implementation, transmitting the control command corresponding to the first touch operation to the terminal device includes:

[0025] The control command corresponding to the first touch operation is converted into a target control command that the terminal device can recognize.

[0026] The target control command is transmitted to the terminal device.

[0027] In one possible implementation, displaying the interface corresponding to the execution result on the display screen includes:

[0028] Obtain the current display parameters of the display device;

[0029] Based on the display parameters and the execution result, the interface corresponding to the execution result is displayed on the display screen.

[0030] In one possible implementation, the terminal device and the display device are connected by a wired connection or a wireless connection.

[0031] In one possible implementation, the processor is further configured to:

[0032] After the display device enters the terminal mode, the memory usage of background applications is reduced.

[0033] In one possible implementation, the processor is further configured to:

[0034] When a touch delay duration is detected to be greater than a preset duration, at least one of the following actions will be performed:

[0035] Switch the connection mode between the display device and the terminal device;

[0036] Redisplay the current interface based on the current display parameters;

[0037] Reduce the memory usage of background applications;

[0038] The touch delay duration is the time between the moment the touch operation is received and the moment the interface displays the result of the touch operation.

[0039] Secondly, embodiments of this application provide a display device terminalization method, applied to the display device described in the first aspect and / or various possible embodiments of the first aspect, wherein the display device includes:

[0040] A display screen used for displaying images;

[0041] The method, involving a processor connected to the display screen, includes:

[0042] When the display device is in terminal mode, the interface of the terminal device is displayed on the display screen;

[0043] In response to a user's first touch operation on the display screen, a control command corresponding to the first touch operation is transmitted to the terminal device, so that the terminal device executes the control command and transmits the generated execution result to the display device;

[0044] Upon receiving the execution result transmitted by the terminal device, the interface corresponding to the execution result is displayed on the display screen.

[0045] Thirdly, embodiments of this application provide a display device terminalization apparatus, comprising:

[0046] The display unit is used to display the interface of the terminal device through the display screen when the display device is in terminal mode;

[0047] The processing unit is configured to respond to a user's first touch operation on the display screen, transmit a control command corresponding to the first touch operation to the terminal device, so that the terminal device executes the control command, and transmit the generated execution result to the display device;

[0048] The display unit is also used to receive the execution result transmitted by the terminal device and display the interface corresponding to the execution result through the display screen.

[0049] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described in the first aspect and / or various possible implementations of the first aspect.

[0050] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in the first aspect and / or various possible implementations of the first aspect.

[0051] This application provides a display device, a terminalization method, apparatus, and storage medium. The display device includes: a display screen for displaying images; and a processor connected to the display screen. The method includes: when the display device is in terminalization mode, displaying the interface of a terminal device through the display screen; responding to a user's first touch operation on the display screen, transmitting a control command corresponding to the first touch operation to the terminal device, causing the terminal device to execute the control command, and transmitting the generated execution result to the display device; receiving the execution result transmitted by the terminal device, and displaying the interface corresponding to the execution result through the display screen. The method provided in this application allows the display device to interact with the terminal device based on the user's touch operation while displaying the terminal device's interface, displaying the display interface corresponding to the user's touch operation. This enables users to interact with the display interface of the terminal device on the display device and achieves a response to the user's touch operation through the display device. Thus, the effect of user interaction with the display device is the same as the effect of user interaction with the terminal device, allowing users to achieve the same touch interaction experience on the large screen of the display device as on the terminal device. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] Figure 1A schematic diagram of the structure of a display device provided in this application;

[0054] Figure 2 A schematic diagram of an interface interaction for user selection of display mode provided in this application;

[0055] Figure 3 A schematic diagram of an interface for switching from landscape to portrait display mode, provided in this application;

[0056] Figure 4 A schematic diagram of an interface interaction for scaling the display interface provided in this application;

[0057] Figure 5 A flowchart illustrating a method for terminalizing a display device provided in this application;

[0058] Figure 6 A schematic diagram of an interface interaction in response to a user's first touch operation, provided for the purposes of this application;

[0059] Figure 7 A schematic diagram of an interface for outputting prompt information is provided in this application;

[0060] Figure 8 This is a schematic diagram of the structure of a processor in a display device provided in an embodiment of this application;

[0061] Figure 9 A flowchart illustrating another method for terminalizing a display device provided in this application;

[0062] Figure 10 This is a schematic diagram of the structure of a display device terminalization device provided in this application.

[0063] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0064] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0065] With the increasing prevalence of display devices (such as smart TVs) and terminal devices (such as smartphones), the demand for home entertainment and information interaction is becoming increasingly diversified. Users want an immersive experience on a large screen when watching movies and TV shows, but they also need the portability and convenience of touch interaction on their phones for social, office, and gaming scenarios.

[0066] For example, during a family gathering, users might want to display photos or videos from their phones on a TV screen while simultaneously performing touch operations directly on the TV screen (such as swiping to switch photos, zooming to view details, commenting, and interacting), without frequently switching devices or relying on a remote control. Furthermore, in remote work scenarios, users may need to project documents, spreadsheets, or other content from their phones onto the TV for editing and complete touch annotations or collaborative operations via the TV screen.

[0067] The existing technologies mainly include the following two types:

[0068] Screen mirroring technology: Copying the content of a mobile phone screen to a TV via wireless or wired means, with the TV acting only as a passive display device.

[0069] However, in screen mirroring technology, the TV screen cannot directly receive touch operation signals, and users still need to interact through their mobile phones. Furthermore, the display ratios of mobile phones and TVs are inconsistent, which may cause application interfaces to appear missized or have a disorganized layout on the large screen.

[0070] Multi-screen collaboration technology: This technology allows TVs and mobile phones to perform basic interactions (such as controlling playback and split-screen display).

[0071] However, the customized systems running on televisions differ architecturally from mobile operating systems (such as Android and iOS), preventing them from directly running mobile applications. They require middleware or adaptation layers, limiting application functionality (e.g., the inability to access hardware like cameras and microphones). Television screens typically use infrared or capacitive touch technology, but their touch signals undergo complex protocol conversions before being recognized by the mobile system, resulting in high touch latency, insufficient multi-touch accuracy, and an inability to achieve smooth gesture operations.

[0072] In summary, current technologies prevent display devices from directly integrating with terminal device systems, resulting in users not experiencing the same touch interaction experience on the large screen of the display device as they do on the terminal device. There is an urgent need for solutions that enable users to achieve the same touch interaction experience on the large screen of display devices as they do on the terminal device, in order to meet the diverse needs of users in scenarios such as home entertainment, office collaboration, and remote education.

[0073] Based on this, the display device and display device terminalization method provided in this application enable the display device to transmit control commands corresponding to user touch operations to the terminal device while displaying the interface of the terminal device. This allows the terminal device to execute the control commands, generate execution results, and transmit the results to the display screen of the display device for display. This achieves the display of the interface corresponding to the user's touch operation. This allows users to interact with the display interface of the terminal device on the display device and enables the display device to respond to user touch operations. Thus, the effect of user interaction with the display device is the same as the effect of user interaction with the terminal device.

[0074] In this application, the display device can be a device with a large screen, such as a television. The terminal device can be a touch device with a smaller screen, such as a mobile phone or tablet computer. This application does not specifically limit the display device and the terminal device.

[0075] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0076] Figure 1 A schematic diagram of a display device provided in this application. Please refer to [link / reference]. Figure 1 The display device 10 may include a processor 11 and a display screen 12.

[0077] Display device 10 can be a device with a display screen 12, which is a touch screen that supports user touch operation.

[0078] The display screen may include a touch module, which is a hardware component for acquiring user touch signals, such as an infrared touch frame, a capacitive touch film, or an optical touch sensor.

[0079] For example, the 65-inch infrared touch bezel supports 10-point touch.

[0080] Processor 11 can be a central processing unit (CPU) or a graphics processing unit (GPU).

[0081] The display screen 12 can display images. The display screen 12 can be connected to the processor 11.

[0082] In this application, after the display device enters the terminal mode, the display screen 12 can be used to display various interfaces of the terminal device.

[0083] This terminal-based mode displays the terminal device's interface on a display device and allows for interactive experiences with the terminal device through interaction with the display device. Essentially, it uses the display device as a giant screen terminal device.

[0084] First, the method for display devices to enter terminal mode will be explained.

[0085] For example, before the display device enters terminal mode, it can display its content on the screen, such as playing a movie. When a user wants to achieve the interactive experience of a terminal device through the display device's screen, a connection can be established between the display device and the terminal device.

[0086] For example, there may be a wired or wireless connection between the terminal device and the display device.

[0087] The wireless connection port also includes Bluetooth connection, WIFI connection and other methods. This application embodiment does not limit the wireless connection method.

[0088] In this way, the connection between the terminal device and the display device can be wired or wireless, making the connection method more flexible.

[0089] After establishing a connection between the display device and the terminal device, an operation can be entered to control the display device to enter terminal mode. In response to this operation, the display device starts the terminal device's operating system and displays the terminal device's initial interface on the screen.

[0090] The operating system is either an operating system obtained from the terminal device or a pre-stored image of the terminal device's operating system.

[0091] The operating system obtained from the terminal device is the mobile phone operating system running on the phone's built-in core motherboard.

[0092] An integrated core motherboard refers to a hardware module that integrates core components such as CPU, GPU, and memory, and is used to directly run the mobile phone's operating system.

[0093] For example, if a mirrored operating system of the terminal device is pre-stored in the display device, the mirrored operating system can be run using virtualization technology. If a mirrored operating system of the terminal device is not pre-stored, the operating system can be obtained from the terminal device.

[0094] Virtualization technology is a processing method that uses virtualization software to create and run a mobile phone system image within a television system.

[0095] Optionally, the initial interface displayed by the display device through the screen can be the interface currently displayed by the terminal device, or it can be the preset interface of the terminal device, such as the main interface of an application. This application embodiment does not limit the initial interface or the preset interface.

[0096] Optionally, after the display device enters terminal mode, a prompt message indicating that it has entered terminal mode can be output. For example, this can be done by playing the language indicating that terminal mode has been entered, and / or by displaying a pop-up window prompting the user that the display device has entered terminal mode. This application embodiment does not limit the method for outputting the prompt message.

[0097] In this way, the terminal mode is entered only after the user's operation is received, and the terminal device's operating system is launched so that the display device can display the terminal device's interface and realize subsequent touch response.

[0098] In this application, after the display device enters terminal mode and before displaying the initial interface of the terminal device, the initial interface can be displayed according to the display method selected by the user.

[0099] For example, after the display device enters terminal mode, a mode selection interface can be displayed, which includes landscape display mode and portrait display mode. In response to a second touch operation by the user selecting the target display mode, the display device can obtain the target display parameters and the initial display content of the initial interface under the target display mode; based on the target display parameters, the layout of the display content is adjusted to display the initial interface on the screen according to the target display mode.

[0100] The target can be displayed in either landscape or portrait mode.

[0101] For example, the target display parameters may include display size, resolution, etc., but this application embodiment does not limit the target display parameters.

[0102] Adjusting the layout of displayed content can include adjusting the position, size, etc. of the displayed content to make the layout of the displayed content suitable for the target display method.

[0103] Taking the target display mode as landscape mode as an example, Figure 2 This is a schematic diagram of an interface interaction for users to select a display mode, as provided in this application.

[0104] like Figure 2 As shown, after the display device enters terminal mode, it can display the following: Figure 2 The mode selection interface is shown in (a) above.

[0105] exist Figure 2The mode selection interface shown in (a) includes landscape and portrait display modes.

[0106] like Figure 2 As shown in (a), the user can use touch to input touch operations for landscape display mode on this interface. The display device responds to the touch operation, determines that the user has selected landscape display mode, and obtains the display parameters for landscape display mode. Further, based on these display parameters and the initial display content of the initial interface, the layout of the displayed content is adjusted to achieve the desired effect. Figure 2 As shown in (b) above, the initial interface is displayed on the screen in landscape mode.

[0107] exist Figure 2 In this embodiment, the initial interface is the main interface of the terminal device, which includes icons of various applications installed on the terminal device, namely Application 1 to Application 6. Application 1 to Application 6 are applications installed on the terminal device, such as video playback applications, music playback applications, office applications, etc. This application embodiment does not specifically limit Application 1 to Application 6.

[0108] In this way, the layout of the content of the interface to be displayed is adjusted according to the display method selected by the user, so that the content of the interface to be displayed can be adapted to the display screen of the display device, thereby improving the display effect of the initial interface.

[0109] When the terminal device displays the interface in the previously selected display mode, the display mode can be adjusted according to the user's operation.

[0110] When the terminal device displays the interface in the previously selected display mode, if the user wants to adjust the display mode, they can input a third touch operation to adjust the display mode. The controller of the display device responds to the third touch operation, obtains the new display mode corresponding to the third touch operation, and displays the terminal device interface in the new display mode.

[0111] For details on the specific implementation method of the interface of the new display terminal device, please refer to the relevant description of displaying the initial interface in the target display mode in the above embodiments, which will not be repeated here.

[0112] For example, when the current display mode is portrait mode, the third touch operation can be to switch the display mode from portrait to landscape mode. Alternatively, regardless of whether the current display mode is portrait or landscape, the third touch operation can be to zoom in or out of the display interface.

[0113] Landscape mode can be used for full-screen display, which means the phone interface is displayed on the TV screen at its maximum scale. For example, a social media app interface can be displayed in full screen on a 65-inch TV.

[0114] by Figure 2 Taking (b) as an example, when the current display mode is landscape mode, Figure 3 This application provides a schematic diagram of an interface interaction for switching from landscape to portrait display mode.

[0115] like Figure 3 As shown in (a), when the display mode is landscape, the current display mode can be displayed in the upper right corner of the screen. When the user wants to switch the display mode, they can input a third touch operation on the landscape display mode touch icon displayed in the upper right corner. The controller can respond to this third touch operation and switch the landscape display mode to portrait display mode, as shown in (a). Figure 3 The interface shown in (b) is shown in the image.

[0116] by Figure 3 Taking (b) as an example, when the current display mode is portrait mode, Figure 4 This application provides a schematic diagram of an interface interaction method for scaling the display interface.

[0117] like Figure 4 As shown in (a), when a user wants to shrink the current display interface, they can input a third touch operation by extending two fingers outwards. The controller responds to this third touch operation by enlarging the currently portrait-oriented display interface, as shown in (a). Figure 4 The interface shown in (b) is shown in the image.

[0118] In this way, in terminal mode, users can adjust the current display method, improving the flexibility of terminal interaction on the display device and enhancing the user experience.

[0119] The display device terminalization method provided in this application embodiment is applied to the display device described in the above embodiment. Figure 5 This is a flowchart illustrating a method for terminalizing a display device provided in this application.

[0120] like Figure 5 As shown, the display device terminalization method includes the following steps:

[0121] S501. When the display device is in terminal mode, the interface of the terminal device is displayed on the screen.

[0122] The method by which the display device enters terminal mode can be found in the relevant description of the above embodiments, and will not be repeated here.

[0123] In this application, displaying the interface of a terminal device via a display screen may include: obtaining the current display parameters of the display device and the display content of the terminal device's interface; and displaying the interface of the terminal device via the display screen according to the display parameters and the display content.

[0124] For example, the process of displaying the interface of the terminal device through the display screen according to the display parameters and display content is similar to the process of displaying the initial interface in the above embodiments. Please refer to the relevant description above. The embodiments of this application will not be repeated here.

[0125] In this way, the interface content of the terminal device is displayed according to the current display parameters of the display device, so that the interface of the terminal device displayed on the display screen is more in line with the screen size of the display device. For example, the interface of the terminal device can be displayed in full screen on the display device, so that the display device can be used as a giant screen terminal device, improving the user experience.

[0126] S502, In response to the user's first touch operation on the display screen, the control command corresponding to the first touch operation is transmitted to the terminal device so that the terminal device executes the control command and transmits the generated execution result to the display device.

[0127] The first touch operation can be a click, a swipe, or other touch operation. This application embodiment does not limit the first touch operation.

[0128] The processor can respond to the user's first touch operation on the display screen, convert the control command corresponding to the first touch operation into a target control command that the terminal device can recognize, and transmit the target control command to the terminal device.

[0129] For example, the control commands corresponding to touch operations can be converted into target control commands that the terminal device can recognize through a protocol conversion unit in the display device. For instance, the STM32 microcontroller converts touch coordinates into Android MotionEvent events.

[0130] In this way, by converting the control commands of the display device into target control commands that the terminal device can recognize, and then transmitting the target control commands to the terminal device, the terminal device can recognize and execute the target control commands. For example, users can perform gesture operations such as swiping and zooming on the TV screen, and the touch signals are converted into mobile phone system commands in real time, significantly improving the smoothness and accuracy of the interaction.

[0131] For example, the first touch operation is clicking the touch icon of application A to open the interface of application A. The terminal device executes the target control instruction converted from the control instruction of the first touch operation, and the generated execution result is the relevant data of the application A's launch interface. This process is similar to the process by which a user clicks the touch icon of application A on the terminal device to generate the relevant data of application A's launch interface.

[0132] S503: Receive the execution result transmitted by the terminal device and display the interface corresponding to the execution result on the display screen.

[0133] Upon receiving the execution result transmitted from the terminal device, the current display parameters of the display device can be obtained; based on the display parameters and the execution result, the interface corresponding to the execution result can be displayed on the screen.

[0134] The execution result is the content corresponding to the first touch operation. For example, if the first touch operation is to play music A, the execution result is the interface for playing music A and the audio of music A. This application embodiment does not limit the execution result.

[0135] The process of displaying the interface corresponding to the execution result on the display screen according to the display parameters and execution result is similar to the process of displaying the initial interface in the above embodiment. Please refer to the relevant description above. The embodiments of this application will not be repeated here.

[0136] Therefore, the display device terminalization method provided in this application allows the display device to transmit control commands corresponding to the user's touch operation to the terminal device while displaying the interface of the terminal device. This enables the terminal device to execute the control commands, generate execution results, and transmit the results to the display screen of the display device for display. This achieves the display of the interface corresponding to the user's touch operation. This allows the user to interact with the display interface of the terminal device on the display device and enables the display device to respond to the user's touch operations. Thus, the effect of user interaction with the display device is the same as the effect of user interaction with the terminal device.

[0137] Based on the above embodiments, when the display device is in terminal mode, the process of the display device responding to the first touch operation and displaying the corresponding interface can be found in [reference needed]. Figure 6 As shown.

[0138] Figure 6 This application provides a schematic diagram of an interface interaction that responds to a user's first touch operation.

[0139] like Figure 6As shown in (a), when the display device displays the icons of applications 1-6 in landscape mode, if the user wants to use application 2, they can click the icon of application 2 to input a first touch operation. In response to this first touch operation, the display device sends a converted target control command corresponding to the first touch operation to the terminal device. The terminal device executes this target control command, that is, it performs the operation to launch application 2, generates the execution result of the application 2 launch interface, and transmits the execution result to the display device. Upon receiving the execution interface, the display device displays the application 2 launch interface, that is, it displays as shown in (a). Figure 6 The interface shown in (b) is shown in the image.

[0140] In this application, after the display device enters terminal mode, the memory usage of background applications is reduced.

[0141] For example, when a display device enters terminal mode while running video playback software, the video playback software runs in the background. At this time, the video playback software can be closed to free up memory and reduce the memory usage of background applications.

[0142] In this way, by reducing the memory usage of background applications, the display device can display the various interfaces of the terminal device more smoothly.

[0143] In this application, after the display device enters terminal mode, it can detect the touch delay of touch operation in real time. When the detected touch delay duration is greater than a preset duration, at least one of the following is performed:

[0144] Switch the connection method between the display device and the terminal device;

[0145] Redisplay the current interface based on the current display parameters;

[0146] Reduce the memory usage of background applications.

[0147] The touch delay duration is the time between the moment the touch operation is received and the moment the interface displays the result of the touch operation.

[0148] For example, switching the connection method may include switching from a wireless connection to a wired connection. If the terminal device and the display device are not connected using a physical connection cable, a prompt message may be displayed to remind the user to connect the terminal device and the display device using a physical connection cable. After the connection is detected, the wireless connection will be automatically switched to a wired connection to reduce touch latency.

[0149] Figure 7 This is a schematic diagram of an interface for outputting prompt information provided in this application.

[0150] like Figure 7As shown, if the touch latency of the display device is high, it is necessary to switch the wireless connection to a wired connection. Furthermore, the terminal device and the display device should not be connected using a physical cable. Output can be achieved. Figure 7 The system displays a prompt box with the message "Current latency is relatively high. We recommend switching to a wired connection. Please connect your terminal device to the display device using a data cable." Upon seeing this prompt, users can connect their terminal device and display device using a data cable. Once the display device detects the connection, it will automatically switch from a wireless to a wired connection to reduce touch latency.

[0151] For example, the current interface is redisplayed based on the current display parameters, that is, the current interface is refreshed.

[0152] For example, reducing the memory usage of background applications may include operations such as closing background applications, but this application embodiment does not limit this.

[0153] In this way, when a large touch latency is detected, adjustments are made to the connection method and background application memory usage to reduce touch latency and improve user experience.

[0154] The following section describes the terminalization method for display devices in a modular manner. Figure 8 This is a schematic diagram of the structure of a processor for a display device provided in an embodiment of this application.

[0155] like Figure 8 As shown, the display device may include a touch module, a connectivity adapter module, a system integration module, a display optimization module, and a control module.

[0156] Touch module: Used to collect touch operation signals from the user on the TV screen, including touch position, touch pressure, and touch gestures (such as click, swipe, zoom, long press, etc.). This module uses an infrared touch frame, capacitive touch film or optical touch sensor to adapt to different sizes of TV screens. The touch accuracy is not less than 10-point touch and the response latency is ≤50ms.

[0157] For example, the module can use a 65-inch infrared touch frame to adapt to a 65-inch 4K TV screen, support 10-point touch, have a touch accuracy of ±2mm, a response latency of 30ms, and connect to the adapter module via a USB interface.

[0158] Connectivity Adapter Module: Enables bidirectional data transmission between the TV and the mobile phone's core motherboard or mobile device. It supports both wired connection (such as Type-C, HDMI 2.1) and wireless connection (such as 5G Wi-Fi, Bluetooth 5.2). The wired connection is used to transmit high-definition display signals and high-speed touch data, while the wireless connection is used to assist in data synchronization and remote control. This module has a built-in protocol conversion unit that can convert the touch module's signals into touch commands that the mobile phone system can recognize, while converting the mobile phone's display signals into an output format compatible with the TV.

[0159] For example, the module can use an adapter board that integrates a Type-C 3.2 interface and a Wi-Fi 6 module. The Type-C interface supports DisplayPort 1.4 display signal transmission and USB 3.2 data transmission, the Wi-Fi 6 module supports high-speed wireless data synchronization, and the protocol conversion unit uses an STM32 microcontroller to realize the conversion between touch signals and Android system MotionEvent events.

[0160] System Integration Module: The core consists of a mobile phone system mounting unit and a resource adaptation unit. The mobile phone system mounting unit can directly run the mobile phone operating system (such as Android or iOS) through the built-in mobile phone core motherboard (integrating core components such as CPU, GPU, memory, and storage), or run the mobile phone system image in the original TV system through virtualization technology. The resource adaptation unit is used to adapt the TV's hardware resources (such as screen, speakers, and camera) to the mobile phone system, enabling the mobile phone system to call the TV's large-screen display, high-fidelity speakers, and other hardware functions.

[0161] For example, the module can adopt a built-in mobile phone core motherboard mode, with the core motherboard equipped with a Qualcomm Snapdragon 8 Gen2 processor, 12GB of RAM, 256GB of storage, and running the Android 14 operating system. The resource adaptation unit uses a customized driver to achieve adaptation between the TV screen, speakers, camera and Android system.

[0162] Display optimization module: Used to adaptively adjust the display interface of the mobile phone system according to the size of the TV screen (such as 55 inches, 65 inches, 75 inches, etc.) and resolution (such as 4K, 8K), including icon scaling, layout rearrangement, font adaptation, etc., to ensure that the application interface is clearly displayed and reasonably laid out on the large screen. It also supports display mode switching (such as full screen mode, mobile phone ratio simulation mode) to meet the needs of different usage scenarios.

[0163] For example, the module can use an FPGA chip as the core processing unit, with a built-in adaptive display algorithm that can automatically adjust the interface layout according to the resolution (3840×2160) and size of the TV screen. It supports an adjustable icon scaling ratio of 0.5-2.0 times, and the display modes include full-screen mode and 16:9 mobile phone ratio simulation mode.

[0164] Control module: As the core control unit of the device, it is electrically connected to the touch module, connection adapter module, system integration module and display optimization module respectively. It is used to coordinate the workflow of each module, handle the reception and parsing of touch signals, the optimization and output of display signals, the scheduling and allocation of hardware resources, etc., to ensure the stability and smoothness of the entire system operation.

[0165] For example, this module can use an ARM Cortex-A76 processor, communicate with other modules via an I2C bus, and incorporate a running status monitoring algorithm and a resource scheduling algorithm to optimize system performance in real time.

[0166] Based on the descriptions of the above modules, the methods for a display device to enter terminal mode and execute actions in terminal mode can include the following four different stages:

[0167] System startup and adaptation phase: Upon receiving the command to enter terminal mode, the control module triggers the system fusion module to run. If the built-in mobile phone core motherboard mode is used, the mobile phone operating system is directly started; if the virtualization mode is used, the mobile phone system image is loaded into the TV system and startup is completed. At the same time, the resource adaptation unit automatically detects the TV's hardware parameters (screen size, resolution, speaker model, etc.) and synchronizes the parameters to the mobile phone system to complete hardware resource adaptation.

[0168] For example, in this stage, the TV receives a command to enter terminal mode, the control module (ARM Cortex-A76 processor) sends a startup command to the system fusion module, and the mobile phone's core motherboard starts and runs the Android 14 operating system. The resource adaptation unit detects the TV's hardware parameters through a customized driver: screen size 65 inches, resolution 3840×2160, and Dolby Atmos sound system, and synchronizes these parameters to the Android system. The system automatically loads the corresponding hardware drivers, completing the adaptation of the sound system, camera, and mobile phone system. At this time, the TV screen displays the Android system startup interface.

[0169] Display optimization phase: Based on the TV screen parameters obtained by the resource adaptation unit, the display optimization module adaptively adjusts the mobile system's display interface: scaling the phone's default small-screen icons proportionally to the screen size to ensure appropriate icon size; rearranging the application layout to distribute interface elements evenly on the large screen; optimizing font resolution to avoid blurring or jagged edges; and providing a display mode selection interface for users to switch between full-screen mode and mobile aspect ratio simulation mode as needed.

[0170] For example, the display optimization module receives screen parameters synchronized by the resource adaptation unit and starts an adaptive display algorithm: scaling the default 48×48 pixel icons of the Android system to 120×120 pixels to ensure clear display on a 65-inch screen; rearranging the layout of controls in the application interface, for example, adjusting the original vertically arranged multi-column controls to a horizontal distribution to avoid users frequently swiping the screen; and optimizing the font resolution to 300 dpi to eliminate blurring issues. Simultaneously, a display mode selection pop-up window appears on the TV screen, allowing users to choose between "full-screen mode" or "16:9 analog mode".

[0171] During the touch interaction phase: The touch module collects the user's touch operation signals on the TV screen in real time and transmits the signals to the connectivity adapter module. The protocol conversion unit converts the touch signals into a command format recognizable by the mobile phone system (such as MotionEvent events in Android), and transmits it to the mobile phone system via wired or wireless means. After the mobile phone system executes the corresponding command, it feeds back the operation result to the display optimization module, which then outputs the updated interface to the TV screen, achieving real-time synchronization between touch operation and display feedback.

[0172] For example, in this stage, the user clicks the social media app icon on the TV screen. The infrared touch frame captures the touch position coordinates (1000, 800) and the click action signal, which is transmitted to the connection adapter module via USB interface. The protocol conversion unit converts this signal into a MotionEvent.ACTION_DOWN event that the Android system can recognize, and transmits it to the phone's core motherboard via Type-C interface. The Android system executes the click command, launches the social media app, and transmits the app interface signal to the display optimization module. The FPGA chip optimizes the interface in real time and outputs it to the TV screen. The entire process has a response delay of 30ms. The user can intuitively see the app launch and perform subsequent operations, such as swiping to view Moments, zooming to view pictures, etc., with touch operation consistent with mobile phone operation.

[0173] Resource scheduling and maintenance phase: The control module monitors the operating status of each module in real time. When it detects that the touch signal latency is too high, it automatically switches to wired connection mode and optimizes the data transmission rate. When it detects that the application display is abnormal, it triggers the display optimization module to re-adapt the interface. At the same time, it dynamically schedules the mobile phone system's operating resources (such as memory and CPU) to ensure the smoothness of multi-tasking.

[0174] For example, during this phase, if a user simultaneously launches social media, video playback, and office software, the control module monitors in real time that CPU usage has risen to 80% and memory usage to 75%. It immediately triggers a resource scheduling algorithm: the video playback software's CPU priority is increased to the highest level to ensure smooth playback. Memory from idle background applications is released, reducing memory usage to 60%. Simultaneously, the control module detects stable touch signal transmission and maintains the Type-C wired connection. When the user closes some applications, CPU and memory usage decrease, and the control module automatically adjusts resource allocation to ensure the system always runs smoothly.

[0175] Based on the description of the above embodiments, the complete process of display device terminalization can be found in [reference needed]. Figure 9 As shown, Figure 9 A flowchart illustrating another method for terminalizing a display device provided in this application.

[0176] like Figure 9 As shown, the process of terminalizing a display device may include:

[0177] The display device initiates terminal mode; that is, after receiving the instruction to enter terminal mode, it enters the system startup and adaptation phase. In this phase, the mobile operating system is started via the built-in mobile phone core motherboard, or the mobile system image is loaded and started via virtualization technology. For details on which method is used to start the mobile system, please refer to the relevant descriptions in the above embodiments, which will not be repeated here. After the mobile system starts, it detects and adapts to the hardware parameters.

[0178] After the hardware parameters are detected and adapted, the display optimization stage begins. In this stage, based on the display method selected by the user (landscape display, portrait display, full-screen display, mobile phone aspect ratio simulation display, etc.), icon scaling, layout rearrangement, and font adaptation are performed.

[0179] After icon scaling, layout rearrangement, and font adaptation are completed, the touch interaction stage begins. In this stage, the following steps are executed in sequence: collecting touch signals, converting signals into phone system instructions, executing instructions and providing feedback on display signals, optimizing display signals, and outputting the results.

[0180] In summary, the display device and display device terminalization method provided in this application embodiment achieve deep integration of television and mobile phone systems. The television is no longer just a display terminal, but directly possesses the complete operating system and application ecosystem of a mobile phone. Users can use various mobile phone applications, such as social software, games, and office software, on the large screen, balancing the visual experience of the large screen with the richness of mobile phone applications. By combining the touch module with adaptation technology, the problems of low accuracy and high latency in traditional television touch operation are solved, achieving 10-point touch and a response latency of ≤50ms. The operation experience is consistent with that of a mobile phone, supporting various gesture operations such as clicking, swiping, and zooming. The display optimization module can adaptively adjust the interface layout according to the television screen parameters, avoiding the problem of application display errors on the large screen. It also supports multiple display mode switching to meet the needs of different scenarios. It supports wired and wireless dual connection modes. The control module can automatically optimize the connection method and resource scheduling according to the operating status, ensuring the stability and smoothness of system operation, adapting to television devices of different sizes and resolutions, and has strong compatibility.

[0181] Figure 10 This application provides a schematic diagram of the structure of a display device terminalization apparatus, as shown below. Figure 10 As shown, the display device terminalization device 100 provided in this embodiment includes:

[0182] The display unit 1001 is used to display the interface of the terminal device through the display screen when the display device is in terminal mode.

[0183] The processing unit 1002 is configured to respond to a user's first touch operation on the display screen, transmit the control command corresponding to the first touch operation to the terminal device, so that the terminal device executes the control command, and transmit the generated execution result to the display device.

[0184] The display unit 1001 is also used to receive the execution result transmitted by the terminal device and display the interface corresponding to the execution result through the display screen.

[0185] The display device terminalization device provided in this embodiment can execute the display device terminalization method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0186] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0187] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0188] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0189] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0190] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0191] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0192] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0193] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0194] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0196] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0197] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0198] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A display device, characterized in that, The display device includes: A display screen used for displaying images; The processor connected to the display screen is configured to: When the display device is in terminal mode, the interface of the terminal device is displayed on the display screen; In response to a user's first touch operation on the display screen, a control command corresponding to the first touch operation is transmitted to the terminal device, so that the terminal device executes the control command and transmits the generated execution result to the display device; Upon receiving the execution result transmitted by the terminal device, the interface corresponding to the execution result is displayed on the display screen.

2. The display device according to claim 1, characterized in that, The process of displaying the interface of the terminal device through the display screen includes: Obtain the current display parameters of the display device and the display content of the interface of the terminal device; The interface of the terminal device is displayed on the display screen according to the display parameters and the display content.

3. The display device according to claim 1 or 2, characterized in that, The display device enters terminal mode, including: In response to the operation of entering terminal mode, the operating system of the terminal device is launched; wherein the operating system is an operating system obtained from the terminal device, or a pre-stored image operating system of the terminal device; The initial interface of the terminal device is displayed on the screen.

4. The display device according to claim 3, characterized in that, The step of displaying the initial interface of the terminal device through the display screen includes: The display mode selection interface includes a landscape display mode and a portrait display mode. In response to a second touch operation by the user selecting a target display mode, the target display parameters under the target display mode and the initial display content of the initial interface are obtained; According to the target display parameters, the layout of the display content is adjusted so that the initial interface is displayed on the display screen in accordance with the target display mode; wherein, the target display mode is the landscape display mode or the portrait display mode.

5. The display device according to claim 4, characterized in that, The processor is also configured to: In response to a third touch operation that adjusts the display mode, the new display mode corresponding to the third touch operation is obtained; The interface of the terminal device is displayed using the new display method.

6. The display device according to claim 1, characterized in that, The step of transmitting the control command corresponding to the first touch operation to the terminal device includes: The control command corresponding to the first touch operation is converted into a target control command that the terminal device can recognize. The target control command is transmitted to the terminal device.

7. The display device according to claim 1, characterized in that, The step of displaying the interface corresponding to the execution result on the display screen includes: Obtain the current display parameters of the display device; Based on the display parameters and the execution result, the interface corresponding to the execution result is displayed on the display screen.

8. The display device according to claim 1, characterized in that, The terminal device and the display device are connected by a wired or wireless connection.

9. The display device according to claim 1, characterized in that, The processor is also configured to: After the display device enters the terminal mode, the memory usage of background applications is reduced.

10. The display device according to claim 1, characterized in that, The processor is also configured to: When a touch delay duration is detected to be greater than a preset duration, at least one of the following actions will be performed: Switch the connection mode between the display device and the terminal device; Redisplay the current interface based on the current display parameters; Reduce the memory usage of background applications; The touch delay duration is the time between the moment the touch operation is received and the moment the interface displays the result of the touch operation.

11. A method for terminalizing a display device, characterized in that, Applied to the display device according to any one of claims 1-10, the display device comprising: A display screen used for displaying images; The method, involving a processor connected to the display screen, includes: When the display device is in terminal mode, the interface of the terminal device is displayed on the display screen; In response to a user's first touch operation on the display screen, a control command corresponding to the first touch operation is transmitted to the terminal device, so that the terminal device executes the control command and transmits the generated execution result to the display device; Upon receiving the execution result transmitted by the terminal device, the interface corresponding to the execution result is displayed on the display screen.

12. A display device terminalization apparatus, characterized in that, include: The display unit is used to display the interface of the terminal device through the display screen when the display device is in terminal mode; The processing unit is configured to respond to a user's first touch operation on the display screen, transmit a control command corresponding to the first touch operation to the terminal device, so that the terminal device executes the control command, and transmit the generated execution result to the display device; The display unit is also used to receive the execution result transmitted by the terminal device and display the interface corresponding to the execution result through the display screen.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in claim 11.

14. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of claim 11.

Images