A display device, a mobile terminal, and a Bluetooth connection method.

Abstract

This application provides a display device, a mobile terminal, and a Bluetooth connection method. The Bluetooth connection method records mutual trust markers in both the display device and the mobile terminal after establishing a network communication connection. When establishing a Bluetooth connection, the mutual trust markers are detected, and mutual trust verification is performed. Upon successful verification, Bluetooth pairing is automatically initiated to establish a Bluetooth connection. The Bluetooth connection method provided by this application can directly and automatically perform Bluetooth pairing for display devices and mobile terminals that have passed trust verification, achieving seamless pairing, reducing manual user interaction, and improving user experience.

Description

Technical Field

[0001] This application relates to the field of smart TV technology, and in particular to a display device, a mobile terminal, and a Bluetooth connection method. Background Technology

[0002] Smart TVs are television products based on Internet application technology, featuring an open operating system and chip, and an open application platform. They enable two-way human-computer interaction and integrate multiple functions such as audio-visual entertainment and data transmission to meet diverse and personalized user needs. Smart TVs are also equipped with a Bluetooth module, which can be used to establish Bluetooth connections with remote controls, smart terminals, and other devices to transmit wireless control commands or multimedia data.

[0003] For example, a smart TV with a Bluetooth module can act as a Bluetooth sink for audio output. After the smart terminal and the smart TV are paired via Bluetooth, the smart TV can be used as a speaker to obtain better sound output quality.

[0004] However, when pairing a smart device with a smart TV via Bluetooth, the user needs to repeatedly perform pairing and connection interactions between the two devices. For example, the user needs to turn on the Bluetooth module on both the smart device and the smart TV, then search for Bluetooth devices on the smart device and select the desired device from the list of connectable Bluetooth devices to initiate a Bluetooth connection request. The user then responds to this Bluetooth connection request on the smart TV to complete the pairing process. If the initiated Bluetooth connection request includes a pairing code, the user needs to manually click the pairing code on the smart TV to pair. Clearly, this Bluetooth pairing process requires multiple manual operations from the user, reducing the user experience. Summary of the Invention

[0005] This application provides a display device, a mobile terminal, and a Bluetooth connection method to solve the problem of excessive operation in traditional Bluetooth connection methods.

[0006] In a first aspect, this application provides a display device, including: a display, a communicator, a Bluetooth module, and a controller. The display is used to display a user interface; the communicator is used to establish a network communication connection with a mobile terminal; and the Bluetooth module is used to establish a Bluetooth connection with the mobile terminal.

[0007] The controller is configured to perform the following procedure steps:

[0008] Receive a Bluetooth connection request sent by the mobile terminal;

[0009] In response to the Bluetooth connection request, the mutual trust mark of the mobile terminal is detected to perform trust verification on the mobile terminal. The mutual trust mark is information recorded when establishing a network communication connection with the mobile terminal.

[0010] If the mobile terminal carries the mutual trust tag, perform Bluetooth connection pairing to establish a Bluetooth connection.

[0011] Secondly, this application also provides a mobile terminal, including: a display unit, a communication unit, a Bluetooth circuit, and a processor. The display unit is used to display a user interface on the mobile terminal; the communication unit is used to establish a network communication connection with the display device; and the Bluetooth circuit is used to establish a Bluetooth connection with the display device.

[0012] The processor is configured to perform the following program steps:

[0013] Send a Bluetooth connection request to the display device;

[0014] The mutual trust marker of the display device is detected to perform trust verification on the display device. The mutual trust marker is information recorded when establishing a network communication connection with the display device.

[0015] If the display device carries the mutual trust tag, perform Bluetooth pairing to establish a Bluetooth connection.

[0016] Thirdly, this application also provides a Bluetooth connection method, including the following steps:

[0017] Establish a network communication connection between the mobile terminal and the display device;

[0018] The mobile terminal sends a Bluetooth connection request to the display device;

[0019] The mobile terminal and the display device respectively perform trust verification to detect the mutual trust mark between the display device and the mobile terminal. The mutual trust mark is information recorded in the mobile terminal and the display device respectively when the mobile terminal and the display device establish a network communication connection.

[0020] If the display device and the mobile terminal have the mutual trust tag, perform Bluetooth connection pairing to establish a Bluetooth connection.

[0021] Fourthly, this application also provides a Bluetooth connection system, including a mobile terminal and a display device. The mobile terminal includes a display unit, a communication unit, a Bluetooth circuit, and a processor; the display device includes a display, a communicator, a Bluetooth module, and a controller.

[0022] The mobile terminal is configured to establish a network communication connection with the display device and send a Bluetooth connection request to the display device after establishing the network communication connection;

[0023] The display device is configured to: receive the Bluetooth connection request, and in response to the Bluetooth connection request detect the mutual trust mark of the mobile terminal to perform trust verification on the mobile terminal; if the mobile terminal carries the mutual trust mark, perform Bluetooth connection pairing to establish a Bluetooth connection;

[0024] The mobile terminal is further configured to: detect a mutual trust tag on the display device to perform trust verification on the display device; if the display device carries the mutual trust tag, perform Bluetooth connection pairing to establish a Bluetooth connection; wherein the mutual trust tag is information recorded in the mobile terminal and the display device respectively when the mobile terminal and the display device establish a network communication connection.

[0025] As can be seen from the above technical solutions, the display device, mobile terminal, and Bluetooth connection method provided in this application can record mutual trust tags in both the display device and the mobile terminal after establishing a network communication connection. Therefore, when establishing a Bluetooth connection between the display device and the mobile terminal, mutual trust verification is performed by detecting the mutual trust tags. Upon successful verification, Bluetooth connection pairing is automatically initiated to establish a Bluetooth connection. The Bluetooth connection method provided in this application can directly and automatically perform Bluetooth connection pairing for display devices and mobile terminals that have passed trust verification, achieving seamless pairing, reducing the number of manual user interactions, and improving the user experience. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control device in an embodiment of this application;

[0028] Figure 2 This is a hardware configuration block diagram of the display device in the embodiments of this application;

[0029] Figure 3 This is a schematic diagram of the display device software configuration in an embodiment of this application;

[0030] Figure 4 This is a schematic diagram showing the icon control interface of the display device application in an embodiment of this application;

[0031] Figure 5This is a schematic diagram of the structure of the mobile terminal in the embodiments of this application;

[0032] Figure 6 This is a schematic diagram of the software architecture of the mobile terminal in the embodiments of this application;

[0033] Figure 7 This is a schematic diagram of the user interface of the mobile terminal in an embodiment of this application;

[0034] Figure 8a This is a schematic diagram of the traditional Bluetooth pairing process;

[0035] Figure 8b A diagram showing the device pairing code confirmation screen during Bluetooth pairing;

[0036] Figure 9a This is a schematic diagram of the screen projection effect in an embodiment of this application;

[0037] Figure 9b This is a schematic diagram of the screen projection operation process in the embodiments of this application;

[0038] Figure 9c This is a schematic diagram of the screen projection data transmission system in the embodiments of this application;

[0039] Figure 10a This is a schematic diagram illustrating the establishment of a Bluetooth connection in an embodiment of this application;

[0040] Figure 10b This is a Bluetooth connection signaling diagram from an embodiment of this application;

[0041] Figure 11 This is a flowchart illustrating the process of determining the application type in an embodiment of this application;

[0042] Figure 12a This is a schematic diagram of Bluetooth pairing connection in an embodiment of this application;

[0043] Figure 12b This is a Bluetooth pairing signaling diagram from an embodiment of this application;

[0044] Figure 13a This is a schematic diagram of the Bluetooth connection process in an embodiment of this application;

[0045] Figure 13b This is a Bluetooth connection signaling diagram from an embodiment of this application;

[0046] Figure 14a This is a schematic diagram of the process by which the mobile terminal performs the judgment in an embodiment of this application;

[0047] Figure 14b This is a signaling diagram showing the judgment process performed by the mobile terminal in this embodiment of the application.

[0048] Figure 15This is a schematic diagram of the display device synchronization process in an embodiment of this application;

[0049] Figure 16 This is a schematic diagram of the Bluetooth connection structure of the display device in the embodiments of this application;

[0050] Figure 17 This is a schematic diagram of the Bluetooth connection structure of the mobile terminal in an embodiment of this application. Detailed Implementation

[0051] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0052] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0053] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0054] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0055] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0056] Figure 1 This is a schematic diagram illustrating a usage scenario of the display device according to an embodiment. For example... Figure 1 As shown, the display device 200 also communicates with the server 400, and the user can operate the display device 200 through the mobile terminal 300 or the control device 100.

[0057] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes at least one of infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via a wired connection. Users can control the display device 200 by inputting user commands through at least one method, such as buttons on the remote control, voice input, or control panel input.

[0058] In some embodiments, the mobile terminal 300 may include any one of a mobile phone, tablet computer, computer, laptop computer, AR / VR device, etc.

[0059] In some embodiments, a mobile terminal 300 may also be used to control the display device 200. For example, an application running on the mobile terminal may be used to control the display device 200.

[0060] In some embodiments, mobile terminal 300 and display device may also be used for data communication.

[0061] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the mobile terminal 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0062] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0063] In some embodiments, software steps executed by one execution entity can be migrated to another execution entity with which it communicates data, as needed. For example, software steps executed by a server can be migrated to a display device with which it communicates data, and vice versa.

[0064] Figure 2 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0065] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 275, an audio output interface 285, a memory, a power supply, and a user interface.

[0066] In some embodiments, the controller includes a central processing unit, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first to an nth interface for input / output.

[0067] In some embodiments, the display 275 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals from the controller output, and a user control UI interface, etc.

[0068] In some embodiments, the display 275 may be at least one of a liquid crystal display, an OLED display, and a projection display, and may also be a projection device and a projection screen.

[0069] In some embodiments, the tuner 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0070] In some embodiments, the communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0071] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0072] In some embodiments, the external device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0073] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0074] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the display 275, the controller 250 can perform operations related to the object selected by the user command.

[0075] In some embodiments, the object can be any of the optional objects, such as a hyperlink, an icon, or other operable controls. Operations related to the selected object include: displaying links to hyperlinked pages, documents, images, etc., or performing operations corresponding to the program associated with the icon.

[0076] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0077] A CPU (CPU) processor is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately for the display and playback of various audio and video content. A CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0078] In some embodiments, a graphics processor is used to generate at least one of various graphical objects, such as icons, operation menus, and user-input-based graphics. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, the rendered objects being displayed on a monitor.

[0079] In some embodiments, a video processor is configured to receive an external video signal and perform at least one of the following video processing operations according to a standard encoding and decoding protocol of the input signal: decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc., to obtain a signal that can be directly displayed or played on a display device 200.

[0080] In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, and a display formatting module. The demultiplexing module demultiplexes the input audio and video data streams. The video decoding module processes the demultiplexed video signal, including decoding and scaling. The image compositing module, such as an image synthesizer, overlays and blends a GUI signal generated by a graphics generator based on user input or its own generation with the scaled video image to generate a displayable image signal. The frame rate conversion module converts the input video frame rate. The display formatting module modifies the received frame rate-converted video output signal to conform to a display format, such as outputting RGB data signals.

[0081] In some embodiments, an audio processor is configured to receive external audio signals, perform decompression and decoding according to a standard codec protocol of the input signal, and at least one of the following processes: noise reduction, digital-to-analog conversion, and amplification, to obtain a sound signal that can be played in a speaker.

[0082] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0083] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, enabling the conversion between the internal form of information and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0084] In some embodiments, the user interface 265 is an interface that can be used to receive control input (e.g., physical buttons on the display device body, or others).

[0085] In some embodiments, the display device's system may include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. Applications are compiled into machine code after startup, forming a process.

[0086] See Figure 3 In some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0087] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0088] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0089] like Figure 3As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0090] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0091] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0092] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 3 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0093] Figure 5 A schematic diagram of the structure of the mobile terminal 300 is shown.

[0094] The following uses a mobile terminal 300 as an example to illustrate the embodiment in detail. It should be understood that... Figure 5 The mobile terminal 300 shown is merely an example, and the mobile terminal 300 may have more than... Figure 5 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0095] like Figure 5 As shown, the mobile terminal 300 includes components such as a radio frequency (RF) circuit 310, a memory 320, a display unit 330, a camera 340, a sensor 350, an audio circuit 360, a wireless Fidelity (Wi-Fi) circuit 370, a processor 380, a Bluetooth circuit 381, and a power supply 390.

[0096] The RF circuit 310 can be used to receive and transmit signals during information transmission or calls. It can receive downlink data from the base station and hand it over to the processor 380 for processing; it can also send uplink data to the base station. Typically, the RF circuit includes, but is not limited to, devices such as antennas, at least one amplifier, transceivers, couplers, low-noise amplifiers, and duplexers.

[0097] The memory 320 can be used to store software programs and data. The processor 380 executes various functions of the mobile terminal 300 and performs data processing by running the software programs or data stored in the memory 320. The memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 320 stores an operating system that enables the mobile terminal 300 to run. In this application, the memory 320 may store the operating system and various applications, and may also store code that executes the methods described in the embodiments of this application.

[0098] The display unit 330 can be used to receive input digital or character information and generate signal inputs related to user settings and function control of the mobile terminal 300. Specifically, the display unit 330 may include a touch screen 331 disposed on the front of the mobile terminal 300, which can collect touch operations of the user on or near it, such as clicking a button, dragging a scroll box, etc.

[0099] The display unit 330 can also be used to display information input by the user or information provided to the user, as well as various menus of the mobile terminal 300, forming a graphical user interface (GUI). Specifically, the display unit 330 may include a display screen 332 disposed on the front of the mobile terminal 300. The display screen 332 may be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 330 can be used to display the various graphical user interfaces described in this application.

[0100] The touchscreen 331 can be placed over the display screen 332, or the touchscreen 331 and the display screen 332 can be integrated to realize the input and output functions of the mobile terminal 300. After integration, it can be referred to as a touch display screen. In this application, the display unit 330 can display the application and the corresponding operation steps.

[0101] Camera 340 can be used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to processor 380 to be converted into a digital image signal.

[0102] The mobile terminal 300 may also include at least one sensor 350, such as an accelerometer 351, a proximity sensor 352, a fingerprint sensor 353, and a temperature sensor 354. The mobile terminal 300 may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0103] Audio circuitry 360, speaker 361, and microphone 362 provide an audio interface between the user and mobile terminal 300. Audio circuitry 360 converts received audio data into electrical signals, which are then transmitted to speaker 361, where they are converted into sound signals for output. Mobile terminal 300 may also be equipped with volume buttons for adjusting the volume of the sound signal. On the other hand, microphone 362 converts collected sound signals into electrical signals, which are received by audio circuitry 360, converted into audio data, and then output to RF circuitry 310 for transmission to, for example, another terminal, or to memory 320 for further processing. In this application, microphone 362 can acquire the user's voice.

[0104] Wi-Fi is a short-range wireless transmission technology. Mobile terminal 300 can use Wi-Fi circuit 370 to help users send and receive emails, browse web pages, and access streaming media, providing users with wireless broadband internet access.

[0105] The processor 380 is the control center of the mobile terminal 300. It connects various parts of the terminal via various interfaces and lines, and performs various functions and processes data by running or executing software programs stored in the memory 320 and calling data stored in the memory 320. In some embodiments, the processor 380 may include one or more processing units; the processor 380 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 380. In this application, the processor 380 can run the operating system, applications, user interface display and touch response, and the processing methods described in the embodiments of this application. Furthermore, the processor 380 is coupled to the input unit and the display unit 330.

[0106] Bluetooth circuit 381 is used to exchange information with other Bluetooth devices that also have Bluetooth circuitry via the Bluetooth protocol. For example, mobile terminal 300 can establish a Bluetooth connection with wearable electronic devices (such as smartwatches) that also have Bluetooth circuitry via Bluetooth circuit 381, thereby exchanging data.

[0107] The mobile terminal 300 also includes a power supply 390 (such as a battery) that supplies power to various components. The power supply can be logically connected to the processor 380 through a power management system, thereby enabling the management of charging, discharging, and power consumption. The mobile terminal 300 may also be equipped with a power button for powering on and off the terminal, as well as locking the screen.

[0108] Figure 6 This is a software structure block diagram of the mobile terminal 300 according to an embodiment of the present invention.

[0109] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0110] The application layer can include a series of application packages.

[0111] like Figure 6 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0112] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0113] like Figure 6 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0114] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0115] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0116] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0117] The phone manager is used to provide communication functions for the mobile terminal 300. For example, it manages call status (including connection and disconnection).

[0118] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0119] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating the mobile device, and flashing indicator lights.

[0120] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0121] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0122] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0123] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0124] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0125] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0126] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0127] A 2D graphics engine is a graphics engine for 2D drawing.

[0128] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0129] The following example, using a photo capture scenario, illustrates the workflow of the mobile terminal 300's software and hardware.

[0130] When the touchscreen 331 receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, touch operation timestamp, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a touch click as an example, where the click corresponds to the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through the camera 340.

[0131] In some specific implementations, such as Figure 7 As shown, users can open the corresponding application by touching the application icon on the user interface, or open the corresponding folder by touching the folder icon on the user interface.

[0132] Based on the aforementioned display device 200 and mobile terminal 300, a Bluetooth connection can be established between them. Bluetooth is a short-range wireless connection that allows the transmission of control commands and media data between the display device 200 and the mobile terminal 300. During data transmission, the Bluetooth connection has a sender (source) and a receiver (sink). For example, the mobile terminal 300 can transmit audio data to the display device 200 via Bluetooth to play sound resources from the mobile terminal 300 using the display device 200's speakers. In the audio data transmission process, the mobile terminal 300 acts as the source, sending the audio data, while the display device 200 acts as the sink, receiving the audio data.

[0133] To establish a Bluetooth connection between display device 200 and mobile terminal 300, Bluetooth pairing operations need to be performed on both devices. For example, the user first searches for Bluetooth devices on the mobile terminal 300 to obtain a list of Bluetooth devices. Then, the user selects the display device 200 from the list and sends a Bluetooth connection request to it. Upon receiving the request, display device 200 can display a pairing interface, which the user then confirms to complete the pairing process. Figure 8a , Figure 8b The initiated Bluetooth connection request will include a pairing code, which the user needs to manually click on the pairing code on the smart TV to pair. Additionally, in some Bluetooth connection pairing processes, after the user performs a confirmation operation on the display device 200, a further confirmation operation for the Bluetooth connection pairing is required on the mobile terminal.

[0134] As can be seen, the Bluetooth connection process in the above embodiments requires the user to perform multiple interactive operations on different devices. This interaction limitation reduces the pairing efficiency of the Bluetooth connection. Because multiple interactions are required, it is difficult to achieve a seamless user experience with Bluetooth connections, thus limiting their use. For example, in some scenarios, the mobile terminal 300 can establish a network communication connection and a Bluetooth connection simultaneously with the display device 200, to transmit one type of data via the network communication connection and another type of data synchronously via the Bluetooth connection. The network communication connection can include communication connections established through networks such as wireless LANs and the Internet, used to transmit control commands and multimedia data, such as screen projection. To establish a wireless network communication connection, a communicator 210 can be provided in the display device 200, and a communication unit can be provided in the mobile terminal 300. The communicator 210 and the communication unit can be in the same network environment to establish a network communication connection.

[0135] During data transmission, data transmitted via Bluetooth needs to be synchronized with data transmitted via network communication. However, the Bluetooth pairing process takes time, making it difficult to guarantee data synchronization and affecting user experience. For example, when mobile terminal 300 sends video data to display device 200 via a screen mirroring connection established based on a WiFi LAN, and simultaneously sends audio data to display device 200 via Bluetooth, the Bluetooth pairing process will cause the audio data to be unable to be sent to display device 200 synchronously with the video data, resulting in screen mirroring / sound desynchronization and affecting user experience.

[0136] To reduce the number of user interactions during Bluetooth pairing, some embodiments of this application provide a Bluetooth connection method. The following will illustrate this method using screen projection data transmission as an example. It should be understood that the Bluetooth connection method provided in these embodiments is not limited to screen projection data transmission but can also be applied to other Bluetooth connection processes.

[0137] Screen mirroring allows the display device 200 to play content from the mobile terminal 300, such as... Figure 9a As shown. In some embodiments, as Figure 9b As shown, to establish a screen mirroring connection, the user can send a screen mirroring connection request through the mobile terminal 300 and transmit the request to the display device 200 via the WiFi network. The display device 200 then completes the transmission protocol configuration based on the screen mirroring connection request, thereby establishing a transmission channel for screen mirroring data with the mobile terminal 300.

[0138] The display device 200 can also establish a screen projection connection with the mobile terminal 300 through different connection methods. For example, when the display device 200 and the mobile terminal 300 are connected to the same wireless local area network, a screen projection connection can be established based on the WiFi network. As another example, when both the display device 200 and the mobile terminal 300 are equipped with NFC (Near Field Communication) components, a screen projection connection can be established through the NFC components. Obviously, the display device 200 and the mobile terminal 300 can also use other wired or wireless connection methods to establish a screen projection connection, such as RF radio frequency connection, infrared connection, cellular network, and other connection methods that those skilled in the art can conceive of based on the connection methods provided in the above embodiments.

[0139] After establishing a screen mirroring connection, the display device 200 can receive relevant data from the mobile terminal 300 through the screen mirroring data channel, thereby playing relevant content on the display device 200. Under different screen mirroring data transmission protocols, the form of the screen mirroring data acquired by the display device 200 varies. For example, under the DLNA (Digital Living Network Alliance) protocol, the display device 200 can obtain the link address of a resource on the mobile terminal 300 through the screen mirroring data channel, and obtain media asset content data by accessing that link address, thereby playing the media asset data. Simultaneously, the display device 200 can also obtain control commands sent by the mobile terminal 300 through the screen mirroring data channel while playing media asset data, such as fast forward, pause, and stop screen mirroring. For example, after establishing a connection through a communication protocol, the mobile terminal 300 (such as a mobile phone) and the display device 200 will negotiate device functions and network conditions to select a suitable audio-visual transmission format. After establishing the online protocol (Session) for audio and video streaming, a series of Real-Time Streaming Protocol (RTSP) control commands are used to control the playback and termination of the audio and video streaming.

[0140] Under other screen mirroring protocols such as Miracast, AirPlay, NFC mirroring, and LeBao screen mirroring, the display device 200 can directly obtain video data streams from the mobile terminal 300. This video data stream can be the video data stream corresponding to the final display screen of the display unit 330 of the mobile terminal 300; that is, the video data stream includes media content, UI interface, and other video content that can be displayed on the display unit 330 of the mobile terminal 300. Alternatively, the video data stream can simply be the video data that can be obtained from the mobile terminal 300, such as currently playing media video data. In this case, the mobile terminal 300 can act as a signal source for the display device 200. The display device 200 can continuously obtain video data through the screen mirroring data channel and display it in real time on the monitor 275, achieving a synchronized display effect with the mobile terminal 300.

[0141] The mobile terminal 300 can obtain different screen projection data according to different screen projection protocols. For example, for system applications or screen projection directly at the system level, the mobile terminal 300 can establish a screen projection data channel using Miracast, Airplay, or WiDi screen projection protocols.

[0142] Miracast is a wireless display standard developed by the Wi-Fi Alliance, based on Wi-Fi Direct. Miracast is compatible with Android devices, allowing them to share video feeds. Typically, Android 5.0 and later devices have the Miracast protocol built-in, enabling direct wireless screen mirroring between the mirroring and receiving devices via Wi-Fi P2P without requiring any additional software. Miracast also supports simultaneous audio and video transmission. However, due to the vast number and variety of Android devices, Miracast is not fully compatible with all of them, and Miracast-based mirroring lacks a screen mirroring code, failing to prevent malicious or accidental mirroring.

[0143] AirPlay is a wireless screen mirroring technology developed by Apple, suitable for wireless screen mirroring of iOS and Mac systems. Apple devices have built-in AirPlay mirroring functionality. When used with a screen mirroring device that supports AirPlay server service, the effect is clear and stable, and the protocol includes a built-in screen mirroring code. However, it is essential to ensure that the Apple device and the screen mirroring device are on the same local area network (same network segment) and cannot be used across network segments or VLANs.

[0144] WiDi screen mirroring is a wireless playback technology spearheaded by Intel, based on WiFi Direct technology. Windows 8 / 10 laptops natively support WiDi screen mirroring, allowing direct screen mirroring without installing any software, and also support USB reverse control functionality.

[0145] The three screen mirroring protocols described above can acquire not only the video data stream from the mobile terminal 300, but also the audio data stream, thus enabling the synchronous transmission of video and audio data to the display device 200, allowing the display device 200 to play the video and audio simultaneously. Because the Miracast and AirPlay screen mirroring protocols require system-level application permissions, they are typically only applicable to system applications, or to display devices 200 and mobile terminals 300 from the same manufacturer that directly establish a screen mirroring connection through built-in operating system rules.

[0146] Because the sound from mobile terminal 300 can only be obtained by system-level applications, or requires system-level modifications within the same manufacturer for both the sending (Source) and receiving (Sink) ends to transmit sound—for example, during NFC screen mirroring, mobile terminal 300 can simultaneously transmit video and audio streams to display device 200. However, if the application on mobile terminal 300 is not developed by the current manufacturer, it can only transmit the video stream to display device 200 for playback, and cannot transmit audio data. In other words, some third-party applications lack system-level permissions and cannot use the Miracast or AirPlay screen mirroring protocols, requiring the use of other screen mirroring protocols to establish a connection. For example, the LeBao screen mirroring application uses the LeBao screen mirroring protocol, and the NFC mirroring application uses the NFC screen mirroring protocol. Because these screen mirroring protocols lack system-level application permissions, they cannot directly obtain audio data from mobile terminal 300 after establishing a connection. Therefore, when using third-party applications for screen mirroring, the problem of only video being mirrored and audio not being transmitted may occur.

[0147] In order to transmit audio data, such as Figure 9c As shown, users can perform interactive operations through the mobile terminal 300 to input control commands for various purposes. For example, the input control commands can be used to establish a screen mirroring connection. The mobile terminal 300 can input control commands for establishing a screen mirroring connection in multiple ways, depending on the interaction principle. For example, the display unit 330 of the mobile terminal 300 includes a touch component; the user can input touch commands through the touch component to interact, enter the screen mirroring application, and establish a screen mirroring connection after clicking the screen mirroring button.

[0148] The screen mirroring application refers to an application installed on the mobile terminal 300, which can be a system application or a third-party application. Typically, system applications are those that come pre-installed on the mobile terminal 300 and cannot be uninstalled under normal operating conditions. It should be noted that in this embodiment, the term "system application" is broad, encompassing not only applications developed by the system operator but also other applications with system-level data access permissions.

[0149] Third-party applications refer to applications that can be installed on and run on the mobile terminal 300 to perform certain specific functions. Examples include media playback applications and game applications. Similarly, the term "third-party applications" is broad, encompassing not only applications developed by non-system operators but also applications developed by system operators that do not have system-level data access permissions.

[0150] In addition, users can also input control commands to establish a screen mirroring connection in other ways. For example, for a mobile terminal 300 with a built-in intelligent voice system, users can input control voice commands, such as "Start screen mirroring" or "Start LeBao screen mirroring," to establish a screen mirroring connection. After the user inputs the aforementioned control voice commands, the mobile terminal 300 can automatically launch the screen mirroring application and establish a screen mirroring connection with the display device 200, enabling the display of content from the mobile terminal 300 on the display device 200.

[0151] It should be noted that during the aforementioned screen mirroring connection process, the connection can be established bidirectionally. That is, the user can input control commands through the mobile terminal 300 to initiate a screen mirroring connection with the display device 200. The user can also input control commands through the display device 200; the specific input method is similar to that of the mobile terminal 300 and will not be elaborated further. After inputting control commands, the display device 200 can also initiate a screen mirroring connection with the mobile terminal 300, thereby projecting the content from the display device 200 onto the mobile terminal 300.

[0152] After receiving the control command for establishing a screen mirroring connection, the mobile terminal 300 can respond to the control command by sending a screen mirroring connection request to the display device through the communication unit to establish a screen mirroring connection. The screen mirroring connection request can be read by the display device 200, and the display device 200 can respond according to the specific content of the screen mirroring connection request, so as to establish a screen mirroring data transmission channel between the mobile terminal 300 and the display device 200.

[0153] Therefore, the screen mirroring connection request needs to include information related to establishing a screen mirroring data transmission channel. This includes, for example, the connection type, protocol, and address of the mobile terminal 300. After receiving the screen mirroring connection request, the display device 200 parses the connection type, protocol, and address of the mobile terminal 300 from the request, and negotiates the connection information using the method corresponding to the transmitted protocol to establish a data transmission channel after the negotiation is complete.

[0154] After establishing the data transmission channel, the screen mirroring application can acquire the video data stream from the mobile terminal 300 in real time and send it to the display device 200 through the screen mirroring data transmission channel. Upon receiving the video data stream, the display device 200 can display it on its monitor 275 in real time, thus achieving screen mirroring of the video content. Obviously, since there is a certain difference between the shape of the display area of ​​the monitor 275 of the display device 200 and the display area of ​​the display unit of the mobile terminal 300, the display state needs to be adjusted during the display of the screen mirrored video. For example, the video image ratio can be scaled to ensure that the screen mirrored video is fully displayed on the monitor 275. Furthermore, the display device 200 can perform further optimization processing on the screen mirrored video data, such as performing interpolation algorithms, to obtain better display effects.

[0155] Since some third-party screen mirroring applications do not have permission to obtain audio data when transferring screen mirroring data, the established screen mirroring data transmission channel can only transmit video signals. Therefore, in order to transmit audio signals, after establishing a screen mirroring connection, the mobile terminal 300 can also detect the screen mirroring application type to determine whether the screen mirroring application used for the current screen mirroring operation is a system application or a third-party application.

[0156] Based on the detection results of the screen mirroring application type, different screen mirroring data transmission methods can be determined. If the screen mirroring application type is a system application, meaning the current screen mirroring application has permission to obtain audio data streams, it can directly send audio data streams to the display device 200 through the screen mirroring data transmission channel, thereby achieving synchronous playback of video and audio signals on the display device 200. If the screen mirroring application type is a third-party application, meaning the current screen mirroring application does not have permission to obtain audio data streams, it can only transmit video data, not audio data, through the screen mirroring data transmission channel. Therefore, it can connect to the display device 200 via Bluetooth to send audio data streams.

[0157] As can be seen, by connecting the display device 200 and the mobile terminal 300 via Bluetooth, the display device 200 is used as the receiving end (sink end) of the Bluetooth connection, and the mobile terminal 300 is used as the sending end (source end) of the Bluetooth connection. That is, the display device 200 is used as the audio playback device of the mobile terminal 300, thereby outputting the sound in the mobile terminal 300 in real time and realizing the synchronous playback of video data and audio data.

[0158] In some embodiments, when establishing a screen mirroring connection, the display device 200 can also detect the type of the screen mirroring application. That is, after receiving a screen mirroring connection request sent by the mobile terminal 300, the display device 200 can detect the type of the screen mirroring application in response to the screen mirroring connection request. If the screen mirroring application type is a third-party application, it connects to the mobile terminal 300 via a Bluetooth module to obtain audio data streams. In other words, in this embodiment, the display device 200 can detect the type of screen mirroring application, thus eliminating the need for additional configuration of the mobile terminal 300 and enabling the above-mentioned screen mirroring data transmission method between mobile terminals 300 from different manufacturers and the display device 200.

[0159] According to the above embodiments, when establishing a screen mirroring connection, the method of sending relevant data can be determined based on the type of screen mirroring application. For third-party applications, Bluetooth connection can be used to transmit audio data, thus alleviating the problem that third-party applications can only transmit video data and not audio data. For system applications, since they have permission to obtain audio data streams, they can directly transmit audio data through the established screen mirroring connection, i.e., Figure 10a , Figure 10b As shown, in some embodiments, if the screen mirroring application type is a system application, a screen mirroring connection can be established with the mobile terminal 300 through the system screen mirroring protocol to transmit video data streams and audio data streams.

[0160] For example, when an application needs to obtain audio stream recording permission from the Android system, it can first obtain sound acquisition permission, such as the permission "android.permission.CAPTURE_AUDIO_OUTPUT". This permission is a system permission and is not allowed for third-party apps. If it is a system application, it can obtain the audio stream through AudioRecord. At the application level, the system permission is added to AndroidManifest.xml, and signature verification is performed. In this way, the system application can use the AudioRecord interface to obtain the audio stream. For system applications, the original data transmission method can still be used to directly transmit video and audio data streams. This data transmission method can reduce the amount of data processing and alleviate the occasional transmission delay that occurs when transmitting audio data via Bluetooth, which is beneficial for maintaining audio-visual synchronization.

[0161] In order to detect the type of screen-casting application, in some embodiments of this application, such as Figure 11 As shown, the display device 200 or mobile terminal 300 can first obtain the package name information of the screen casting application, and then extract the judgment attribute value from the package name information. Based on the judgment attribute value, the type of the current screen casting application is determined. If the judgment attribute value is equal to 0, the screen casting application is marked as a third-party application; if the judgment attribute value is not equal to 0, the screen casting application is marked as a system application.

[0162] For example, the method for distinguishing between system applications and third-party applications is as follows: By obtaining application installation package information, including user-installed applications and system-installed applications, the packageName of their android.content.pm.PackageInfo is filtered, and the attribute value is checked. For example, if ApplicationInfo.FLAG_SYSTEM is not 0, it is considered a system-installed application. Obviously, based on the above judgment result, if the result is not a system-installed application, it belongs to a third-party application.

[0163] After establishing a screen mirroring connection through a screen mirroring application, if the application is a third-party application, a Bluetooth connection also needs to be established between the display device 200 and the mobile terminal 300. For example... Figure 12a , Figure 12b As shown, in some embodiments, the step of connecting the display device 200 to the mobile terminal 300 via the Bluetooth module further includes: sending the address information of the display device 200 to the mobile terminal 300 via an application projection protocol, so that the mobile terminal 300 performs Bluetooth pairing according to the address information; receiving a pairing request sent by the mobile terminal 300 via the Bluetooth transmission protocol; and in response to the pairing request, setting the Bluetooth module to receive mode and establishing a Bluetooth connection with the mobile terminal 300.

[0164] To establish a Bluetooth connection between the display device 200 and the mobile terminal 300, upon detecting that the screen-casting application is a third-party application, the display device 200 can send its current device address information to the mobile terminal 300. For example, the display device 200's Bluetooth MAC address could be 94:17:00:xx:64:8c. This allows the mobile terminal 300 to perform Bluetooth pairing using the received address information and automatically establish a Bluetooth connection with the device at the Bluetooth address 94:17:00:xx:64:8c. Since no Bluetooth connection is currently established between the display device 200 and the mobile terminal 300, the address information of the display device 200 can be sent to the mobile terminal 300 using the protocol corresponding to the screen-casting application.

[0165] In some embodiments, the address information sent by the display device 200 may also include control instructions, such as control instructions for controlling the mobile terminal to perform trust verification, to control the mobile terminal 300 so that after receiving the address information, the mobile terminal 300 sends a pairing request through the Bluetooth transmission protocol, and the display device 200 may respond to the pairing request by setting the Bluetooth module to receive mode and establishing a Bluetooth connection with the mobile terminal 300.

[0166] After receiving the address information, the mobile terminal 300 can also manage the address information locally for direct use when establishing a Bluetooth connection later. For example, the mobile terminal 300 scans the list and matches it with the Bluetooth MAC address received from the Sink (i.e., matches it with the display device 200). If they agree, Bluetooth pairing and connection can proceed. During screen mirroring, video data can be transmitted to the display device 200 via the mirroring connection, and audio data can also be transmitted to the display device 200 via Bluetooth, achieving synchronous transmission of video and audio.

[0167] To facilitate the establishment of a Bluetooth connection between the mobile terminal 300 and the display device 200, some embodiments of this application also provide a Bluetooth pairing method. This pairing method can automatically establish a Bluetooth connection without the user's awareness, ensuring security and eliminating the need for manual confirmation by the user, thus reducing the number of user interactions.

[0168] like Figure 13a , Figure 13b As shown, the Bluetooth pairing and connection method can first set up a server in the receiving end (sink end) display device 200, which is responsible for listening to the Bluetooth information of the receiving end and synchronously receiving the Bluetooth request information of the sending end (source end) mobile terminal 300. Since network communication (such as screen projection connection) has been established between the display device 200 and the mobile terminal 300, the mobile terminal 300 can transmit relevant information to the display device 200 through the protocol layer, and the display device 200 can synchronously transmit device-related information to the mobile terminal 300.

[0169] During the transmission of relevant information through the protocol layer, a device trust tag can be established between the display device 200 and the mobile terminal 300. This tag indicates that the display device 200 and the mobile terminal 300 are secure, enabling seamless Bluetooth pairing and connection. Specifically, after recording the trust tag, if the mobile terminal 300 and the display device 200 establish a Bluetooth connection, trust verification can be performed by detecting the trust tag. After successful trust verification, Bluetooth pairing and direct connection can then proceed.

[0170] For example, after the mobile terminal 300 establishes a screen mirroring connection with the display device 200, the relevant information of the trusted device can be recorded in both the display device 200 and the mobile terminal 300. That is, the mobile terminal 300 records that the current display device 200 is a trusted device, and the display device 200 records that the current mobile terminal 300 is a trusted device. When the mobile terminal 300 initiates a Bluetooth connection request to the display device 200, the display device 200 can determine the trusted device after receiving the Bluetooth connection request. If the information corresponding to the mobile terminal 300 that initiated the Bluetooth connection request is the same as the recorded trusted device information, Bluetooth pairing and connection can be directly performed.

[0171] Similarly, when the display device 200 initiates a Bluetooth connection request to the mobile terminal 300, the mobile terminal 300 can also perform trusted device verification on the display device 200 after receiving the Bluetooth connection request. When the device information corresponding to the display device 200 is the same as one of the recorded trusted device information, it is determined that the current display device 200 has passed the trust verification, and therefore a Bluetooth connection can be established directly without the user having to manually confirm pairing codes and other information.

[0172] As can be seen, in this embodiment, by recording trusted device information, pairing and connection can be directly performed at the system level when the display device 200 and the mobile terminal 300 establish a Bluetooth connection, without the need for manual operation by the user. Under the premise of ensuring security, the number of user interactions is reduced, enabling the display device 200 and the mobile terminal 300 to quickly transmit video and audio data, which facilitates the synchronous transmission of audio and video.

[0173] In some embodiments, in order to record trusted device information in the display device 200 and the mobile terminal 300, trusted device lists can be created in the display device 200 and the mobile terminal 300 respectively, and after the display device 200 and the mobile terminal 300 establish a screen projection connection, the device information of both the display device 200 and the mobile terminal 300 is saved to the trusted device list.

[0174] For example, for display device 200, a trusted device list can be created first. The trusted list can include entries such as device name, device network address, and device Bluetooth MAC address, as shown in the table below:

[0175] Equipment Name Network address Bluetooth MAC address ××'s phone 1 Fe80::6917:ff:xxxx:8984 94:17:00: xx: 64:01 ××'s phone 2 Fe80::6917:ff:xxxx:8985 94:17:00:xx:64:8c …… …… ……

[0176] When the display device 200 establishes a network communication connection with the mobile terminal 300, such as a screen mirroring connection, the display device 200 can extract information such as the device name, network address, and Bluetooth MAC address of the mobile terminal 300 from the device information established during the screen mirroring connection, and add it to the trusted list, thus completing the recording of trusted device information. Since the security of the devices has already been verified when establishing the network communication connection, meaning that the display device 200 and the mobile terminal 300 that can establish a network communication connection already meet the security requirements, subsequent connection methods do not require security verification and can proceed directly to pairing.

[0177] In some embodiments, in order to achieve trust verification between the display device 200 and the mobile terminal 300, after receiving a Bluetooth connection request, the display device 200 and the mobile terminal 300 can match the device information contained in the Bluetooth connection request with the recorded list of trusted devices to determine whether the device currently initiating the Bluetooth connection request is a trusted device.

[0178] For example, when display device 200 receives a Bluetooth connection request from mobile terminal 300, it can extract device information of mobile terminal 300 from the Bluetooth connection request, including device name, device network address, and device Bluetooth MAC address. The extracted device information is then matched against a list of trusted devices. If all or part of the device name, device network address, and device Bluetooth MAC address are recorded in the list of trusted devices, it is determined that the current mobile terminal 300 device information has passed trust verification. Therefore, it can directly respond to the Bluetooth connection request and automatically complete the Bluetooth connection pairing without requiring manual pairing by the user.

[0179] Similarly, for mobile terminal 300, the same method can be used to record the device information of display device 200 in the list of trusted devices. When display device 200 sends a Bluetooth connection request to mobile terminal 300, it can match the device information of display device 200 in the list of trusted devices. When any entry is matched, it can determine that the current display device 200 is a trusted device, thereby automatically completing the Bluetooth connection pairing without the need for the user to manually complete the pairing operation.

[0180] Furthermore, to achieve seamless Bluetooth pairing for users, after the trust verification of the display device 200 or mobile terminal 300 is successful, the mobile terminal 300 or display device 200 can directly complete the pairing connection at the system level. System-level pairing connection means that once the current application has system-level Bluetooth connection permissions, Bluetooth pairing is completed directly at the system level, without requiring application-level processes such as calling display dialog boxes.

[0181] For example, a system service can be built into the display device 200. After the display device 200 establishes a screen mirroring connection with the mobile terminal 300, the display device 200 can listen for changes in the Bluetooth status through this system service and register relevant Bluetooth status broadcast information, such as "android.bluetooth.device.action.FOUND" and "android.bluetooth.device.action.PAIRING_REQUEST". Upon detecting the "FOUND" information, a search of the Bluetooth device list is performed, and a match is made with a trusted device. Upon detecting the pairing request "PAIRING_REQUEST" information, the broadcast information is processed, and system-level pairing is performed on the trusted device. The entire process does not call the application layer to display the pairing interface, so the user does not need to actively click to confirm, and the Bluetooth device pairing connection can be established.

[0182] In some embodiments, during the process of establishing a screen mirroring connection between the display device 200 and the mobile terminal 300, security verification can also be performed on the other device. For example, after receiving a screen mirroring connection request from the mobile terminal 300, the display device 200 can respond to the request by performing security verification on the mobile terminal 300. The specific method of security verification can be as follows: A built-in security verification program extracts information such as the hardware information, network information, and data transmission permissions of the mobile terminal 300, and matches it with data rules stored in a security database. If the information corresponding to the mobile terminal 300 is not in a blacklist, or if the information corresponding to the mobile terminal 300 has not been maliciously tampered with, then the mobile terminal 300 can be determined to have passed the security verification.

[0183] For a mobile terminal 300 that has passed security verification, the display device 200 establishes a network communication connection between itself and the mobile terminal 300 by performing network communication connection pairing, address matching, protocol configuration, and other operations. For example, the display device 200 can configure a screen projection protocol for the mobile terminal 300 that has passed security verification, that is, configure the screen projection protocol according to the screen projection application, and form a screen projection data transmission channel through multiple handshakes between the mobile terminal 300 and the display device 200.

[0184] It should be noted that during the process of establishing a screen mirroring connection, such as Figure 14a , Figure 14bAs shown, the mobile terminal 300 can also detect the type of screen mirroring application. When a third-party application is detected, it actively performs a Bluetooth device search to obtain a list of all connectable Bluetooth devices. Then, based on the Bluetooth MAC address and other information fed back by the display device 200, it matches the Bluetooth device in the list to determine the display device 200 to establish a screen mirroring connection, and directly initiates a Bluetooth connection pairing with the display device 200.

[0185] During Bluetooth pairing, in some usage scenarios, the Bluetooth function switch of the display device 200 or mobile terminal 300 may not be turned on. This will prevent the display device 200 or mobile terminal 300 from using Bluetooth to establish a Bluetooth connection. Therefore, in some embodiments, the controller is further configured to:

[0186] The system detects the on / off status of the Bluetooth module. If the switch is on, it proceeds to obtain address information; if the switch is off, it sends an enable command to the Bluetooth module to activate it. For example, display device 200 and mobile terminal 300 can set the current Bluetooth status of the devices via the Bluetooth interface, such as setting Bluetooth Manager and BluetoothAdapter to be available, so that Bluetooth can be searched for after it is turned on.

[0187] After establishing the screen mirroring and Bluetooth connections, the display device 200 can acquire video and audio data from the mobile terminal 300 in real time, and play the video and audio data through the monitor 275 and speakers, thus achieving screen mirroring. That is... Figure 15 As shown, in some embodiments, the step of connecting the mobile terminal via the Bluetooth module further includes: acquiring an audio data stream from the mobile terminal via the Bluetooth module, and acquiring a video data stream from the mobile terminal via a communicator. After acquiring the audio and video data streams, the audio and video data streams are played synchronously.

[0188] In practical applications, after establishing a screen mirroring connection and a Bluetooth connection, the display device 200 can acquire video data through the screen mirroring data channel and display the video data content on the monitor 275. Simultaneously, via Bluetooth, the display device 200 can act as an audio output device for the mobile terminal 300, thereby outputting sound from the mobile terminal 300 through its audio output device, such as a speaker. Since the display device 200 can directly play sound signals during audio output via Bluetooth, it can still play the content being played on the mobile terminal 300 in real time even when the screen mirroring application lacks audio data acquisition permissions.

[0189] It should be noted that, due to the audio output method of Bluetooth connection, audio-visual desynchronization issues can occur due to Bluetooth data transmission delays. Therefore, during the synchronous playback of audio and video data streams, the mobile terminal can mark the video and audio data streams, for example, by marking the start time of playback. After the display device 200 receives the audio and video data streams, it can first buffer the audio and video data streams, recognize the marks, and start playback according to the marked times to ensure audio-visual synchronization.

[0190] Based on the example of the above screen projection data transmission process, it can be seen that the Bluetooth connection method provided in this application can record mutual trust tags in both the display device 200 and the mobile terminal 300 after establishing a network communication connection. Therefore, when the display device 200 and the mobile terminal 300 establish a Bluetooth connection, they can detect the mutual trust tags and perform mutual trust verification. Upon successful verification, Bluetooth connection pairing is automatically performed to establish a Bluetooth connection. The Bluetooth connection method provided in this application can directly and automatically perform Bluetooth connection pairing for the display device 200 and the mobile terminal 300 after successful trust verification, achieving seamless pairing, reducing the number of manual user interactions, and improving the user experience.

[0191] It should be noted that the above Bluetooth connection method is not limited to screen mirroring. It can still be implemented in scenarios where the mobile terminal 300 and the display device 200 lack a communication protocol. For example, by obtaining relevant information about the mobile terminal 300's Bluetooth device through other means and performing trust authentication, a seamless pairing connection can still be achieved after successful trust verification.

[0192] Based on the above Bluetooth connection method, such as Figure 16 As shown, some embodiments of this application also provide a display device 200, including a display 275, a communicator 220, a Bluetooth module, and a controller 250. The display 275 is used to display a user interface; the communicator 220 is used to establish a network communication connection with a mobile terminal 300; and the Bluetooth module is used to establish a Bluetooth connection with the mobile terminal 300. The controller 250 is configured to perform the following steps: receive a Bluetooth connection request sent by the mobile terminal 300; and in response to the Bluetooth connection request, detect the mutual trust mark of the mobile terminal to perform trust verification on the mobile terminal; if the mobile terminal 300 has a mutual trust mark, automatically perform Bluetooth connection pairing to establish a Bluetooth connection.

[0193] As can be seen, the display device 200 provided in this embodiment includes: a display 275, a communicator 220, a Bluetooth module, and a controller 250. In practical applications, after establishing a network communication connection such as a screen mirroring connection with the mobile terminal 300, the display device 200 can receive a Bluetooth connection request sent by the mobile terminal 300, and perform trust verification on the mobile terminal 300 by detecting the mutual trust mark of the mobile terminal 300. After passing the trust verification, it can automatically perform Bluetooth connection pairing, so as to establish a Bluetooth connection without the user's awareness and reduce the number of user interactions.

[0194] To complete Bluetooth pairing, such as Figure 17 As shown, in some embodiments of this application, a mobile terminal 300 is also provided, including: a display unit 330, a communication unit, a Bluetooth circuit 381, and a processor 380. The display unit 330 is used to present a user interface on the mobile terminal 300; the communication unit is used to establish a network communication connection with the display device 200; and the Bluetooth circuit 381 is used to establish a Bluetooth connection with the display device 200. The processor is also configured to execute the following program steps: sending a Bluetooth connection request to the display device; detecting a mutual trust tag on the display device to perform trust verification on the display device; and if the display device has a mutual trust tag, automatically performing Bluetooth connection pairing to establish a Bluetooth connection.

[0195] As can be seen, the mobile terminal 300 provided in this embodiment can detect the mutual trust mark of the display device 200 when establishing a Bluetooth connection with the display device 200, realize the trust verification of the display device 200, and directly complete the Bluetooth connection pairing after the trust verification is passed, so that the user does not need to perform Bluetooth pairing interaction operation, reducing the number of user interactions and improving the user experience.

[0196] It should be noted that, for the other embodiments described above, different implementation methods can be configured by different functional units or additional functional modules on the basis of the display device 200 and the mobile terminal 300 to form a new screen projection data transmission method, which will not be elaborated here.

[0197] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: monitor; communicator; Bluetooth module; The controller is configured as follows: In response to a screen mirroring connection request sent by the mobile terminal, a screen mirroring connection is established; The device information of the mobile terminal is obtained through the screen casting connection, and the device information of the mobile terminal is added to the first trusted device list, and the screen casting application type is extracted from the screen casting connection request; If the screen mirroring application type is a third-party application, a control command is sent to the mobile terminal. The third-party application includes applications that cannot transmit audio data streams through the data channel of the screen mirroring connection. Receive the Bluetooth connection request sent by the mobile terminal according to the control command; In response to the Bluetooth connection request, the device information of the mobile terminal in the Bluetooth connection request is parsed, and the first trusted device list is invoked; Match the device information of the mobile terminal in the Bluetooth connection request to the first trusted device list; If the first trusted device list includes the device information of the mobile terminal in the Bluetooth connection request, it is determined that the mobile terminal has passed the trust verification, and Bluetooth connection pairing is performed to establish a Bluetooth connection. The video stream data of the mobile terminal is obtained through the data channel of the screen projection connection, and the audio stream data of the mobile terminal is obtained through the data channel of the Bluetooth connection; If the screen mirroring application type is a system application, the video stream data and audio stream data of the mobile terminal are obtained through the data channel of the screen mirroring connection. The system application includes applications that can transmit audio data streams through the data channel of the screen mirroring connection. The video stream data and the audio stream data are played synchronously.

2. The display device according to claim 1, characterized in that, The controller establishes a network communication connection and is further configured as follows: Perform security verification on the mobile terminal; If the mobile terminal passes the security verification, a screen mirroring connection is established.

3. The display device according to claim 1, characterized in that, The controller is further configured to: Detect the on / off status of the Bluetooth module; If the switch is in the ON state, perform the step of receiving the Bluetooth connection request sent by the mobile terminal; If the switch is in the off state, an enable command is sent to the Bluetooth module to enable the Bluetooth module.

4. The display device according to claim 1, characterized in that, In the Bluetooth pairing process, the controller is further configured to: Configure a system service for listening to the Bluetooth connection request; If the system service detects the Bluetooth connection request, it executes a dialog box at the system level to confirm the settings; A confirmation command is sent to the mobile terminal to establish a Bluetooth connection.

5. A mobile terminal, characterized in that, include: Display unit; Communication unit; Bluetooth circuitry; The processor is configured as follows: Send a screen mirroring connection request to the display device so that the display device establishes a screen mirroring connection; The device information of the display device is obtained through the screen mirroring connection, and the device information of the display device is added to the second trusted device list; Upon receiving a control command from the display device, a Bluetooth connection request is sent to the display device according to the control command. The control command is sent by the display device when the screen casting application type extracted from the screen casting connection request is a third-party application. The third-party application includes applications that cannot transmit audio data streams through the data channel of the screen casting connection. Parse the device information of the display device in the Bluetooth connection request, and call the second trusted device list; Match the device information of the displayed device in the Bluetooth connection request to the second trusted device list; If the second trusted device list includes the device information of the display device in the Bluetooth connection request, it is determined that the display device has passed the trust verification, and Bluetooth connection pairing is performed to establish a Bluetooth connection; The video stream data is sent to the display device through the data channel of the screen projection connection, and the audio stream data is sent to the display device through the data channel of the Bluetooth connection, so that the display device can play the video stream data and the audio stream data synchronously; When the display device receives video stream data and audio stream data from the system application type of the screen projection, the display device sends video stream data and audio stream data to the display device through the data channel of the screen projection connection, so that the display device can play the video stream data and the audio stream data synchronously. The system application includes applications that can transmit audio data streams through the data channel of the screen projection connection.

6. A Bluetooth connection method, characterized in that, include: A screen mirroring connection is established between a mobile terminal and a display device. The display device adds the device information of the mobile terminal to a first trusted device list and extracts the screen mirroring application type from the screen mirroring connection request. If the screen mirroring application type is a third-party application, the display device sends a control command to the mobile terminal. The third-party application includes applications that cannot transmit audio data streams through the data channel of the screen mirroring connection. The mobile terminal adds the device information of the display device to a second trusted device list. The mobile terminal sends a Bluetooth connection request to the display device according to the control command; The display device parses the device information of the mobile terminal in the Bluetooth connection request, and calls the first trusted device list to match the device information of the mobile terminal in the Bluetooth connection request in the first trusted device list; The mobile terminal parses the device information of the display device in the Bluetooth connection request and calls the second trusted device list to match the device information of the display device in the Bluetooth connection request in the second trusted device list; if the first trusted device list includes the device information of the mobile terminal in the Bluetooth connection request and the second trusted device list includes the device information of the display device in the Bluetooth connection request, it determines that the display device and the mobile terminal have passed the trust verification, performs Bluetooth connection pairing, and establishes a Bluetooth connection; The display device acquires the video stream data of the mobile terminal through the data channel of the projection connection, and The audio stream data of the mobile terminal is acquired through a data channel connected via Bluetooth; If the screen mirroring application type is a system application, the display device obtains the video stream data and audio stream data of the mobile terminal through the data channel of the screen mirroring connection. The system application includes applications that can transmit audio data streams through the data channel of the screen mirroring connection. The display device simultaneously plays the video stream data and the audio stream data.

Images