Intelligent terminal, augmented reality device, augmented reality system and task processing method thereof

Abstract

The invention provides an intelligent terminal, an augmented reality device, an augmented reality system and a task processing method thereof.The intelligent terminal can be in communication connection with the augmented reality device through a first communication port, and a processor is used for running a first application program in a first working mode; the operation result of the first application program is sent to the augmented reality equipment through the first communication port, so that the purpose of providing computing power support for the augmented reality equipment under the condition that communication connection is established between the terminal equipment and the augmented reality equipment is achieved; besides, the processor can be in communication connection with the external input device through the second communication port, the processor is utilized to run the second application program according to the input operation of the user on the external input device in the second working mode, and the running result of the second application program is sent to the augmented reality device, so that the input mode of the augmented reality device can be enriched, and the user experience can be improved. And diversified use requirements of the user on the augmented reality equipment in different scenes can be met.

Description

Technical Field

[0001] This specification relates to the field of smart hardware technology, and more specifically, to a smart terminal, an extended reality device, an extended reality system, and a task processing method thereof. Background Technology

[0002] With the continuous development of Extended Reality (XR) technology, the functions of extended reality devices such as AR (Augmented Reality) glasses, VR (Virtual Reality) glasses, and MR (Mixed Reality) glasses are constantly being enriched, meeting users' needs for immersive or real-time interaction.

[0003] However, the current usage of these extended reality devices is relatively limited and cannot meet the diverse needs of users. Summary of the Invention

[0004] This specification provides an intelligent terminal, an extended reality device, an extended reality system, and a task processing method thereof, so as to realize the usage of the extended reality device and meet the diverse usage needs of users.

[0005] To achieve the above technical objectives, the embodiments described in this specification provide the following technical solutions:

[0006] In a first aspect, a smart terminal is provided, comprising: a first communication port configured to communicate with an extended reality device; a second communication port configured to communicate with an external input device; at least one processor configured to run a first application in a first operating mode and send the running result of the first application to the extended reality device through the first communication port; and at least one processor further configured to run a second application in a second operating mode based on input operations by a user on the external input device and send the running result of the second application to the extended reality device through the first communication port.

[0007] In conjunction with the first aspect, in some embodiments of the first aspect, the first application includes at least one of audio / video playback software and map software, and the second application is one of office software and game software.

[0008] In conjunction with the first aspect, in some embodiments of the first aspect, at least one processor is configured to: in response to the smart terminal establishing a communication connection with the extended reality device, configure the smart terminal to operate in a first operating mode; and in response to the smart terminal establishing a communication connection with an external input device, configure the smart terminal to switch from the first operating mode to a second operating mode corresponding to the type of the external input device, depending on the type of the external input device.

[0009] In conjunction with the first aspect, in some embodiments of the first aspect, at least one processor is configured to: determine the type of the external input device based on target information before configuring the smart terminal to switch from a first operating mode to a second operating mode corresponding to the type of the external input device; the target information is used to characterize the type of the external input device.

[0010] In conjunction with the first aspect, in some embodiments of the first aspect, the target information includes the connection method between the external input device and the smart terminal or the type identifier of the external input device carried in the communication information between the external input device and the smart terminal, wherein the connection method corresponds to the type of the external input device.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, at least one processor exhibits different performance in different operating modes.

[0012] In conjunction with the first aspect, in some embodiments of the first aspect, the external input device includes a first type of peripheral and a second type of peripheral; wherein the computing power requirement of the working scenario corresponding to the first type of peripheral is greater than the computing power requirement of the working scenario corresponding to the second type of peripheral; the second working mode includes a first sub-mode corresponding to the first type of peripheral and a second sub-mode corresponding to the second type of peripheral, and the performance of at least one processor when the smart terminal is working in the first sub-mode is greater than the performance of at least one processor when it is working in the second sub-mode.

[0013] In conjunction with the first aspect, in some embodiments of the first aspect, at least one processor includes at least one of a general-purpose processor and a dedicated processor; the smart terminal further includes: other modules; the other modules are configured to provide at least one of heat dissipation, touch, Bluetooth, and voice functions; in a first operating mode, the clock frequency of the general-purpose processor is limited to below a predetermined frequency, the other modules are configured to provide touch functions, and the heat dissipation and voice functions provided by the other modules are disabled; in a first sub-mode, the power consumption limit or frequency limit of the dedicated processor is lifted to enable the dedicated processor to operate in a high-performance state, and the power consumption of the general-purpose processor controlling the heat dissipation function in the other modules is a first power consumption; in a second sub-mode, the general-purpose processor is configured to enable the voice and Bluetooth functions provided by the other modules, and the power consumption of the heat dissipation function in the other modules is a second power consumption, wherein the first power consumption is greater than the second power consumption.

[0014] In conjunction with the first aspect, in some embodiments of the first aspect, the dedicated processor includes at least one of a graphics processor, a neural network processor, a tensor processor, a deep learning processor, and a brain processor.

[0015] In conjunction with the first aspect, in some embodiments of the first aspect, in the first operating mode, at least one processor is further configured to execute a touch operation triggered event in response to a touch operation on another module.

[0016] In conjunction with the first aspect, in some embodiments of the first aspect, the smart terminal includes a handheld portion; other modules include a heat dissipation module for providing heat dissipation; the heat dissipation module includes a wind-cooled heat dissipation component, the exhaust direction of the wind-cooled heat dissipation component is toward a predetermined direction, the predetermined direction being away from the handheld portion.

[0017] In conjunction with the first aspect, in some embodiments of the first aspect, the smart terminal further includes: a plurality of function keys; at least one processor is further configured to execute an event corresponding to the function key in response to a triggering operation of the function key, the event corresponding to the function key including: a power switch event, a volume control event, and a working mode switching event.

[0018] In conjunction with the first aspect, in some embodiments of the first aspect, the external input device includes a keyboard, the smart terminal is also provided with a touchpad, and at least one processor is configured to run a second application based on user input operations on the keyboard and touchpad.

[0019] In conjunction with the first aspect, in some embodiments of the first aspect, the smart terminal is detachably connected to an external input device.

[0020] In conjunction with the first aspect, in some embodiments of the first aspect, the second communication port includes a magnetic connector and / or the smart terminal includes at least one card connector, the external input device includes a keyboard and / or a game controller, and the smart terminal is detachably connected to the external input device via at least one card connector and / or magnetic connector.

[0021] In conjunction with the first aspect, in some embodiments of the first aspect, the smart terminal communicates with an external input device wirelessly and / or via a wired connection.

[0022] In conjunction with the first aspect, in some embodiments of the first aspect, the smart terminal communicates with an external input device via a USB Type-C port and / or a Bluetooth module.

[0023] In conjunction with the first aspect, in some embodiments of the first aspect, extended reality devices include augmented reality (AR) devices, virtual reality (VR) devices, or mixed reality (MR) devices.

[0024] In a second aspect, an extended reality system is provided, comprising: an external input device communicatively connected to a handheld smart terminal for receiving user input operations; a smart terminal as described above; and an extended reality device communicatively connected to the smart terminal for displaying the output results of the smart terminal.

[0025] In conjunction with the second aspect, in some embodiments of the second aspect, the external input device is a game controller, the smart terminal is connected to the game controller via a snap-on method, and communicates with the game controller via a USB Type-C port.

[0026] In conjunction with the second aspect, in some embodiments of the second aspect, the external input device is a keyboard, the smart terminal is connected to the keyboard by magnetic attraction and / or snap-fit, and communicates with the external input device via Bluetooth. The smart terminal is also equipped with a touchpad.

[0027] In conjunction with the second aspect, in some embodiments of the second aspect, the first application includes at least one of audio / video playback software and map software, and the second application is one of office software and game software.

[0028] In conjunction with the second aspect, in some embodiments of the second aspect, at least one processor is configured to: in response to the smart terminal establishing a communication connection with the extended reality device, configure the smart terminal to operate in a first operating mode; and in response to the smart terminal establishing a communication connection with an external input device, configure the smart terminal to switch from the first operating mode to a second operating mode corresponding to the type of the external input device, depending on the type of the external input device.

[0029] In conjunction with the second aspect, in some embodiments of the second aspect, at least one processor is configured to: determine the type of the external input device based on target information before configuring the smart terminal to switch from a first operating mode to a second operating mode corresponding to the type of the external input device; the target information is used to characterize the type of the external input device.

[0030] In conjunction with the second aspect, in some embodiments of the second aspect, the target information includes the connection method between the external input device and the smart terminal or the type identifier of the external input device carried in the communication information between the external input device and the smart terminal, wherein the connection method corresponds to the type of the external input device.

[0031] In conjunction with the second aspect, in some implementations of the second aspect, at least one processor exhibits different performance under different operating modes.

[0032] In conjunction with the second aspect, in some embodiments of the second aspect, the external input device includes a first type of peripheral and a second type of peripheral; wherein the computing power requirement of the working scenario corresponding to the first type of peripheral is greater than the computing power requirement of the working scenario corresponding to the second type of peripheral; the second working mode includes a first sub-mode corresponding to the first type of peripheral and a second sub-mode corresponding to the second type of peripheral, and the performance of at least one processor when the smart terminal is working in the first sub-mode is greater than the performance of at least one processor when it is working in the second sub-mode.

[0033] In conjunction with the second aspect, in some embodiments of the second aspect, at least one processor includes at least one of a general-purpose processor and a dedicated processor; the smart terminal further includes: other modules; the other modules are configured to provide at least one of heat dissipation, touch, Bluetooth, and voice functions; in a first operating mode, the clock frequency of the general-purpose processor is limited to below a predetermined frequency, the other modules are configured to provide touch functions, and the heat dissipation and voice functions provided by the other modules are disabled; in a first sub-mode, the power consumption limit or frequency limit of the dedicated processor is lifted to enable the dedicated processor to operate in a high-performance state, and the power consumption of the general-purpose processor controlling the heat dissipation function in the other modules is a first power consumption; in a second sub-mode, the general-purpose processor is configured to enable the voice and Bluetooth functions provided by the other modules, and the power consumption of the heat dissipation function in the other modules is a second power consumption, wherein the first power consumption is greater than the second power consumption.

[0034] In conjunction with the second aspect, in some embodiments of the second aspect, the dedicated processor includes at least one of a graphics processor, a neural network processor, a tensor processor, a deep learning processor, and a brain processor.

[0035] In conjunction with the second aspect, in some embodiments of the second aspect, in the first operating mode, at least one processor is further configured to execute a touch operation triggered event in response to a touch operation on another module.

[0036] In conjunction with the second aspect, in some embodiments of the second aspect, the smart terminal includes a handheld portion; other modules include a heat dissipation module for providing heat dissipation; the heat dissipation module includes a wind-cooled heat dissipation component, the exhaust direction of the wind-cooled heat dissipation component is toward a predetermined direction, the predetermined direction being away from the handheld portion.

[0037] In conjunction with the second aspect, in some embodiments of the second aspect, the smart terminal further includes: a plurality of function keys; at least one processor is further configured to execute an event corresponding to the function key in response to a trigger operation of the function key, the event corresponding to the function key including: a power switch event, a volume control event, and a working mode switching event.

[0038] In conjunction with the second aspect, in some embodiments of the second aspect, the external input device includes a keyboard, the smart terminal is also provided with a touchpad, and at least one processor is configured to run a second application based on user input operations on the keyboard and touchpad.

[0039] In conjunction with the second aspect, in some embodiments of the second aspect, the smart terminal is detachably connected to the external input device.

[0040] In conjunction with the second aspect, in some embodiments of the second aspect, the second communication port includes a magnetic connector and / or the smart terminal also includes at least one card connector, the external input device includes a keyboard and / or a game controller, and the smart terminal is detachably connected to the external input device via at least one card connector and / or magnetic connector.

[0041] In conjunction with the second aspect, in some embodiments of the second aspect, the smart terminal communicates with external input devices wirelessly and / or via wired means.

[0042] In conjunction with the second aspect, in some embodiments of the second aspect, the smart terminal communicates with an external input device via a USB Type-C port and / or a Bluetooth module.

[0043] In conjunction with the second aspect, in some embodiments of the second aspect, extended reality devices include augmented reality (AR) devices, virtual reality (VR) devices, or mixed reality (MR) devices.

[0044] Thirdly, an extended reality device is provided, comprising: a processor, a display device, and a communication port;

[0045] The communication port is configured to communicate with the smart terminal;

[0046] The processor is configured to, in a first operating mode, receive the running results of a first application through a communication port and present the running results of the first application through a display device; and, in a second operating mode, receive the running results of a second application through a communication port and present the running results of the second application through a display device.

[0047] Fourthly, a task processing method for an extended reality system is provided, applicable to smart terminals, including:

[0048] The first application runs in the first working mode, and the running results of the first application are sent to the extended reality device through the first communication port of the smart terminal;

[0049] In the second working mode, the second application runs based on the user's input on an external input device connected to the second communication port of the smart terminal, and sends the running results of the second application to the extended reality device through the first communication port.

[0050] In conjunction with the fifth aspect, in some embodiments of the fifth aspect, the task processing method for the extended reality system further includes:

[0051] In response to the establishment of a communication connection between the smart terminal and the extended reality device, the smart terminal is configured to work in the first working mode;

[0052] In response to the establishment of a communication connection between the smart terminal and the external input device, the smart terminal is configured to switch from a first working mode to a second working mode corresponding to the type of the external input device, based on the type of the external input device.

[0053] Sixthly, a computing device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the task processing method of the extended reality system described above.

[0054] In a seventh aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the task processing method of the extended reality system described above.

[0055] Eighthly, a computer program product or computer program is provided. The computer program product includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and when the processor executes the computer program, it implements the steps of the task processing method of the extended reality system described above. Optionally, the computer program may be stored in a computer-readable storage medium or in the cloud; the processor of the computer device reads the computer program from the readable storage medium or in the cloud.

[0056] As can be seen from the above technical solutions, the intelligent terminal provided in this specification includes a first communication port, a second communication port, and at least one processor. The intelligent terminal can communicate with the extended reality device through the first communication port, run a first application in a first working mode using at least one processor, and send the running results of the first application to the extended reality device through the first communication port, thereby achieving the purpose of providing computing power support to the extended reality device when a communication connection is established between the terminal device and the extended reality device. In addition, it can also communicate with an external input device through the second communication port, run a second application in a second working mode based on the user's input operation on the external input device, and send the running results of the second application to the extended reality device. In this way, the input methods of the extended reality device can be enriched, meeting the diverse usage needs of users for the extended reality device in different scenarios. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this specification. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0058] Figure 1 A schematic diagram illustrating the application scenarios of the smart terminal provided in the embodiments of this specification;

[0059] Figure 2 A schematic diagram of the structure of a smart terminal provided for the embodiments of this specification;

[0060] Figure 3 A schematic diagram illustrating the connection between a smart terminal and an extended reality device provided for embodiments of this specification;

[0061] Figure 4 A schematic diagram illustrating the connection between a smart terminal and an extended reality device provided for embodiments of this specification;

[0062] Figure 5 A schematic diagram illustrating the connection between a smart terminal, an extended reality device, and an external input device, provided for embodiments of this specification.

[0063] Figure 6 A schematic diagram illustrating the connection between a smart terminal, an extended reality device, and an external input device, provided for embodiments of this specification.

[0064] Figure 7 A schematic diagram illustrating the connection between a smart terminal, an extended reality device, and another external input device, provided for embodiments of this specification.

[0065] Figure 8 A schematic diagram illustrating the connection between a smart terminal, an extended reality device, and another external input device, provided for embodiments of this specification.

[0066] Figure 9 A schematic diagram illustrating the connection between a smart terminal, an extended reality device, and another external input device, provided for embodiments of this specification.

[0067] Figure 10 A schematic diagram illustrating a predetermined direction for embodiments of this specification;

[0068] Figure 11 A schematic diagram of the structure of an extended reality system provided for the implementation of this specification;

[0069] Figure 12 A schematic diagram of the structure of an extended reality device provided for the embodiments of this specification;

[0070] Figure 13 A flowchart illustrating a task processing method based on a smart device provided for embodiments of this specification;

[0071] Figure 14 This is a schematic diagram of the structure of a computing device provided for the implementation of this specification.

[0072] Figure label:

[0073] 100 - Smart terminal; 200 - External input device; 300 - AR / VR glasses; 400 - Near-eye virtual screen; 500 - Extended reality system; 600 - Extended reality device; 101 - First communication port; 102 - Second communication port; 103 / 602 - Processor; 104 - AR / VR application; 105 - Function key; 106 - USB Type-C port; 107 - Magnetic connector; 108 - Handheld part; 109 - Predetermined direction; 201 - Game controller; 202 - Keyboard; 601 - Communication port; 603 - Display device. Detailed Implementation

[0074] Unless otherwise defined, the technical or scientific terms used in the embodiments of this specification shall have the ordinary meaning understood by one of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to avoid confusion of constituent elements.

[0075] Unless the context otherwise requires, throughout this specification, "a plurality of" means "at least two," and "including" is interpreted as open-ended or encompassing, that is, "including, but not limited to." In the description of this specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this specification. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example.

[0076] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments in this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0077] Overview

[0078] Taking AR glasses as an example, AR glasses can enhance the perception and operational efficiency of the real environment through virtual information. Users can use AR glasses to meet their immersive interaction needs in different scenarios. For example, users can use AR glasses to achieve AR real-world navigation, AR games, and other needs.

[0079] Currently, the operation of augmented reality devices such as AR glasses is relatively simple, typically requiring users to input information through gestures. This limited usability makes it difficult to meet the needs of multi-tasking or multi-device collaborative interaction scenarios. Furthermore, users often need to work while traveling. While laptops can be carried, their large size makes them inconvenient to carry, and in public places, the user interface is easily visible to others, posing a privacy risk. Similarly, users often clip their phones to game controllers for gaming. While convenient, this interferes with normal phone use, and the small screen size doesn't provide an immersive gaming experience. Finally, when using digital tablets (e.g., drawing tablets and handwriting tablets) for artistic creation, there is a need for an immersive user experience and security in public places.

[0080] To address the aforementioned issues, the inventors discovered that a handheld smart terminal can be provided. This terminal establishes communication connections with both external input devices and extended reality devices. The smart terminal acts as the host, while the extended reality device serves as the screen. User input is received via external input devices, allowing for convenient and quick real-time construction of extended reality systems suitable for different application scenarios. This not only expands the operation methods of extended reality devices but also meets the needs of users in various usage scenarios.

[0081] For example, in an office setting, external input devices can include a keyboard, allowing users to quickly input information. The smart terminal can run office software, and the extended reality device can display the user interface in real time, thus meeting various office needs. Similarly, in a gaming setting, external input devices can include game controllers, allowing users to easily control game characters to perform corresponding actions. The smart terminal can run game software, and the extended reality device can display the game screen in real time, thus meeting gaming needs. Furthermore, when users create artwork (e.g., movie characters or scenery) using a graphics tablet, the smart terminal can run drawing software, and the extended reality device can display the realistic images and lifelike characters drawn by the user in real time, thus meeting the needs of artistic creation.

[0082] Meanwhile, in various scenarios, smart terminals can provide computing power (computation + storage) support for extended reality devices, offloading at least some of the tasks that originally needed to be performed by the extended reality device to themselves, reducing the computing power requirements of the extended reality device. In some cases, the extended reality device can even only have display and imaging functions, offloading all computing tasks to the smart terminal, thereby minimizing the computing power requirements of the extended reality device and laying the foundation for the miniaturization design of extended reality devices.

[0083] Based on the above concept, this specification provides a smart terminal. The smart terminal provided in this specification will be described exemplarily below with reference to the accompanying drawings.

[0084] Scenario Example

[0085] Figure 1 This specification illustrates a feasible application scenario for the smart terminal 100 provided in the embodiments of this specification. In this scenario, the extended reality device can be AR / VR glasses 300, and the external input device 200 can include a keyboard (e.g., a Bluetooth keyboard), a game controller (e.g., a game controller with a back clip), a graphics tablet (e.g., a drawing tablet and a handwriting tablet), etc. The smart terminal 100 can establish a communication connection with the AR / VR glasses 300 through a first communication port. During use, the smart terminal 100 can always maintain a communication connection with the AR / VR glasses 300. The first communication port can include a wireless communication port (e.g., Bluetooth or Zigbee) or a wired communication port. In a wired connection, the first communication port may include a USB Type-C port, through which the smart terminal 100 can establish a communication connection with the AR / VR glasses 300. In a wireless connection, the first communication port may include a wireless communication module such as a Bluetooth module, through which the smart terminal 100 can establish a communication connection with the AR / VR glasses 300. The smart terminal 100 can also establish a communication connection with an external input device through a second communication port, which may also include a wired or wireless communication port. Furthermore, the physical connection between the smart terminal 100 and the external input device can be detachable, meaning that the smart terminal 100 and the AR / VR glasses 300 can operate without an external input device, or the smart terminal 100 can work in conjunction with the external input device 200 while maintaining a communication connection. Applications, referred to as AR / VR applications 104, can run on the smart terminal 100 to meet different user needs. In some scenarios, the smart terminal 100 can also establish a communication connection with the server provided by the service provider of the AR / VR glasses 300 through the network, so that the smart terminal can use the server's powerful computing power to meet the operation requirements of the AR / VR application 104 with high computing power requirements.

[0086] In scenarios such as office work, gaming, movie watching, and drawing, the smart terminal 100 provided in this specification can provide computing power support for extended reality devices (specifically, AR / VR glasses 300 in this scenario). Specifically, the smart terminal 100 can communicate with the extended reality device through the first communication port 101, run a first application using the processor 103 in a first working mode, and send the running results of the first application to the extended reality device through the first communication port 101. This achieves the purpose of providing computing power support to the extended reality device when a communication connection is established between the terminal device 100 and the extended reality device. Furthermore, it can also connect to an external input device 200 through the second communication port 102, run a second application based on user input on the external input device 200 in the second working mode using the processor 103, and send the running results of the second application to the extended reality device. This enriches the input methods of the extended reality device and meets the diverse usage needs of users in different scenarios.

[0087] In some implementations, the extended reality device may only need to fulfill the function of displaying the program execution results, thereby minimizing the hardware configuration required for the extended reality device and meeting the lightweight requirements of the extended reality device.

[0088] Understandable, Figure 1 This specification only illustrates one feasible use case. In other scenarios, the extended reality device could be, for example, mixed reality glasses, and the external input device could be, for example, a game controller or a graphics tablet. This specification does not limit the specific use case and the choice depends on the actual situation.

[0089] Example terminal

[0090] refer to Figure 2 In conjunction with references Figure 1 This specification provides a smart terminal 100, comprising:

[0091] The first communication port 101 is configured to communicate with the extended reality device.

[0092] The second communication port 102 is configured to communicate with the external input device 200.

[0093] At least one processor 103 is configured to run a first application in a first operating mode and send the running results of the first application to an extended reality device via a first communication port 101; at least one processor 103 is also configured to run a second application in a second operating mode based on input operations by a user on an external input device 200 and send the running results of the second application to the extended reality device via the first communication port 101.

[0094] It should be understood that the smart terminal also includes a memory for storing a first application and a second application, as well as computer instructions for implementing task processing methods.

[0095] In one embodiment, the first communication port 101 may include a wired communication port and / or a wireless communication port. Similarly, the second communication port 102 may also include a wired communication port and / or a wireless communication port. The wired communication port includes, but is not limited to, wired ports such as USB Type-C ports and magnetic connectors. The wireless communication port includes, but is not limited to, wireless communication modules such as Bluetooth modules and Zigebee modules. This specification does not limit the specific choice; it depends on the actual situation. For example, when the external input device 200 is a Bluetooth keyboard, the Bluetooth keyboard can communicate with the smart terminal via Bluetooth and can be detachably connected to the smart terminal via magnetic or snap-fit ​​methods.

[0096] Processor 103 can refer to a hardware unit with data processing capabilities. For example, processor 103 may include a general-purpose processor (Central Processing Unit, CPU) and a dedicated processor. The dedicated processor may include at least one of the following: a graphics processing unit (GPU), a tensor processing unit (TPU), a data processing unit (DPU), a neural processing unit (NPU), and a brain processing unit (BPU). The number of processors included in the smart terminal 100 may be one or more (in this specification, "more" refers to two or more), and this specification does not limit this.

[0097] In some implementations, after establishing a connection with the extended reality device, the smart terminal 100 can run a first application in a first operating mode to perform corresponding computing tasks. These computing tasks include, but are not limited to, at least one of image rendering, voice processing, and sensor fusion tasks. (See reference) Figure 3 , Figure 3 This diagram illustrates the connection between the smart terminal 100 and the extended reality device. Figure 3 In this context, the extended reality device may include AR / VR glasses 300, and the first communication port may include a USB Type-C port 106. The AR / VR glasses 300 can establish a communication connection with the smart terminal 100 through the USB Type-C port 106. (Reference) Figure 4 , Figure 4 This diagram illustrates the usage state of the smart terminal 100 in conjunction with the extended reality device. Figure 4 In this system, the smart terminal 100 can be used as a touch-sensitive handheld device. The results of the first application executed by the smart terminal 100 can be fed back to the near-eye virtual screen 400 via AR / VR glasses 300. Figure 3 and Figure 4 In the illustrated scenario, at least one processor 103 can operate in a first operating mode. In some embodiments, in the first operating mode, the smart terminal 100 can perform computational tasks such as image rendering and support extended reality devices to complete tasks such as video playback. Furthermore, in the first operating mode, the smart terminal 100 can also meet the needs of AR real-scene navigation. In this case, the smart terminal 100 can perform tasks such as real-time positioning and multi-sensor fusion (meeting positioning requirements of different accuracy through multiple sensors). That is, in the first operating mode, the first application can include at least one of audio-visual playback software and map software, satisfying needs such as audio-visual playback and navigation.

[0098] refer to Figure 5 and Figure 6 , Figure 5 and Figure 6 This diagram illustrates the connection relationship between the external input device 200, the smart terminal 100, and the extended reality device when the external input device 200 is a game controller 201; (Refer to...) Figures 7-9 , Figures 7-9 This diagram illustrates the connection relationship between the external input device 200, the smart terminal 100, and the extended reality device when the external input device 200 is a keyboard 202.

[0099] In some implementations, reference Figure 5 and Figure 6The smart terminal 100 can be detachably connected to the external input device 200, and the smart terminal 100 can communicate with the external input device 200 in a wired and / or wireless manner. Specifically, the smart terminal 100 can be detachably physically connected to the game controller 201 via a snap-fit ​​connection. The game controller 201 provides a USB Type-C plug (not shown in the figure), which, together with the USB Type-C port on the smart terminal 100, enables communication between the smart terminal 100 and the game controller 201. This USB Type-C plug can be located on one side of the slot of the game controller 201. The accommodating space of the slot of the game controller 201 can be adjusted by pulling or other means to meet the needs of the smart terminal 100 being tightly snapped into the slot and the smart terminal 100 being removed and inserted. Specifically, when it is necessary to insert the smart terminal 100, the game controller 201 can be pulled horizontally to increase the accommodating space of the slot of the game controller 201, making it easier to insert the smart terminal 100. Then, the two ends of the game controller 201 can be pushed in the opposite direction to reduce the accommodating space of the slot of the game controller 201, tightly snapping the smart terminal 100 into place, while simultaneously inserting the USB Type-C plug located on one side of the slot into the USB port of the smart terminal 100. The Type-C port enables communication between the smart terminal 100 and the game controller 201. Figure 5 and Figure 6 In the scenario shown, the AR / VR glasses 300 can establish a communication connection with the port provided by the game controller 201, and then establish a communication connection with the smart terminal 100 plugged into the game controller 201.

[0100] In this scenario, in one implementation, the communication link between the smart terminal 100 and the game controller 201 can operate in a time-division multiplexing manner to meet the need for bidirectional communication between the smart terminal 100 and the game controller 201. Specifically, for a period of time, the communication link transmits data from the game controller 201 to the smart terminal 100, fulfilling the requirement of transmitting user operations on the game controller 201 to the smart terminal 100; for another period of time, the communication link transmits data from the smart terminal 100 to the game controller 201, fulfilling the purpose of returning the results to the AR / VR glasses via the game controller 201. In this scenario, the second application can be game software to meet the user's gaming needs.

[0101] In another implementation, multiple communication links can be established between the smart terminal 100 and the game controller 201. At least one communication link transmits data from the game controller 201 to the smart terminal 100, fulfilling the requirement of transmitting user operations on the game controller 201 to the smart terminal 100; at least another communication link transmits data from the smart terminal 100 to the game controller 201, fulfilling the purpose of returning the running results to the AR / VR glasses via the game controller 201. For example, the smart terminal 100 can be indirectly connected to the AR / VR glasses via the USB Type-C port on the game controller. This specification does not limit the specific implementation of the communication links; the specific implementation depends on the actual situation.

[0102] Figures 7-9 This diagram illustrates the connection relationship between the smart terminal 100 and the keyboard 202 when the external input device 200 includes the keyboard 202. Figures 7-9 In the illustrated embodiment, the second communication port may include a magnetic connector 107. The magnetic connector 107 in the smart terminal 100 can be a pin end, and a corresponding magnetic connector 107 can also be provided in the keyboard 202. The magnetic connector 107 in the keyboard 202 can be a spring-loaded pin end. When the spring-loaded pin end is close to the pin end, the two devices can be tightly connected magnetically. In addition, in some embodiments, the keyboard 202 may also include a Bluetooth module, and the second communication port may also include a Bluetooth module. The keyboard 202 establishes a communication connection with the smart terminal 100 through the Bluetooth module. In this case, the magnetic connector can still serve the purpose of relatively fixing the keyboard 202 and the smart terminal 100. In this scenario, the second application can be office software to meet the user's office needs. In general, the smart terminal 100 and the external input device 200 can be physically connected in a detachable manner. In some embodiments, the smart terminal 100 can be detachably connected to the external input device 200 via at least one snap-fit ​​connector and / or magnetic connector 107. Specifically, the snap-fit ​​connector can be a snap-fit ​​slot or a snap-fit ​​hook. For example, the smart terminal 100 may be provided with one of the snap-fit ​​slots and snap-fit ​​hooks, while the external input device 200 may be provided with the other. The smart terminal 100 and the external input device 200 can establish a communication connection wirelessly and / or via a wired connection. Specifically, the smart terminal 100 can communicate with the external input device 200 via a USB Type-C port 106 and / or a Bluetooth module.

[0103] In summary, the intelligent terminal provided in this embodiment includes a first communication port, a second communication port, and at least one processor. The intelligent terminal can communicate with an extended reality device through the first communication port, run a first application in a first working mode using at least one processor, and send the running results of the first application to the extended reality device through the first communication port, thereby providing computing power support to the extended reality device when a communication connection is established between the terminal device and the extended reality device. In addition, it can also communicate with an external input device through the second communication port, run a second application in a second working mode based on the user's input operations on the external input device, and send the running results of the second application to the extended reality device. In this way, the input methods of the extended reality device can be enriched, meeting the diverse usage needs of users for the extended reality device in different scenarios.

[0104] Understandably, when the smart terminal 100 connects to different types of external input devices 200, its operating scenario may change accordingly. For example, when the smart terminal 100 is connected to a game controller 201, it is likely to operate in a gaming scenario, where the user needs to run game applications. In this case, the smart terminal 100 can operate in high-performance mode to unleash the performance of the processor 103 and meet the high resource consumption requirements of various games, such as real-time image processing. When the smart terminal 100 is connected to a keyboard 202, it is likely to operate in an office scenario, where the user needs to run office applications. In this case, the smart terminal 100 can operate in a relatively low-performance mode. Compared to the gaming scenario, the performance of the processor 103 can be relatively weaker, reducing the power consumption of the smart terminal 100 while meeting the user's office needs. Thus, in one embodiment, at least one processor 103 is configured to: in response to the smart terminal 100 establishing a communication connection with the extended reality device, configure the smart terminal 100 to operate in a first operating mode, and in response to the smart terminal 100 establishing a communication connection with the external input device 200, configure the smart terminal to switch from the first operating mode to a second operating mode corresponding to the type of the external input device 200, depending on the type of the external input device 200.

[0105] By switching the working mode, the performance of at least one processor 103 can be adapted to the usage requirements of different scenarios, avoiding the waste of processor performance. In specific scenarios, the performance of at least one processor 103 can be fully utilized to meet the needs of resource-intensive applications.

[0106] In summary, when at least one processor 103 establishes a communication connection with the extended reality device, the at least one processor 103 can configure the smart terminal 100 to operate in a first operating mode; when at least one processor 103 establishes a communication connection with the external input device 200, the at least one processor 103 can configure the smart terminal 100 to switch from the first operating mode to a second operating mode corresponding to the type of the external input device 200, depending on the type of the external input device 200. In both the first and second operating modes, at least one processor 103 can execute the computational tasks corresponding to the operating mode, providing computing power support for the extended reality device. Optionally, in some implementations, the performance of at least one processor 103 may differ under different operating modes (first operating mode and second operating mode). Thus, automatic switching of operating modes can be achieved according to the different types of external input devices 200 connected, meeting the computing power requirements of the smart terminal 100 when establishing communication connections with different types of external input devices 200. This satisfies the computing power requirements of the extended reality device when it is working in different scenarios. For example, when the extended reality device is in scenarios such as office, gaming, or watching movies, corresponding computing power support can be provided to meet the computing power requirements of the extended reality device in these scenarios.

[0107] As mentioned above, when the smart terminal 100 operates in different operating modes (e.g., the first operating mode, and the second operating mode corresponding to different external input devices), the computing tasks to be performed by at least one processor 103 may differ. For example, when the smart terminal 100 operates in the first operating mode, the user may be watching a movie using an extended reality device. In this case, the computing tasks performed by the smart terminal 100 may include tasks such as graphics rendering. When the smart terminal operates in the second operating mode corresponding to the keyboard 202, the user may be working and has needs such as voice input and making calls. In this case, the computing tasks performed by the smart terminal 100 may include tasks such as voice processing. This specification does not limit this, and it depends on the actual situation.

[0108] The smart terminal 100 and the extended reality device can establish a communication connection through wired or wireless means. For example, in some embodiments, the smart terminal 100 can establish a wireless communication connection with the extended reality device through wireless communication protocols such as Bluetooth and Zigbee; in other embodiments, the smart terminal 100 can establish a wired communication connection with the extended reality device through various types of data cables, which is not limited in this specification.

[0109] Figure 2 This is a schematic diagram of a feasible three-dimensional structure for a smart terminal 100. Figure 2In this embodiment, the first communication port 101 may include a USB Type-C port 106, and the second communication port 102 may include a magnetic connector 107. In some embodiments, in addition to the first communication port 101 and the second communication port 102, it may also include function buttons 105. These function buttons 105 may correspond to different functions. When a function button 105 is triggered, it can trigger the processor 103 to execute a corresponding event. That is, the processor 103 is also configured to execute an event corresponding to the function button 105 in response to the triggering operation of the function button 105. The events corresponding to the function button 105 include one of the following: a power switch event, a volume control event, and a working mode switching event. For example, in some embodiments… Figure 2 The function button 105 shown can be a power button. When the power button is triggered, it can trigger the processor 103 to execute a power switch event, thereby enabling the power on / off operation of the smart terminal 100.

[0110] Still referencing Figure 2 To ensure the mechanical strength and waterproof performance of the smart terminal 100, the casing of the smart terminal 100 can form an accommodating space that can accommodate at least one hardware unit such as a processor 103. The size of the casing can be designed to facilitate one-handed or two-handed operation by the user. For example, in one embodiment, the size of the casing can be 120mm × 60mm × 12mm, thus making the entire smart terminal 100 appear as a flat, handheld form of 120mm × 60mm × 12mm. In addition, the casing can be an aluminum alloy casing, and air ducts for heat dissipation can be provided in the accommodating space. The aluminum alloy casing can be designed to be isolated from the internal air ducts.

[0111] In one implementation, the second working mode may include two sub-modes to meet the working requirements of different types of external input devices 200. Specifically, the external input devices 200 include a first type of peripheral and a second type of peripheral; wherein the computing power requirement of the working scenario corresponding to the first type of peripheral is greater than the computing power requirement of the working scenario corresponding to the second type of peripheral.

[0112] The second working mode includes a first sub-mode corresponding to the first type of peripheral and a second sub-mode corresponding to the second type of peripheral. When the smart terminal 100 is working in the first sub-mode, the performance of the processor 103 is greater than that when it is working in the second sub-mode.

[0113] In some implementations, the first type of peripherals may include peripherals with high processor performance requirements, such as a game controller 201, while the second type of peripherals may include peripherals with relatively low processor performance requirements, such as a keyboard 202. By adaptively switching the working mode of the smart terminal 100, the performance needs of users in different scenarios can be met.

[0114] More specifically, in one embodiment, processor 103 includes at least one of a general-purpose processor and a special-purpose processor; smart terminal 100 also includes: other modules; the other modules are configured to provide at least one of heat dissipation, touch control, Bluetooth and voice functions;

[0115] In the first working mode, the clock speed of the general-purpose processor is limited to below a predetermined frequency. The general-purpose processor is configured with other modules to provide touch functionality, and the heat dissipation and voice functions provided by other modules are turned off. In this mode, natural heat dissipation of the general-purpose processor, etc., is not excluded.

[0116] In the first sub-mode, the power consumption or frequency limit of the dedicated processor is removed to enable the dedicated processor to operate in a high-performance state, and the power consumption of the general-purpose processor to control the heat dissipation function in other modules is set as the first power consumption.

[0117] In the second sub-mode, the general-purpose processor is configured to enable the voice and Bluetooth functions provided by other modules, and control the power consumption of the heat dissipation function in other modules to a second power consumption, where the first power consumption is greater than the second power consumption.

[0118] In this embodiment, when the smart terminal 100 is not connected to the external input device 200, it can work in the first working mode. At this time, it can be assumed that the user may be in a movie-watching scenario, and the performance requirements of the smart terminal 100 are relatively low. At the same time, the requirements for voice function and heat dissipation are also relatively low. The main frequency of the general-purpose processor can be limited to below a predetermined frequency, other modules can be configured to provide touch function, and heat dissipation and voice function can be turned off to meet the performance and function requirements of the user in the movie-watching scenario.

[0119] When the smart terminal 100 connects to the first type of peripheral, it can be assumed that the user may be running applications with high GPU performance requirements, such as games. In this case, the power consumption or frequency limits of the dedicated processor can be lifted, releasing its performance to meet the operational needs of high-performance applications. Simultaneously, since the dedicated processor generates significant heat in high-performance mode, the cooling functions in other modules can be controlled to operate at high power consumption (i.e., the first power consumption), improving high-performance computing power while meeting cooling requirements. In some implementations, to meet the demands of high-speed data transmission, the smart terminal 100 can establish a communication connection with the first type of peripheral via a wired method such as a data cable.

[0120] When the smart terminal 100 connects to the second type of peripheral device, it can be assumed that the user may be running applications with relatively low computing power requirements, such as office software, but may still have functional needs such as voice input and Bluetooth connectivity. Therefore, the voice and Bluetooth functions provided by other modules can be enabled, and the heat dissipation functions in other modules can be controlled to operate at relatively low power consumption (i.e., second power consumption). This ensures the operation of office software while providing various additional functions (voice, Bluetooth), meeting the computing power and functional requirements of office scenarios. In some implementations, the smart terminal 100 can establish a communication connection with the second type of peripheral device via Bluetooth, or it can fix the second type of peripheral device using magnetic or snap-fit ​​methods.

[0121] refer to Figure 4 In some implementations, in the first operating mode, at least one processor 103 is also configured to execute touch operation triggered events in response to touch operations on other modules.

[0122] In the first working mode, in order to facilitate user control, other control methods besides gesture control are provided. The smart terminal 100 can provide touch function for the user. The user can perform operations such as selecting relevant information on the near-eye virtual screen 400 by touching other modules in the smart terminal 100. At least one processor 103 can execute the event triggered by the touch operation (for example, if the touch operation is a touch operation on the play button, the event triggered by the touch operation can be playing a movie. This specification does not exhaustively list the types of events that may be triggered by touch operations) to meet the user's touch needs in this scenario.

[0123] Understandably, other modules can consist of multiple sub-modules, each providing a corresponding function. For example, a touch module can provide touch functionality, a microphone module can provide voice functionality, a Bluetooth module can provide Bluetooth functionality, a heat dissipation module can provide heat dissipation functionality, and so on.

[0124] In order to avoid interference to the user that the heat dissipation module may cause during operation, in one embodiment, the smart terminal 100 includes a handheld part 108; the heat dissipation module includes a wind-cooled heat dissipation component, the exhaust direction of the wind-cooled heat dissipation component is towards a predetermined direction 109, and the predetermined direction 109 is away from the handheld part 108.

[0125] In this embodiment, reference Figure 10 In some scenarios, the smart terminal 100 can be operated by hand. The part that the user usually holds can be referred to as the handheld part 108, for example, in... Figure 10In this context, the lower part of the smart terminal 100 can be referred to as the handheld part 108, which is typically held by the user during operation. To prevent the exhaust gas from the air-cooled heat dissipation component in the heat dissipation module from blowing onto the user's hand and causing interference, the predetermined direction 109 can be set away from the handheld part 108, for example, in... Figure 10 In this configuration, the handheld portion 108 is located below the smart terminal 100. Therefore, the predetermined direction 109 can be moved away from the handheld portion 108. This allows the user to operate the smart terminal 100 while holding it, and the exhaust gas from the air-cooled heat dissipation component is discharged from the top of the smart terminal 100, preventing the exhaust gas from interfering with the user's handheld operation and improving the user experience. In one embodiment, the heat dissipation module of the smart terminal 100 may include an active air-cooled heat dissipation component, such as a fan, whose speed can be dynamically adjusted according to the system load. For example, in the second mode, if the smart terminal is connected to a game controller for gaming, indicating a high system load, the fan speed can be increased. Conversely, in the second mode, if the smart terminal is connected to a keyboard for office work, indicating a lower system load, the fan speed can be decreased.

[0126] In one embodiment, a feasible method is provided for a smart terminal 100 to automatically determine the type of an external input device 200, specifically, at least one processor 103 is configured to:

[0127] Before configuring the smart terminal 100 to switch from the first working mode to the second working mode corresponding to the type of the external input device 200 according to the type of the external input device 200, the type of the external input device 200 connected to the second communication port 102 is determined according to the target information; the target information is used to characterize the type of the external input device 200.

[0128] The target information can be information representing the type of the external input device 200 acquired by the smart terminal 100 itself, or information representing the type of the external input device 200 sent by the external input device 200 during communication with the smart terminal 100. This specification does not limit this. The smart terminal 100 can autonomously determine the type of the external input device 200 based on the above target information, thereby realizing automatic switching of working modes without manual operation by the user, which is beneficial to improving the user experience.

[0129] One embodiment of this specification provides several feasible representations of target information. Specifically, the target information may include the connection method between the target external input device 200 and the second communication port 102 or the type identifier of the external input device 200 carried in the communication information between the target external input device 200 and at least one processor 103; the target external input device 200 includes an external input device 200 connected to the second communication port 102; the connection method corresponds to the type of the external input device 200.

[0130] In this embodiment, the second communication port 102 can include various types. For example, the second communication port 102 can include at least one of wired communication components such as a magnetic connector and wireless communication modules such as a Bluetooth module. The magnetic connector can be a magnetic Pogo Pin connector, which combines magnetic attraction and spring-loaded pin technology. During operation, the magnetic connector can use its magnetic attraction function to automatically align and tightly contact the contact points of the external input device 200 to achieve a communication connection.

[0131] Different types of second communication ports 102 may connect to external input devices 200 in different ways. For example, external input devices 200 such as game controllers 201 can connect to smart terminals 100 via USB Type-C port 106; external input devices 200 such as keyboards 202 can connect to smart terminals 100 via magnetic connectors; external input devices 200 such as keyboards 202 can also connect to smart terminals 100 wirelessly via Bluetooth modules. Depending on the connection method, at least one processor 103 can determine the type of the connected external input device 200, thereby achieving the purpose of automatically identifying the type of external input device 200.

[0132] In one embodiment, the first communication port 101 may include a plurality of physical communication interfaces configured to connect to an extended reality device or other devices, including a power supply device.

[0133] The physical communication interfaces include, but are not limited to, a USB Type-C (Universal Serial Bus Type-C) port 106. Multiple physical communication interfaces can meet the connection requirements of extended reality devices or other devices. For example, in one embodiment, the smart terminal 100 may include three USB Type-C ports 106. One USB Type-C port 106 can be used to connect an extended reality device, another USB Type-C port 106 can be used to connect a power supply device, and the third USB Type-C port 106 can serve as a redundant interface to improve the scalability of the smart terminal 100.

[0134] In one embodiment, the second communication port 102 includes a magnetic connector; the smart terminal 100 also includes at least one card connector, and the external input device 200 includes a keyboard 202 and / or a game controller 201. The smart terminal 100 is detachably connected to the external input device through at least one card connector and / or magnetic connector 107, and communicates with the external input device through a USB Type-C port and / or a Bluetooth module.

[0135] In this embodiment, the smart terminal 100 can also connect to external input devices 200 via snap-fit ​​connectors and / or magnetic connectors 107 located on the back / side of the device. These external input devices 200 may include a keyboard 202 and a game controller 201. Connection methods include magnetic contact points with pin fixing, elastic clips, etc. Thus, in a gaming scenario, the game controller 201 can be snapped into place via snap-fit ​​connectors to physically fix the game controller 201 to the smart terminal 100, and a wired connection can be established between the game controller 201 and the smart terminal 100 via a USB Type-C port 106 or similar means. In an office setting, the keyboard 202 can be connected via the magnetic connector 107 combined with a Bluetooth module.

[0136] In one embodiment, when the external input device 200 includes a keyboard 202, the smart terminal is also equipped with a touchpad, and the processor is configured to run a second application based on the user's input operations on the keyboard and touchpad. At this time, the smart terminal 100 can provide touch functionality and be used as a touchpad, thus meeting the convenient operation needs of touch, typing and other functions in office scenarios.

[0137] In this embodiment, the smart terminal 100 serves as a comfortable handheld computing unit for an extended reality device. Its size is similar to a small smartphone, and it may not have a display function but can provide touch functionality on the front. The smart terminal 100 can meet the needs of three main application scenarios: movie watching, gaming, and office work. In the movie watching scenario, the user can hold the smart terminal 100 and connect it to the extended reality device. In the gaming scenario, the user can use the game controller 201 attached to the clip, and a wired connection can be established between the game controller 201 and the smart terminal 100 via a data cable. In the office work scenario, the smart terminal 100 can be magnetically or clipped onto a matching Bluetooth keyboard, establishing a Bluetooth connection. In this case, the smart terminal 100 can also be used as a touchpad for the keyboard 202, allowing the smart terminal 100 to switch between various usage modes in different scenarios.

[0138] To ensure the stable operation of the smart terminal 100, the smart terminal 100 may have an air outlet for an active cooling fan. To facilitate the control of the smart terminal 100, the side wall of the smart terminal may provide multiple physical function buttons to meet the needs of powering on / off, adjusting volume, and switching manual modes.

[0139] Exemplary System

[0140] Accordingly, refer to Figure 11 This specification also provides an extended reality system 500, comprising: an external input device 200, communicatively connected to a smart terminal 100, for receiving user input operations; a smart terminal 100 as described in any of the above embodiments; and an extended reality device, communicatively connected to the smart terminal 100, for displaying the output results of the smart terminal 100.

[0141] exist Figure 11 In this embodiment, the augmented reality device may include AR / VR glasses 300. In other embodiments, the augmented reality device may also include MR glasses. This specification does not limit this, and it depends on the actual situation.

[0142] When at least one processor 103 establishes a communication connection with the extended reality device, the at least one processor 103 can configure the smart terminal 100 to operate in a first working mode. When at least one processor 103 establishes a communication connection with an external input device 200, the at least one processor 103 can configure the smart terminal 100 to switch from the first working mode to a second working mode corresponding to the type of the external input device 200, based on the type of the external input device 200. In both the first and second working modes, at least one processor 103 can execute the computing tasks in the corresponding working mode, providing computing power support for the extended reality device. In some embodiments, optionally, the performance of at least one processor 103 can be different in different working modes (first working mode and second working mode). In this way, automatic working mode switching can be achieved according to the different types of external input devices 200 connected, meeting the computing power requirements of the smart terminal 100 when establishing communication connections with different types of external input devices 200, thereby meeting the computing power requirements of the extended reality device when operating in different scenarios. For example, when the extended reality device is in scenarios such as office work, gaming, or watching movies, corresponding computing power support can be provided to meet the computing power requirements of the extended reality device in these scenarios.

[0143] In one implementation, reference Figure 5 and Figure 6 The external input device can be a game controller 201. The smart terminal 100 is connected to the game controller 201 via a snap-on method and communicates with the game controller via the USB Type-C port 106.

[0144] Figure 5 and Figure 6 This diagram illustrates the connection relationship between the external input device 200, the smart terminal 100, and the extended reality device when the external input device 200 is a game controller 201; (Refer to...) Figures 7-9 , Figures 7-9 This diagram illustrates the connection relationship between the external input device 200, the smart terminal 100, and the extended reality device when the external input device 200 is a keyboard 202. In some embodiments, reference is made to... Figure 5 and Figure 6 The smart terminal 100 can be detachably connected to the external input device 200, and the smart terminal 100 can communicate with the external input device 200 in a wired and / or wireless manner. Specifically, the smart terminal 100 can be detachably physically connected to the game controller 201 via a snap-fit ​​connection. The game controller 201 provides a USB Type-C plug (not shown in the figure), which, together with the USB Type-C port on the smart terminal 100, enables communication between the smart terminal 100 and the game controller 201. This USB Type-C plug can be located on one side of the slot of the game controller 201. The accommodating space of the slot of the game controller 201 can be adjusted by pulling or other means to meet the needs of the smart terminal 100 being tightly snapped into the slot and the smart terminal 100 being removed and inserted. Specifically, when it is necessary to insert the smart terminal 100, the game controller 201 can be pulled horizontally to increase the accommodating space of the slot of the game controller 201, making it easier to insert the smart terminal 100. Then, the two ends of the game controller 201 can be pushed in the opposite direction to reduce the accommodating space of the slot of the game controller 201, tightly snapping the smart terminal 100 into place, while simultaneously inserting the USB Type-C plug located on one side of the slot into the USB port of the smart terminal 100. The Type-C port enables communication between the smart terminal 100 and the game controller 201. Figure 5 and Figure 6 In the scenario shown, the AR / VR glasses 300 can establish a communication connection with the port provided by the game controller 201, and then establish a communication connection with the smart terminal 100 plugged into the game controller 201.

[0145] In this scenario, in one implementation, the communication link between the smart terminal 100 and the game controller 201 can operate in a time-division multiplexing manner to meet the need for bidirectional communication between the smart terminal 100 and the game controller 201. Specifically, for a period of time, the communication link transmits data from the game controller 201 to the smart terminal 100, fulfilling the requirement of transmitting user operations on the game controller 201 to the smart terminal 100; for another period of time, the communication link transmits data from the smart terminal 100 to the game controller 201, fulfilling the purpose of returning the results to the AR / VR glasses via the game controller 201. In this scenario, the second application can be game software to meet the user's gaming needs.

[0146] In another implementation, multiple communication links can be established between the smart terminal 100 and the game controller 201. At least one communication link transmits data from the game controller 201 to the smart terminal 100, fulfilling the requirement of transmitting user operations on the game controller 201 to the smart terminal 100; at least another communication link transmits data from the smart terminal 100 to the game controller 201, fulfilling the purpose of returning the running results to the AR / VR glasses via the game controller 201. This specification does not limit the specific implementation of the communication links; the specific implementation depends on the actual situation.

[0147] In one implementation, reference Figures 7-9 The external input device is a keyboard. The smart terminal is connected to the keyboard by magnetic and / or snap-on means and communicates with the external input device via Bluetooth. The smart terminal is also equipped with a touchpad.

[0148] Figures 7-9 This diagram illustrates the connection relationship between the smart terminal 100 and the keyboard 202 when the external input device 200 includes the keyboard 202. Figures 7-9 In the illustrated embodiment, the second communication port may include a magnetic connector 107. The magnetic connector 107 in the smart terminal 100 can be a pin end, and a corresponding magnetic connector 107 can also be provided in the keyboard 202. The magnetic connector in the keyboard 202 can be a spring-loaded pin end. When the spring-loaded pin end is close to the pin end, the two devices can be tightly connected magnetically. In addition, in some embodiments, the keyboard 202 may also include a Bluetooth module, and the second communication port may also include a Bluetooth module. The keyboard 202 establishes a communication connection with the smart terminal 100 through the Bluetooth module. In this case, the magnetic connector can still serve the purpose of relatively fixing the keyboard 202 and the smart terminal 100. In this scenario, the second application can be office software to meet the user's office needs.

[0149] In one embodiment, the smart terminal 100 includes a first connector, and the keyboard 202 includes a second connector. The first and second connectors can be magnetic connectors or snap-fit ​​connectors. The smart terminal 100 is detachably connected to the keyboard 202 via the first and second connectors. The first connector is located on the long side of the smart terminal 100, and the second connector is located on the right side of the keyboard 202. The length of the long side of the smart terminal 100 is the same as the length of the short side of the keyboard 202. This allows the keyboard 202 and the smart terminal 100 to form a full keyboard structure, thus visually concealing the smart terminal 100 from the user and improving the tidiness of the office environment. Furthermore, using a touchpad on the surface of the smart terminal 100 to function as a mouse and placing the smart terminal 100 on the right side of the keyboard conforms to the user's mouse-using habits. Of course, it should be understood that to meet the needs of users accustomed to using a left-handed mouse, the second connector can also be located on the left side of the keyboard.

[0150] In another embodiment, the smart terminal 100 includes a first connector, and the keyboard 202 includes a second connector. The first and second connectors can be magnetic connectors or snap-fit ​​connectors. The smart terminal 100 is detachably connected to the keyboard 202 via the first and second connectors. The first connector is located on the long side of the smart terminal 100, and the second connector is located on the side of the keyboard 202 closer to the user. This allows the keyboard 202 and the smart terminal 100 to visually resemble a laptop keyboard, satisfying the needs of users accustomed to using laptops for work while simultaneously concealing the smart terminal 100's presence from the user through the visual effect of a laptop keyboard, thus improving the tidiness of the office environment.

[0151] In summary, the smart terminal provided in this specification includes a first communication port, a second communication port, and at least one processor. The smart terminal can communicate with an extended reality device through the first communication port, run a first application in a first working mode using the at least one processor, and send the running results of the first application to the extended reality device through the first communication port, thereby providing computing power support to the extended reality device when a communication connection is established between the terminal device and the extended reality device. In addition, it can also communicate with an external input device through the second communication port, run a second application in a second working mode based on the user's input operations on the external input device, and send the running results of the second application to the extended reality device. In this way, the input methods of the extended reality device can be enriched, meeting the diverse usage needs of users for the extended reality device in different scenarios.

[0152] Exemplary device

[0153] refer to Figure 12The embodiments of this specification also provide an extended reality device 600, including: a processor 602, a display device 603 and a communication port 601;

[0154] Communication port 601 is configured to communicate with smart terminal 100;

[0155] The processor 602 is configured to receive the running results of a first application through the communication port 601 and present the running results of the first application through the display device 603 in a first operating mode; and to receive the running results of a second application through the communication port 601 and present the running results of the second application through the display device 603 in a second operating mode.

[0156] The communication port 601 of the extended reality device 600 can be a wired communication port 601 or a wireless communication port 601. In some embodiments, the communication port 601 of the extended reality device 600 is used to connect to the smart terminal 100 described in any of the above embodiments.

[0157] The processor 602 of the extended reality device 600 can be used to receive the execution results of corresponding programs in different working modes and present the corresponding execution results through the display device 603. It is understood that the display requirements of the extended reality device 600 may differ in different working modes. For example, in game mode (i.e., the mode in which the user uses the extended reality device 600 to play games), the extended reality device 600 needs to minimize the delay between movement and display to avoid motion sickness caused by image ghosting; while in office mode (i.e., the mode in which the user uses the extended reality device 600 to do office work), it needs to improve the sharpness and contrast of text in the image to adapt to document reading, spreadsheet editing, and other needs. Therefore, in some embodiments, the extended reality device 600 may have corresponding working modes in different working modes. The data processing focus of the processor 602 of the extended reality device 600 may differ in different working modes, and / or the performance of the processor 602 of the extended reality device 600 may also differ. For example, in game mode, the data processing focus of the extended reality device 600 is the real-time display of images, which places higher demands on the performance of the processor 602, allowing the processor 602 to operate in high-performance mode; while in office mode, the data processing focus of the extended reality device 600 is the processing and display of text, which places relatively lower demands on the performance of the processor 602, allowing the processor 602 to operate in low-power mode.

[0158] Exemplary methods

[0159] Accordingly, this specification also provides a task processing method based on an extended reality system, applicable to smart terminals, for reference. Figure 13 and combined Figures 1 to 11 The task processing methods for extended reality systems include:

[0160] S1201: Run the first application in the first working mode and send the running results of the first application to the extended reality device through the first communication port of the smart terminal;

[0161] S1202: In the second working mode, the second application is run according to the user's input operation on an external input device connected to the second communication port of the smart terminal, and the running result of the second application is sent to the extended reality device through the first communication port.

[0162] In some implementations, the task processing method of the extended reality system further includes:

[0163] In response to the establishment of a communication connection between the smart terminal and the extended reality device, the smart terminal is configured to operate in a first working mode;

[0164] In response to the establishment of a communication connection between the smart terminal and the external input device, the smart terminal is configured to switch from the first working mode to the second working mode corresponding to the type of the external input device, based on the type of the external input device.

[0165] For specific limitations regarding the task processing method based on the smart terminal 100, please refer to the limitations regarding the smart terminal 100 above, which will not be repeated here. For further limitations regarding the smart terminal 100 and the interaction methods between the smart terminal 100 and other devices, please refer to the description of the smart terminal 100 in the exemplary devices section above, which will not be repeated here.

[0166] Exemplary computing device

[0167] Another embodiment of this specification also proposes a computing device, see [link to documentation]. Figure 14 As shown, an exemplary embodiment of this specification also provides a computing device, including: a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform steps in the task processing method of the extended reality system according to various embodiments of this specification described above.

[0168] The internal structure of the computing device can be as follows: Figure 14As shown, the computing device includes a processor, memory, network interface, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it follows the steps of the task processing method of the extended reality system according to various embodiments of this specification, as described in the above embodiments.

[0169] The processor may include the main processor, as well as baseband chips, modems, etc.

[0170] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0171] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0172] Input devices may include devices that receive data and information input by the user, such as keyboards, mice, cameras, scanners, light pens, voice input devices, touch screens, pedometers, or gravity sensors.

[0173] Output devices may include devices that allow information to be output to the user, such as displays, printers, speakers, etc.

[0174] The communication interface may include any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0175] The computing device may also include a display component and a voice component. The display component may be an LCD screen or an e-ink screen. The input device of the computing device may be a touch layer covering the display component, or buttons, trackballs or touchpads set on the casing of the computing device, or external keyboards, touchpads or mice, etc.

[0176] Those skilled in the art will understand that Figure 14 The structures shown are merely block diagrams of some structures related to the solutions in this specification and do not constitute a limitation on the computing devices on which the solutions in this specification are applied. Specific computing devices may include more or fewer components than those shown in the figures, or combine certain components, or have different component arrangements.

[0177] Exemplary computer program products and storage media

[0178] In addition to the methods and devices described above, the task processing method of the extended reality system provided in the embodiments of this specification can also be a computer program product, which includes computer program instructions that, when executed by a processor, cause the processor to perform the steps in the task processing method of the extended reality system according to various embodiments of this specification as described in the "Exemplary Methods" section above.

[0179] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0180] The computer program product described herein can be written in any combination of one or more programming languages ​​to perform the operations described herein. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0181] Furthermore, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor of the steps in the task processing method of the extended reality system according to various embodiments of this specification as described in the "Exemplary Methods" section above.

[0182] Those skilled in the art will understand that all or part of the processes in the above-described embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this specification can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0183] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0184] The embodiments described above are merely illustrative of several implementations of this specification, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the solutions provided in this specification. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this specification, and these all fall within the scope of protection of this specification. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A smart terminal, characterized by, The application comprises: a first communication port configured to be communicatively connected with an extended reality device; a second communication port configured to be communicatively connected with an external input device; at least one processor configured to run a first application in a first working mode and send a running result of the first application to the extended reality device through the first communication port; the at least one processor is further configured to run a second application according to a user's input operation on the external input device in a second working mode and send a running result of the second application to the extended reality device through the first communication port.

2. The intelligent terminal of claim 1, wherein, The first application comprises at least one of audio and video playing software and map software, and the second application is at least one of office software, game software and drawing software.

3. The intelligent terminal of claim 1, wherein, The at least one processor is configured to configure the smart terminal to work in the first working mode in response to the smart terminal establishing a communication connection with the extended reality device, and to switch the smart terminal from the first working mode to the second working mode corresponding to the type of the external input device according to the type of the external input device in response to the smart terminal establishing a communication connection with the external input device.

4. The intelligent terminal of claim 3, wherein, The at least one processor is configured to: determine the type of the external input device according to target information before switching the smart terminal from the first working mode to the second working mode corresponding to the type of the external input device according to the type of the external input device; the target information is used to represent the type of the external input device.

5. The intelligent terminal of claim 4, wherein, The target information comprises a type identifier of the external input device carried in communication information of the external input device and the smart terminal or a connection mode of the external input device and the smart terminal, and the connection mode corresponds to the type of the external input device.

6. The intelligent terminal of claim 1, wherein, The performance of the at least one processor is different in different working modes.

7. The intelligent terminal of claim 6, wherein, The external input device comprises a first type of peripheral device and a second type of peripheral device; wherein the computing power requirement of the working scene corresponding to the first type of peripheral device is greater than the computing power requirement of the working scene corresponding to the second type of peripheral device; The second working mode comprises a first sub-mode corresponding to the first type of peripheral device and a second sub-mode corresponding to the second type of peripheral device, and the performance of the at least one processor when the smart terminal works in the first sub-mode is greater than the performance of the at least one processor when the smart terminal works in the second sub-mode.

8. The intelligent terminal of claim 6, wherein, The at least one processor comprises at least one of a general-purpose processor and a special-purpose processor; the smart terminal further comprises other modules; the other modules are configured to provide at least one of heat dissipation, touch, Bluetooth and voice functions; In the first working mode, the frequency of the general-purpose processor is limited to below a predetermined frequency, the general-purpose processor configures the other modules to provide the touch function, and the general-purpose processor closes the heat dissipation function and the voice function provided by the other modules. In the first sub-mode, the power consumption limit or frequency limit of the special processor is released to make the special processor work in a high-performance running state, and the general processor is configured to control the power consumption of the heat dissipation function in the other modules to be a first power consumption; In the second sub-mode, the general processor is configured to start the voice function and the Bluetooth function provided by the other modules, and control the power consumption of the heat dissipation function in the other modules to be a second power consumption, the first power consumption being greater than the second power consumption.

9. The intelligent terminal of claim 8, wherein, The special processor includes at least one of a graphics processor, a neural network processor, a tensor processor, a deep learning processor, and a brain processor.

10. The intelligent terminal of claim 8, wherein, In the first working mode, the at least one processor is further configured to execute an event triggered by a touch operation on the other modules in response to the touch operation.

11. The intelligent terminal of claim 1, wherein, The smart terminal includes a handheld part; The smart terminal further includes a heat dissipation module for providing a heat dissipation function; The heat dissipation module includes an air-cooled heat dissipation assembly, and an exhaust direction of the air-cooled heat dissipation assembly is towards a predetermined direction away from the handheld part.

12. The intelligent terminal of claim 1, wherein, The smart terminal further includes a plurality of function keys. The at least one processor is further configured to execute an event corresponding to the function key in response to a triggering operation of the function key, the event corresponding to the function key including one of a power switch event, a volume control event, and a working mode switching event.

13. The intelligent terminal of claim 1, wherein, The external input device includes a keyboard, and the smart terminal is further provided with a touch panel, and the at least one processor is configured to run the first application according to input operations of the user on the touch panel in the first mode, and run the second application according to input operations of the user on the keyboard and the touch panel in the second mode.

14. The intelligent terminal according to any one of claims 1 to 13, characterized by The smart terminal is detachably connected with the external input device.

15. The intelligent terminal of claim 14, wherein, The second communication port includes a magnetic attraction connector, and / or the smart terminal includes at least one clamping piece, the external input device includes a keyboard and / or a gamepad, and the smart terminal is detachably connected with the external input device through the at least one clamping piece and / or the magnetic attraction connector.

16. The intelligent terminal according to any one of claims 1 to 13, wherein The smart terminal is wirelessly and / or wiredly connected in communication with the external input device.

17. The intelligent terminal of claim 16, wherein, The smart terminal is connected in communication with the external input device through a USB Type-C port and / or a Bluetooth module.

18. The intelligent terminal according to any one of claims 1 to 13, wherein The extended reality device includes an augmented reality AR device, a virtual reality VR device, or a mixed reality MR device.

19. An extended reality system, comprising: The smart terminal includes: An external input device connected in communication with the smart terminal, configured to receive input operations of a user; The smart terminal as claimed in any one of claims 1 to 18; An extended reality device connected in communication with the smart terminal, configured to display output results of the smart terminal.

20. The extended reality system of claim 19, the external input device being a gamepad, the smart terminal being connected to the gamepad in a clamping manner and connected in communication with the gamepad through a USB Type-C port.

21. The extended reality system of claim 19, wherein the external input device is a keyboard, the smart terminal is connected to the keyboard by magnetic attraction and / or clamping, and the smart terminal is communicatively connected to the external input device by Bluetooth, and the smart terminal is further provided with a touchpad.

22. The extended reality system of claim 21, wherein, The smart terminal comprises a first connector, and the keyboard comprises a second connector, the first connector and the second connector being magnetic connectors or clamping members, the smart terminal being detachably connected to the keyboard by the first connector and the second connector, and the first connector being arranged on a long side of the smart terminal. The second connector is arranged on a right side of the keyboard, or the second connector is arranged on a side of the keyboard close to a user.

23. The extended reality system of claim 22, wherein, In the case where the second connector is arranged on the right side of the keyboard, the length of the long side of the smart terminal is the same as the length of a short side of the keyboard.

24. An extended reality device, comprising: It comprises: a processor, a display device, and a communication port; the communication port is configured to be communicatively connected to a smart terminal; the processor is configured to, in a first working mode, receive a running result of a first application program through the communication port, and present the running result of the first application program through the display device; and, in a second working mode, receive a running result of a second application program through the communication port, and present the running result of the second application program through the display device. 25.A task processing method of an extended reality system, applicable to a smart terminal, and characterized in that, It comprises: running a first application program in a first working mode, and sending a running result of the first application program to an extended reality device through a first communication port of the smart terminal; running a second application program according to a user's input operation on an external input device connected to a second communication port of the smart terminal in a second working mode, and sending a running result of the second application program to the extended reality device through the first communication port.

26. The task processing method of claim 25, wherein, It further comprises: in response to the smart terminal establishing a communication connection with the extended reality device, configuring the smart terminal to work in the first working mode; in response to the smart terminal establishing a communication connection with the external input device, configuring the smart terminal to switch from the first working mode to the second working mode corresponding to the type of the external input device according to the type of the external input device.

27. A non-volatile storage medium, comprising: A computer program is stored thereon, and the computer program is executed by a processor to implement the steps in the task processing method of the extended reality system according to claim 25 or 26.

Images