Intelligent terminal, handheld game machine, virtual system and spatial positioning method of intelligent terminal

Abstract

The application provides a smart terminal, a handheld game console, a virtual system and a spatial positioning method of the smart terminal, and solves the technical problem that a mobile terminal cannot experience the functions of virtual and handheld game console products and cannot be accurately positioned in the prior art. The smart terminal provided by the application can be detachably connected with a virtual head-mounted device, and a main control chip in the smart terminal switches the function of the smart terminal from virtual head-mounted device function to handheld game console function; when the smart terminal is installed on the virtual head-mounted device, the main control chip in the smart terminal switches the function of the smart terminal from handheld game console function to virtual head-mounted device function, so that a user can experience the functions of virtual system and handheld game console products through one smart terminal. The main control chip can also determine the spatial positioning information of the smart terminal according to the environmental information of the smart terminal captured by a first camera and the IMU data of the smart terminal detected by a first inertial sensor, so that accurate positioning of the smart terminal is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile devices, in particular to a smart terminal, a handheld game console, a virtual system and a spatial positioning method of the smart terminal. BACKGROUND

[0002] At present, a new type of virtual reality (VR) head-mounted device composed of a VR box and a mobile terminal has appeared in the market. The performance of the new type of VR head-mounted device is mainly determined by the performance of the mobile terminal, and the performance is far inferior to that of a traditional computer-VR head-mounted device or a VR all-in-one machine. The display of the computer-VR head-mounted device or the VR all-in-one machine is generally inseparable from the VR virtual head-mounted device, and can only be used in conjunction, so the convenience is limited.

[0003] At the same time, handheld devices for game consoles, such as handheld game consoles, currently have products that include a main machine and a handle, and input instructions are realized through the handle to experience games. At this time, the handheld game console does not have the function of experiencing virtual immersion.

[0004] Therefore, it is necessary to provide a new type of combined smart terminal device that has the separable feature of a virtual box and can also reflect the function of a handheld game console, so as to meet the purpose of users experiencing the functions of two different products by purchasing only one product.

[0005] In addition, in the prior art, when the mobile terminal changes the working mode, for example, switches from the function of a handheld game console to the function of a virtual head-mounted device, the mobile terminal and the control handle cannot be accurately repositioned. SUMMARY

[0006] Therefore, the present application provides a smart terminal, a handheld game console, a virtual system and a spatial positioning method of the smart terminal, which solves the technical problem that in the prior art, a mobile terminal cannot experience the functions of two products, i.e., a virtual and a handheld game console, and when the mobile terminal changes the working mode, the space of the mobile terminal cannot be accurately positioned.

[0007] According to one aspect of the present application, the present application provides a smart terminal, comprising: a smart terminal body, the smart terminal body being detachably installed on a virtual head-mounted device or a handheld device; a display screen arranged on a first side of the smart terminal body; a function switching module arranged in the smart terminal body, the function switching module being used for controlling the smart terminal to switch between a virtual head-mounted function and a handheld console function; a master control chip arranged in the smart terminal body, the master control chip being in communication connection with the function switching module; a first camera device arranged on a second side of the smart terminal body, the first side and the second side being opposite sides; and a first inertial sensor arranged on the smart terminal body, the first inertial sensor being used for detecting IMU data of the smart terminal body; wherein the first camera device and the first inertial sensor are respectively in communication connection with the master control chip.

[0008] In an embodiment of the present application, the smart terminal further comprises: a brightness controller arranged in the smart terminal body, the brightness controller being in communication connection with the master control chip and the display screen respectively; wherein when the smart terminal is switched from the handheld console function to the virtual head-mounted function, the brightness controller reduces the display brightness of the display; and when the smart terminal is switched from the virtual head-mounted function to the handheld console function, the brightness controller increases the display brightness of the display.

[0009] In an embodiment of the present application, the smart terminal further comprises: an image processor arranged in the smart terminal body, the image processor being in communication connection with the master control chip; wherein the image processor is used for performing asynchronous space distortion, asynchronous time distortion processing and image rendering processing on image information displayed on the display screen.

[0010] In an embodiment of the present application, the smart terminal further comprises: a second camera device arranged on the first side of the smart terminal body; wherein the second camera device is in communication connection with the master control chip.

[0011] In an embodiment of the present application, the number of the first camera devices is four, and the four first camera devices are divided into two groups of camera devices, and the two first camera devices in each group of camera devices are symmetric about the center of the smart terminal body.

[0012] As a second aspect of the present application, the present application also provides a handheld game machine, comprising: the intelligent terminal described above; and a handheld game machine handle connected with the intelligent terminal; wherein the handheld game machine handle is provided with a first infrared sensor and a second inertial sensor, the second inertial sensor is used for detecting the IMU data of the handheld game machine handle; and the second inertial sensor is in communication connection with the master control chip in the intelligent terminal.

[0013] In an embodiment of the present application, the master control chip comprises: a first control unit, the first control unit is in communication connection with the function switching module, the first camera device and the first inertial sensor respectively, and is used for controlling the first camera device to capture a first image of the surrounding environment where the intelligent terminal is located, and controlling the first inertial sensor to detect the IMU data of the intelligent terminal; and a first calculation unit, the first calculation unit is in communication connection with the first control unit, the first camera device, the first infrared sensor, the first inertial sensor and the second inertial sensor respectively, and is used for acquiring the first image and the IMU data of the intelligent terminal body, and performing calculation on the IMU data of the intelligent terminal body and the first image to generate the spatial positioning information of the intelligent terminal.

[0014] In an embodiment of the present application, the master control chip further comprises: a second control unit, the second control unit is in communication connection with the function switching module, the first camera device and the second inertial sensor respectively; wherein the second control unit is used for, when the function switching module switches the intelligent terminal to the handheld game machine function, controlling the first camera device to capture the first infrared sensor located on the handheld game machine handle, and controlling the second inertial sensor located on the handheld game machine handle to detect the IMU data of the handheld game machine handle; and a second calculation unit, the second calculation unit is in communication connection with the first calculation unit, the second control unit, the first camera device and the second inertial sensor respectively, and is used for acquiring the first light spot image of the first infrared sensor transmitted by the first camera device and the IMU data of the handheld game machine handle transmitted by the second inertial sensor, and performing calculation on the first light spot image, the IMU data of the handheld game machine handle and the spatial positioning information of the intelligent terminal to generate the spatial positioning information of the handheld game machine handle.

[0015] In an embodiment of the present application, the palm machine further comprises a connector, which is communicatively connected with the palm machine handle and the smart terminal respectively; a multifunctional button is arranged on the palm machine handle; and a multifunctional processor is connected with the multifunctional button and a master control chip of the smart terminal respectively, and the multifunctional processor is used for receiving an operation instruction input by a user through the multifunctional button and sending the operation instruction to the master control chip.

[0016] As a third aspect of the present application, the present application further provides a virtual system, comprising: the above-mentioned smart terminal; a virtual head-mounted device, wherein the smart terminal is detachably installed on a body of the virtual head-mounted device; and a virtual head-mounted handle; wherein the virtual head-mounted handle comprises: a control handle; a second infrared sensor arranged on the control handle; and a third inertial sensor arranged on the control handle, and the third inertial sensor is used for measuring IMU data of the control handle; wherein the control handle and the third inertial sensor are communicatively connected with the master control chip respectively.

[0017] In an embodiment of the present application, the virtual head-mounted handle further comprises: a handle shell, which comprises a ring-shaped part and a holding part, wherein a recessed part is arranged at the center of the holding part to accommodate and fix the control handle.

[0018] In an embodiment of the present application, the master control chip comprises: a third control unit, which is communicatively connected with the function switching module, the first camera and the first inertial sensor respectively, and the third control unit is used for controlling the first camera to capture a first image of a surrounding environment where the smart terminal is located and controlling the first inertial sensor to detect IMU data of the smart terminal; and a third calculation unit, which is communicatively connected with the third control unit, the first camera, the first infrared sensor and the first inertial sensor respectively, and the third calculation unit is used for acquiring the first image and the IMU data of the smart terminal body, and calculating the IMU data of the smart terminal body and the first image to generate spatial positioning information of the smart terminal.

[0019] In an embodiment of the present application, the master chip further comprises: a fourth control unit, which is in communication connection with the function switching module, the first camera, the second infrared sensor and the third inertial sensor respectively; the fourth control unit is configured to control the first camera to capture the second infrared sensor on the virtual operation handle and control the third inertial sensor on the virtual operation handle to detect the IMU data of the virtual operation handle when the function switching module switches the smart terminal to the virtual head-mounted display function; a fourth calculation unit, which is in communication connection with the third calculation unit, the first camera and the third inertial sensor respectively, and is configured to acquire the second light spot image of the second infrared sensor transmitted by the first camera and the IMU data of the virtual operation handle transmitted by the third inertial sensor, and calculate the spatial positioning information of the smart terminal, the second light spot image and the IMU data of the virtual operation handle to generate the spatial positioning information of the virtual operation handle.

[0020] As a fourth aspect of the present application, the present application further provides a spatial positioning method of a smart terminal for positioning the above-mentioned smart terminal, wherein the spatial positioning method of the smart terminal comprises: the master chip controls the first camera on the smart terminal to capture a first image of the surrounding environment where the smart terminal is located; and controls the first inertial sensor to detect the IMU data of the smart terminal; the master chip acquires the first image of the surrounding environment where the smart terminal is located captured by the first camera; the master chip acquires the IMU data of the smart terminal body detected by the first inertial sensor; and the master chip calculates the IMU data of the smart terminal body and the first image to generate the spatial positioning information of the smart terminal.

[0021] In an embodiment of the present application, when the smart terminal is in communication connection with the console handle, the spatial positioning method of the smart terminal further comprises: the function switching module switches the smart terminal to the console function; the master chip controls the first camera to capture the first infrared sensor on the console handle, and controls the second inertial sensor on the console handle to detect the IMU data of the console handle; the master chip acquires the first light spot image of the first infrared sensor transmitted by the first camera and the IMU data of the console handle transmitted by the second inertial sensor, and calculates the spatial positioning information of the smart terminal, the first light spot image and the IMU data of the console handle to generate the spatial positioning information of the console handle.

[0022] In an embodiment of the present application, the intelligent terminal is installed on a virtual head-mounted device, and when the intelligent terminal is in communication connection with a virtual operation handle, the spatial positioning method of the intelligent terminal further comprises:

[0023] The function switching module switches the intelligent terminal to a virtual head-mounted function; the control chip controls the first camera to take a picture of a second infrared sensor located on the virtual operation handle, and controls a third inertial sensor located on the virtual operation handle to detect IMU data of the virtual operation handle; the master control chip acquires a second light spot image of the second infrared sensor transmitted by the first camera and the IMU data of the virtual operation handle transmitted by the second inertial sensor, and calculates the spatial positioning information of the intelligent terminal, the second light spot image and the IMU data of the virtual operation handle to generate spatial positioning information of the virtual operation handle.

[0024] The intelligent terminal provided by the present application can be detachably connected with a virtual head-mounted device, that is, the virtual system can be a separated virtual system, that is, the intelligent terminal and the virtual head-mounted device can be separately arranged; after the intelligent terminal and the virtual head-mounted device are separated, the intelligent terminal can be in communication connection with a console handle, and the master control chip in the intelligent terminal switches the function of the intelligent terminal from a virtual head-mounted function to a console function, so that the intelligent terminal and the console handle form a console; after the intelligent terminal is installed on the virtual head-mounted device, the master control chip in the intelligent terminal switches the function of the intelligent terminal from a console function to a virtual head-mounted function, so that the intelligent terminal and the virtual console and the virtual head-mounted device form a virtual system, so that the intelligent terminal can have two functions, and a user can experience the functions of two products of a virtual system and a console through one intelligent terminal. In addition, the first inertial sensor and the first camera are arranged on the intelligent terminal, and the master control chip can determine the spatial positioning information, that is, the 6DOF data, of the intelligent terminal according to the environmental information of the intelligent terminal photographed by the first camera and the IMU data of the intelligent terminal detected by the first inertial sensor, so that accurate positioning of the intelligent terminal is realized. BRIEF DESCRIPTION OF DRAWINGS

[0025] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0026] Figure 1 Fig. 1 shows a front view of an intelligent terminal provided by an embodiment of the present application;

[0027] Figure 2 Fig. 6 shows a rear view of a smart terminal according to an embodiment of the present application;

[0028] Figure 3 Fig. 7 shows a working principle diagram of a smart terminal according to an embodiment of the present application;

[0029] Figure 4 Fig. 8 shows a working principle diagram of a smart terminal according to another embodiment of the present application;

[0030] Figure 5 Fig. 9 shows a working principle diagram of a smart terminal according to another embodiment of the present application;

[0031] Figure 6 Fig. 10 shows a working principle diagram of a palm computer according to an embodiment of the present application;

[0032] Figure 7 Fig. 11 shows a working principle diagram of a palm computer according to another embodiment of the present application;

[0033] Figure 8 Fig. 12 shows a palm computer according to an embodiment of the present application; Figure 7 Fig. 13 shows a flowchart of a spatial positioning method of a smart terminal in the palm computer according to an embodiment of the present application;

[0034] Figure 9 Fig. 14 shows a working principle diagram of a palm computer according to another embodiment of the present application;

[0035] Figure 10 Fig. 15 shows a palm computer according to another embodiment of the present application; Figure 9 Fig. 16 shows a flowchart of a spatial positioning method of a smart terminal in the palm computer according to another embodiment of the present application;

[0036] Figure 11 Fig. 17 shows a working principle diagram of a virtual system according to an embodiment of the present application;

[0037] Figure 12 Fig. 18 shows a working principle diagram of a virtual system according to another embodiment of the present application;

[0038] Figure 13 Fig. 19 shows a virtual system according to another embodiment of the present application; Figure 12 Fig. 20 shows a flowchart of a spatial positioning method of a smart terminal in the virtual system according to an embodiment of the present application;

[0039] Figure 14 Fig. 21 shows a working principle diagram of a virtual system according to another embodiment of the present application;

[0040] Figure 15 Fig. 22 shows a virtual system according to another embodiment of the present application; Figure 14 Fig. 23 shows a flowchart of a spatial positioning method of a smart terminal in the virtual system according to another embodiment of the present application;

[0041] Figure 16Fig. 1 shows a structural schematic diagram of a virtual operation handle provided by another embodiment of the present application;

[0042] Figure 17 Fig. 1 shows a structural schematic diagram of a virtual operation handle provided by another embodiment of the present application;

[0043] Reference signs:

[0044] 1 - intelligent terminal; 100 - intelligent terminal body; 101 - first side; 102 - second side; 200 - display screen; 300 - third camera device; 400 - first camera device; 500 - second camera device; 600 - master control chip; 601 - function switching module; 700 - first inertial sensor; 800 - image processor; 201 - brightness controller; 602 - first control unit; 603 - first calculation unit; second control unit 604; second calculation unit 605; third control unit 607; third calculation unit 608; fourth control unit 609; fourth calculation unit 6091;

[0045] 2 - palm handle; 22 - first infrared sensor; 23 - second inertial sensor; 24 - connector; 25 - multifunctional button; 26 - function processor; 31 - virtual head-mounted device; 32 - virtual operation handle; 33 - control handle; 34 - second infrared sensor; 35 - third inertial sensor; 36 - handle shell; 361 - annular part; 362 - holding part; 37 - third infrared sensor; 38 - fourth inertial sensor; 900 - electronic device; 901 - processor; 902 - memory; 903 - input device; 904 - output device DETAILED DESCRIPTION

[0046] In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly and specifically limited. All directional indications (such as up, down, left, right, front, back, top, bottom, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directional indications also change accordingly. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to the process, method, product or device.

[0047] In addition, reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combined with other embodiments.

[0048] The technical solutions in the embodiments of the present application will be apparently and completely described below with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the protection scope of the present application.

[0049] Figure 1 Fig. 1 shows a front view of an intelligent terminal 1 provided by the present application, Figure 2 Fig. 2 shows a rear view of the intelligent terminal 1 provided by the present application, Figure 3 Fig. 3 shows a working principle diagram of the intelligent terminal 1 provided by the present application, as Figure 1 , Figure 2 and Figure 3 Fig. 4 shows that the intelligent terminal 1 comprises:

[0050] An intelligent terminal body 100, which is detachably installed on a virtual head-mounted device 31 or a handheld device, such as a handheld game console;

[0051] A display screen 200 arranged on a first side 101 of the intelligent terminal body 100, which has a normal display function and can display video information, image information, etc.;

[0052] A function switching module 601 arranged in the intelligent terminal body 100, which can control the intelligent terminal 1 to switch back and forth between the virtual head-mounted function and the handheld game console function; for example, when the intelligent terminal 1 is installed on the virtual head-mounted device, the intelligent terminal 1 and the virtual head-mounted device form a virtual system, such as a virtual reality system (hereinafter referred to as VR system) or an augmented reality system (hereinafter referred to as AR system), at this time, the function switching module 601 can receive information that the intelligent terminal 1 is installed on the virtual head-mounted device, and start the virtual head-mounted function according to the received information, so that the intelligent terminal 1 exercises the virtual head-mounted function;

[0053] A main control chip 600 arranged in the intelligent terminal body 100, which is in communication connection with the function switching module 601;

[0054] The first camera 400 is arranged on the second side 102 of the smart terminal body 100, and the first side 101 and the second side 102 are opposite sides.

[0055] The first camera 400 can be used to shoot the infrared lamp. When the smart terminal 1 is installed on the virtual head-mounted device 31, the smart terminal 1, the virtual operation handle 32 and the virtual head-mounted device 31 form a virtual system. At this time, the smart terminal 1 needs to establish a communication connection with the virtual operation handle 32, and the virtual operation handle 32 is provided with an infrared lamp. At this time, the main control chip 600 of the smart terminal 1 can control the first camera 400 to be turned on, so that the first camera 400 shoots the infrared lamp on the virtual operation handle 32, and transmits the light spot image of the infrared lamp on the virtual operation handle 32 to the main control chip 600.

[0056] The first camera 400 can also be used to shoot the surrounding environment of the smart terminal 1. Whether the smart terminal 1 is used as a virtual head-mounted device or a game console, the smart terminal 1 needs to be positioned. For example, when the smart terminal 1 is combined with the game console handle 2 to form a game console, the smart terminal 1 and the game console handle 2 need to be positioned. When the smart terminal 1 is installed on the virtual head-mounted device 31 and combined with the virtual operation handle 32 to form a virtual system, the smart terminal 1 and the virtual operation handle 32 also need to be positioned. The first camera 400 can shoot the surrounding environment of the smart terminal 1 to form image information or video information reflecting the surrounding environment, and send the image information or video information to the main control chip 600. The main control chip 600 can obtain the position information of the smart terminal 1 according to the image information or video information.

[0057] The first inertial sensor 700 is arranged on the smart terminal body 100, and is used to detect the IMU data of the smart terminal body 100. After the first inertial sensor 700 detects the IMU data of the smart terminal 1, the IMU data is transmitted to the main control chip 600. The main control chip 600 can determine the spatial positioning information of the smart terminal 1, i.e. the 6DOF data of the smart terminal 1, according to the IMU data and the image information or video information of the environment where the smart terminal 1 is located shot by the first camera 400, i.e. the 6 degrees of freedom based on the translational freedom and the rotational freedom.

[0058] Specifically, the inertial sensor (Inertial Measurement Unit, IMU for short) is a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object. Generally, an IMU includes three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object on the independent three-axis of the carrier coordinate system, and the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system. Therefore, the inertial sensor can measure the angular velocity and acceleration of an object in three-dimensional space, and calculate the attitude of the object, for example, the rotational degrees of freedom of the object, which refers to the three position-related degrees of freedom of up and down, front and back, and left and right. The IMU data is the result data detected by the inertial sensor, that is, the angular velocity and acceleration data of an object in three-dimensional space detected by the inertial sensor. Therefore, the first inertial sensor 700 can detect the IMU data of the smart terminal body 100, and the IMU data of the smart terminal body 100 can be used to calculate the attitude of the smart terminal body 100, for example, the rotational degrees of freedom of the smart terminal body 100, which refers to the three position-related degrees of freedom of up and down, front and back, and left and right.

[0059] The smart terminal 1 provided by the application can be detachably connected with the virtual head-mounted device 31, that is, the virtual system is a separated virtual system, that is, the smart terminal 1 and the virtual head-mounted device 31 are separately arranged. When the smart terminal 1 is separated from the virtual head-mounted device 31, the smart terminal 1 can be in communication connection with the handheld game controller 2, and the main control chip 600 in the smart terminal 1 switches the function of the smart terminal 1 from the virtual head-mounted function to the handheld game function, so that the smart terminal 1 and the handheld game controller 2 form a handheld game machine. When the smart terminal 1 is installed on the virtual head-mounted device 31, the main control chip 600 in the smart terminal 1 switches the function of the smart terminal 1 from the handheld game function to the virtual head-mounted function, so that the smart terminal 1, the virtual handheld game machine and the virtual head-mounted device 31 form a virtual system, thereby realizing that the smart terminal 1 can have two functions, and the user can experience the functions of the virtual system and the handheld game machine through one smart terminal 1. In addition, the first inertial sensor 700 and the first camera 400 are arranged on the smart terminal 1, and the main control chip 600 can determine the spatial positioning information of the smart terminal 1, that is, the 6DOF data, according to the environmental information of the smart terminal 1 photographed by the first camera 400 and the IMU data of the smart terminal 1 detected by the first inertial sensor 700, thereby realizing the accurate positioning of the smart terminal 1.

[0060] Optionally, as Figure 1As shown, the smart terminal 1 also includes a third camera device 300 disposed on the second side 102 of the smart terminal body 100. The third camera device 300 is a depth camera device, which can be used to capture images of the surrounding environment of the smart terminal 1. In addition to acquiring planar images of the surrounding environment of the smart terminal 1, the depth camera device can also acquire depth information of the surrounding environment of the smart terminal 1, that is, the three-dimensional position and size information of the surrounding environment of the smart terminal 1. More complex positional relationships between objects can be obtained through distance information. Therefore, when the main control chip 600 performs spatial positioning of the smart terminal 1, it can determine the spatial positioning information of the smart terminal 1 based on the environmental information captured by the first camera device 400, the depth information of the environment captured by the third camera device 300, and the IMU data of the smart terminal 1, thereby improving the accuracy of the spatial positioning of the smart terminal 1.

[0061] Optionally, the number of first camera devices 400 is four, and the four first camera devices 400 are divided into two groups. The two first camera devices 400 in each group are symmetrical about the center of the smart terminal body 100. For example, when the smart terminal body 100 is a cuboid, that is, the second side 102 on the smart terminal body 100 is a quadrilateral, the four first camera devices 400 can be distributed at the four corners of the quadrilateral. This maximizes the position of the first camera devices 400, enabling them to capture the light spot image of the infrared sensor and the surrounding environment of the smart terminal 1.

[0062] In one embodiment of the present invention, Figure 4 The diagram shown is a schematic diagram of the working principle of a smart terminal 1 according to another embodiment of the present invention. Figure 4 As shown, the smart terminal also includes a brightness controller 201 disposed within the smart terminal body. The brightness controller 201 is communicatively connected to the main control chip 600 and the display screen 200. When the smart terminal 1 is installed on a virtual head-mounted display, the function switching module 601 switches the function of the smart terminal 1 to the virtual head-mounted display function, and the brightness controller 201 reduces the brightness of the display screen 200. Because the brightness of the display screen 200 is reduced, the illumination time of the display pixels when displaying one frame of an image is also shortened. That is, the time it takes for the pixels to be illuminated when displaying one frame of an image is shortened, and the afterglow time is also shortened. Therefore, the reduction in the clarity of the display image on the display screen 200 due to the trailing phenomenon is mitigated, improving the user experience. When the smart terminal 1 is installed on a handheld device, the function switching module 601 switches the function of the smart terminal 1 to the handheld device function, and the brightness controller 201 increases the brightness of the display screen 200.

[0063] In one embodiment of the present invention, Figure 5As shown in the working principle diagram of the intelligent terminal 1 provided by another embodiment of the application, as shown in the figure, Figure 5 As shown, the intelligent terminal 1 further comprises an image processor 800 arranged in the intelligent terminal body 100, the image processor 800 is in communication connection with the master control chip 600, when the intelligent terminal 1 is installed on the virtual head-mounted device 31, the function switching module 601 in the master control chip 600 switches the function of the intelligent terminal 1 to the virtual head-mounted function, the master control chip 600 then transmits the information that the intelligent terminal 1 has switched to the virtual head-mounted function to the image processor 800, the image processor 800 then starts working according to the information, so that the image information to be displayed on the display screen 200 can be subjected to asynchronous space distortion, asynchronous time distortion processing and image rendering processing, so as to improve the picture clarity.

[0064] In an embodiment of the application, as shown in the figure, Figure 1 As shown, the intelligent terminal 1 further comprises a second camera 500 arranged on the first side 101 of the intelligent terminal body 100, the second camera 500 is used for photographing the user of the intelligent terminal 1; wherein the second camera 500 is in communication connection with the master control chip 600. The second camera 500 can photograph the user, that is, realize the user's self-photographing, the master control chip 600 can acquire the image information of the user, so that the user authentication or the generation of the virtual avatar of the user can be realized according to the image information of the user.

[0065] As a second aspect of the application, the application further provides a handheld game machine, Figure 6 As shown in the working principle diagram of the handheld game machine provided by an embodiment of the application, as shown in the figure, Figure 6 As shown, the handheld game machine comprises the above-mentioned intelligent terminal 1 and a handheld game machine handle 2 connected with the intelligent terminal 1. The handheld game machine handle 2 is provided with a first infrared sensor 22 and a second inertial sensor 23, the second inertial sensor 23 is used for detecting the IMU data of the handheld game machine handle 2. The second inertial sensor 23 is in communication connection with the master control chip 600 in the intelligent terminal 1; the first camera 400 on the intelligent terminal 1 is used for photographing the first infrared sensor 22 and the surrounding environment where the intelligent terminal 1 is located. When the handheld game machine handle 2 is in communication connection with the intelligent terminal 1, the function switching module 601 then switches the function of the intelligent terminal 1 to the handheld game machine function, at this time, the intelligent terminal 1 and the handheld game machine handle 2 constitute the handheld game machine.

[0066] Optionally, as shown in the figure, Figure 6As shown, the palm machine further comprises: a connector 24, which is communicatively connected with the palm machine handle 2 and the smart terminal 1 respectively, and the connector 24 can realize the communication connection between the palm machine handle 2 and the smart terminal 1; a multifunctional key 25 arranged on the palm machine handle 2; and a function processor 26, which is connected with the multifunctional key 25 and the main control chip 600 of the smart terminal 1 respectively, and the function processor 26 is used for receiving the operation instruction input by the user through the multifunctional key 25 and sending the operation instruction to the main control chip 600.

[0067] In an embodiment of the present application, when the smart terminal 1 and the palm machine handle 2 form the palm machine, during the operation of the palm machine by the user, the smart terminal 1 and the palm machine handle 2 both need spatial positioning, Figure 7 As shown in the working principle diagram of the palm machine provided by another embodiment of the present application, Figure 7 As shown, the main control chip 600 comprises:

[0068] A first control unit 602, which is communicatively connected with the function switching module 601, the first camera 400 and the first inertial sensor 700 respectively, and the first control unit 602 is used for controlling the first camera 400 to shoot the first image of the surrounding environment where the smart terminal 1 is located and controlling the first inertial sensor 700 to detect the IMU data of the smart terminal 1; and

[0069] A first calculation unit 603, which is communicatively connected with the first control unit 602, the first camera 400, the first infrared sensor 22, the first inertial sensor 700 and the second inertial sensor 23 respectively, and the first calculation unit 603 is used for acquiring the first image and the IMU data of the smart terminal body, and calculating the IMU data of the smart terminal body and the first image to generate the spatial positioning information of the smart terminal.

[0070] Specifically, Figure 8 As shown in the working principle diagram of the palm machine provided by another embodiment of the present application, Figure 7 As shown in the flowchart of the spatial positioning method of the smart terminal 1 in the palm machine, namely Figure 7 As shown in the spatial positioning method of the smart terminal in the palm machine, Figure 8 As shown, the spatial positioning method of the smart terminal 1 comprises the following steps:

[0071] Step S101: The function switching module 601 receives the connection information that the smart terminal 1 is communicatively connected with the palm machine handle 2, and switches the function of the smart terminal 1 to the palm machine function according to the connection information;

[0072] Step S102: When the function switching module 601 successfully switches the smart terminal 1 to the handheld console function, the first control unit 602 controls the first camera 400 on the smart terminal 1 to capture a first image of the surrounding environment of the smart terminal 1; and controls the first inertial sensor 700 to detect the IMU data of the smart terminal 1.

[0073] The first camera 400 captures the surrounding environment of the smart terminal 1 under the control of the first control unit 602, forms a first image, and transmits the first image to the first computing unit 603. The first inertial sensor 700 detects the IMU data of the smart terminal 1 under the control of the first control unit 602, and transmits the IMU data of the smart terminal 1 to the first computing unit 603.

[0074] Step S103: After the first computing unit 603 receives the first image transmitted by the first camera 400 and the IMU data of the smart terminal 1 transmitted by the first inertial sensor 700, the first computing unit 603 calculates the IMU data of the smart terminal body 100 and the first image, and generates the spatial positioning information of the smart terminal 1.

[0075] The first inertial sensor 700 is used to detect the IMU data of the smart terminal body 100, and the IMU data refers to the freedom of 3 rotation angles. After the first inertial sensor 700 detects the IMU data of the smart terminal 1, the IMU data is transmitted to the first computing unit 603. The first computing unit 603 can determine the spatial positioning information of the smart terminal 1, i.e. the 6DOF data (hereinafter referred to as 6DOF data) of the smart terminal 1, according to the IMU data and the image information or video information of the environment where the smart terminal 1 is located captured by the first camera 400, i.e. based on the translational freedom and the rotational freedom, the freedom of 6 angles can be obtained.

[0076] Steps S101-S103 can realize the spatial positioning of the smart terminal 1, i.e. through the first inertial sensor 700 and the first camera 400 arranged on the smart terminal 1, the spatial positioning of the smart terminal 1 can be realized.

[0077] Further, as shown in Figure 9 The master control chip 600 further comprises:

[0078] The second control unit 604 is in communication connection with the function switching module 601, the first camera 400 and the second inertial sensor 23 respectively; wherein the second control unit 604 is configured to control the first camera 400 to capture the first infrared sensor 22 located on the handle of the console when the function switching module 601 switches the smart terminal to the console function, and control the second inertial sensor 23 located on the handle of the console to detect the IMU data of the handle of the console.

[0079] The second calculation unit 605 is in communication connection with the first calculation unit 603, the second control unit 604, the first camera 400 and the second inertial sensor 23 respectively, and the second calculation unit 604 is configured to acquire the spatial positioning information of the smart terminal 1 transmitted by the first calculation unit 603, the first light spot image of the first infrared sensor 22 transmitted by the first camera 400 and the IMU data of the handle of the console transmitted by the second inertial sensor 23, and calculate the spatial positioning information of the smart terminal 1, the first light spot image and the IMU data of the handle of the console to generate the spatial positioning information of the handle of the console

[0080] Specifically, Figure 10 The flowchart shown in FIG. 1 is a flowchart of the spatial positioning method of the smart terminal 1, that is, Figure 9 The positioning method of the spatial positioning of the smart terminal in the console shown in FIG. 2 is as follows: Figure 10 As shown in FIG. 1, the spatial positioning method of the smart terminal 1 further includes the following steps:

[0081] Step S104: The second control unit 604 controls the first camera 400 to capture the first infrared sensor 22 located on the handle 2 of the console, and controls the second inertial sensor 23 located on the handle 2 of the console to detect the IMU data of the handle 2 of the console.

[0082] The first camera 400 captures the first infrared sensor 22 on the handle 2 of the console under the control of the second control unit 604 to form the first light spot image of the first infrared sensor 22. The second inertial sensor 23 detects the IMU data of the handle 2 of the console under the control of the second control unit 604, and transmits the IMU data of the handle 2 of the console to the first spatial positioning unit 606.

[0083] Step S105: The second computing unit 605 acquires the spatial positioning information of the intelligent terminal 1 transmitted by the first computing unit 603, the first light spot image of the first infrared sensor 22 transmitted by the first camera 400, and the IMU data of the gamepad 2 transmitted by the second inertial sensor 23, and performs calculation on the spatial positioning information of the intelligent terminal 1, the first light spot image, and the IMU data of the gamepad 2 to generate the spatial positioning information of the gamepad 2. That is, the 6DOF data (hereinafter referred to as 6DOF data) of the gamepad 2, that is, the 6-degree-of-freedom can be obtained based on the translational freedom and the rotational freedom.

[0084] The spatial positioning of the gamepad 2 can be realized through steps S104-S105. That is, the spatial positioning information of the gamepad 2 can be determined by the first light spot image of the first infrared sensor 22 on the gamepad 2 captured by the first camera 400 on the intelligent terminal 1 and the IMU data of the gamepad 2 detected by the second inertial sensor 23 on the gamepad 2.

[0085] Exemplary Virtual System

[0086] As a third aspect of the present application, the present application also provides a virtual system, Figure 11 As shown in the working principle diagram of the gamepad provided by an embodiment of the present application, Figure 11 As shown, the virtual system comprises the above-mentioned intelligent terminal 1, a virtual head-mounted device 31, and a virtual operation handle 32. The intelligent terminal 1 is detachably installed on the body of the virtual head-mounted device 31. The virtual operation handle 32 comprises a control handle 33, a second infrared sensor 34 arranged on the control handle 33, and a third inertial sensor 35 arranged on the control handle 33, and the third inertial sensor 35 is used for measuring the IMU data of the control handle 33. The control handle 33 and the third inertial sensor 35 are respectively in communication connection with the master control chip 600.

[0087] When the intelligent terminal 1 is installed on the virtual head-mounted device 31, the function switching module 601 switches the function of the intelligent terminal 1 to the virtual head-mounted function. At this time, the intelligent terminal 1, the virtual head-mounted device 31, and the virtual operation handle 32 form a virtual system, for example, the intelligent terminal 1, the VR helmet, and the VR handle form a VR system.

[0088] When the intelligent terminal 1 is installed on the virtual head-mounted device 31, in the process of operating the virtual system by the user, the spatial positioning of the intelligent terminal 1 and the virtual operation handle 32 is required, Figure 12 As shown in the working principle diagram of the virtual system provided by another embodiment of the present application, Figure 12 As shown, the master control chip 600 comprises:

[0089] The third control unit 607 is in communication connection with the function switching module 601, the first camera 400 and the first inertial sensor 700 respectively, and is configured to control the first camera 400 to capture a first image of the surrounding environment where the smart terminal is located, and control the first inertial sensor 700 to detect the IMU data of the smart terminal.

[0090] The third calculation unit 408 is in communication connection with the third control unit 407, the first camera 400, the first infrared sensor 22 and the first inertial sensor 700 respectively, and is configured to acquire the first image and the IMU data of the smart terminal body, and calculate the IMU data of the smart terminal body and the first image to generate the spatial positioning information of the smart terminal.

[0091] Specifically, Figure 13 The flowchart shown is the spatial positioning method of the smart terminal in the virtual system, that is, Figure 13 The positioning method when the spatial positioning of the smart terminal in the virtual system is as shown, Figure 13

[0092] The spatial positioning method of the smart terminal 1 includes the following steps:

[0093] Step S201: After the function switching module 601 receives the installation information that the smart terminal 1 is installed on the virtual head-mounted device 31, the function switching module 601 switches the function of the smart terminal 1 to the virtual head-mounted function according to the installation information;

[0094] Step S202: When the function switching module 601 successfully switches the smart terminal 1 to the virtual head-mounted function, the third control unit 607 controls the first camera 400 located on the smart terminal 1 to capture a first image of the surrounding environment where the smart terminal 1 is located, and controls the first inertial sensor 700 to detect the IMU data of the smart terminal 1.

[0095] The first camera captures the surrounding environment where the smart terminal 1 is located under the control of the third control unit 607, forms a first image, and transmits the first image to the third calculation unit 608. The first inertial sensor 700 detects the IMU data of the smart terminal 1 under the control of the third control unit 607, and transmits the IMU data of the smart terminal 1 to the third calculation unit 608.

[0096] Step S203: After the third calculation unit 608 receives the first image transmitted by the first camera 400 and the IMU data of the smart terminal body 100 transmitted by the first inertial sensor 700, the third calculation unit 608 calculates the IMU data of the smart terminal body 100 and the first image to generate the spatial positioning information of the smart terminal 1. ​

[0097] The first inertial sensor 700 is used for detecting the IMU data of the smart terminal body 100, and the IMU data refers to three rotational angle degrees of freedom. After the first inertial sensor 700 detects the IMU data of the smart terminal body 100, the IMU data is transmitted to the third computing unit 608. The third computing unit 608 can determine the spatial positioning information of the smart terminal 1 according to the IMU data and the image information or video information of the environment in which the smart terminal 1 is located, which is captured by the first camera 400, that is, the 6DOF data (hereinafter referred to as 6DOF data) of the smart terminal 1, that is, the 6 angle degrees of freedom based on the translational freedom and the rotational freedom.

[0098] The steps S201-S203 can realize the spatial positioning of the smart terminal 1. That is, the spatial positioning of the smart terminal 1 can be realized by the first inertial sensor 700 and the first camera 400 arranged on the smart terminal 1.

[0099] Further, Figure 14 The working principle diagram of the virtual system provided by another embodiment of the application is shown in the figure. Figure 14 As shown, the master control chip 600 further includes:

[0100] The fourth control unit 609 is in communication connection with the function switching module 601, the first camera 400, the second infrared sensor 34 and the third inertial sensor 35 respectively. When the function switching module 601 switches the smart terminal to the virtual head-mounted function, the fourth control unit 609 controls the first camera 400 to capture the second infrared sensor 34 on the virtual operation handle, and controls the third inertial sensor 35 on the virtual operation handle to detect the IMU data of the virtual operation handle.

[0101] The fourth computing unit 6091 is in communication connection with the third computing unit 608, the first camera 400 and the third inertial sensor 35 respectively. The fourth computing unit 6091 is used for acquiring the spatial positioning information of the smart terminal 1 calculated by the third computing unit 608, the second light spot image of the second infrared sensor 34 transmitted by the first camera 400 and the IMU data of the virtual operation handle transmitted by the third inertial sensor 35, and calculating the spatial positioning information of the smart terminal 1, the second light spot image and the IMU data of the virtual operation handle to generate the spatial positioning information of the virtual operation handle.

[0102] Specifically, Figure 15 The working principle diagram of the virtual system provided by another embodiment of the application is shown in the figure. Figure 14 The flowchart of the spatial positioning method of the smart terminal in the virtual system is shown in the figure, that is, Figure 15The image shows a positioning method for spatial positioning of a smart terminal in a virtual system. Figure 15 As shown, the spatial positioning method of smart terminal 1 further includes the following steps:

[0103] Step S204: The fourth control unit 609 controls the first camera device 400 to take pictures of the second infrared sensor 34 located on the virtual operating handle 32, and controls the third inertial sensor 35 located on the virtual operating handle 32 to detect the IMU data of the virtual operating handle 32.

[0104] Step S205: The fourth calculation unit 6091 acquires the spatial positioning information of the smart terminal 1 calculated by the third calculation unit 608, the second light spot image of the second infrared sensor 34 transmitted by the first camera device 400, and the IMU data of the virtual operating handle 32 transmitted by the second inertial sensor 23. It then calculates the spatial positioning information of the smart terminal 1, the second light spot image, and the IMU data of the virtual operating handle 32 to generate the spatial positioning information of the virtual operating handle 32. This is the 6DOF data of the virtual operating handle 32 (hereinafter referred to as 6DOF data), which means that six degrees of freedom (6 angular degrees of freedom) can be obtained based on translational and rotational degrees of freedom.

[0105] The spatial positioning of the virtual controller 32 can be achieved through steps S204-S205. Specifically, the spatial positioning information of the virtual controller 32 can be determined by capturing the second light spot image of the second infrared sensor 34 on the virtual controller 32 using the first camera device 400 on the smart terminal 1, and by detecting the IMU data of the handheld controller 2 using the third inertial sensor 35 located on the virtual controller 32.

[0106] Optional, Figure 16 The diagram shown is a structural schematic of a virtual operating handle 32 provided in another embodiment of the present invention. Figure 16 As shown, the virtual control handle 32 also includes a handle housing 36, which includes an annular portion 361 and a grip portion 362. The grip portion 362 has a recess in the center to accommodate and fix the control handle 33.

[0107] Below, for reference Figure 17 To describe an electronic device according to an embodiment of the present invention. Figure 17 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention.

[0108] like Figure 17 As shown, the electronic device 900 includes one or more processors 901 and memory 902.

[0109] The processor 901 can be a central processing unit (CPU) or other form of processing unit having data processing and / or information executing capabilities, and can control other components in the electronic device 900 to perform desired functions.

[0110] The memory 901 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program information can be stored on the computer-readable storage media, and the processor 901 can run the program information to implement the spatial positioning method of the intelligent terminal according to various embodiments of the present application described above or other desired functions.

[0111] In one example, the electronic device 900 can further include an input device 903 and an output device 904, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0112] The input device 903 can include, for example, a keyboard, a mouse, and / or the like.

[0113] The output device 904 can output various information to the outside. The output device 904 can include, for example, a display, a communication network and a remote output device connected thereto, and / or the like.

[0114] Of course, in order to simplify, Figure 17 Only some of the components in the electronic device 900 related to the present application are shown in the figure, and components such as buses, input / output interfaces, and the like are omitted. In addition, the electronic device 900 can include any other appropriate components according to specific application cases.

[0115] In addition to the above method and device, embodiments of the present application can also be a computer program product including computer program information that, when executed by a processor, causes the processor to perform the steps in the spatial positioning method of the intelligent terminal according to various embodiments of the present application described in the specification.

[0116] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the present application are not limited by the

[0117] In addition, embodiments of the present application can also be a computer readable storage medium, which stores computer program information, when the computer program information is run by a processor, the processor executes the steps of the spatial positioning method of the intelligent terminal according to various embodiments of the present application in the specification.

[0118] The computer readable storage medium can take any combination of one or more of the following readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage medium include a communication connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0119] The above describes the basic principles of the present application in combination with specific embodiments, but it should be noted that the advantages, advantages, effects and the like mentioned in the present application are only examples and are not limited, and these advantages, advantages, effects and the like cannot be considered as the must-have of each embodiment of the present application. In addition, the above-mentioned specific details are only for the purpose of example and for the purpose of understanding, and are not limited, and the above-mentioned details do not limit the present application to the must-use of the above-mentioned specific details to realize.

[0120] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be as shown in the block diagrams. As those skilled in the art will recognize, the devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have", etc. are open-ended words, mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably, unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0121] It is also necessary to point out that in the devices, apparatuses and methods of the present application, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions of the present application.

[0122] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the present application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0123] The above description is only the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A smart terminal, characterized by, The smart terminal comprises: a smart terminal body, which is detachably installed on a virtual head-mounted device or a handheld device; a display screen arranged on a first side of the smart terminal body; a function switching module arranged in the smart terminal body, which is used to control the smart terminal to switch between a virtual head-mounted function and a handheld console function; a main control chip arranged in the smart terminal body, which is in communication connection with the function switching module; a first camera arranged on a second side of the smart terminal body, the first side and the second side being opposite sides, the first camera being used to take pictures of a first infrared sensor installed on a handheld console handle or a second infrared sensor on a control handle in a virtual head-mounted device, and to take pictures of the surrounding environment of the smart terminal; and a first inertial sensor arranged on the smart terminal body, which is used to detect IMU data of the smart terminal body; wherein the first camera and the first inertial sensor are respectively in communication connection with the main control chip; a third camera arranged on the second side of the smart terminal body, which is in communication connection with the main control chip, the third camera being a depth camera, and the third camera being used to take three-dimensional position and size information of the surrounding environment of the smart terminal.

2. The intelligent terminal of claim 1, wherein Further comprising: a brightness controller arranged in the smart terminal body, which is in communication connection with the main control chip and the display screen respectively; wherein when the smart terminal is switched from the handheld console function to the virtual head-mounted function, the brightness controller reduces the display brightness of the display screen; and when the smart terminal is switched from the virtual head-mounted function to the handheld console function, the brightness controller increases the display brightness of the display screen. Further comprising:

3. The intelligent terminal of claim 1, wherein an image processor arranged in the smart terminal body, which is in communication connection with the main control chip; wherein the image processor is used to perform asynchronous spatial distortion, asynchronous time distortion and image rendering processing on image information displayed on the display screen. Further comprising:

4. The intelligent terminal of claim 1, wherein a second camera arranged on the first side of the smart terminal body, wherein the second camera is in communication connection with the main control chip. The number of the first cameras is four, and the four first cameras are divided into two groups of cameras, and the two first cameras in each group of cameras are symmetric about the center of the smart terminal body.

5. The intelligent terminal of claim 1, wherein The smart terminal of claim 1; and a handheld console handle connected with the smart terminal; 6. A hand-held game machine characterized by comprising: wherein the handheld console handle is provided with a first infrared sensor and a second inertial sensor, the second inertial sensor being used to detect IMU data of the handheld console handle; the second inertial sensor being in communication connection with the main control chip in the smart terminal. The main control chip comprises: ​ 7. The handheld game console of claim 6, wherein, ​ A first control unit, which is in communication connection with the function switching module, the first camera and the first inertial sensor respectively, and is configured to control the first camera to capture a first image of the surrounding environment where the smart terminal is located, and control the first inertial sensor to detect the IMU data of the smart terminal; A first calculation unit, which is in communication connection with the first control unit, the first camera, the first infrared sensor, the first inertial sensor and the second inertial sensor respectively, and is configured to acquire the first image and the IMU data of the smart terminal body, and calculate the IMU data of the smart terminal body and the first image to generate the spatial positioning information of the smart terminal.

8. The handheld controller of claim 7, wherein, The master control chip further comprises: A second control unit, which is in communication connection with the function switching module, the first camera and the second inertial sensor respectively; wherein the second control unit is configured to control the first camera to capture the first infrared sensor located on the console handle when the function switching module switches the smart terminal to the console function, and control the second inertial sensor located on the console handle to detect the IMU data of the console handle; A second calculation unit, which is in communication connection with the first calculation unit, the second control unit, the first camera and the second inertial sensor respectively, and is configured to acquire the first image of the first infrared sensor transmitted by the first camera and the IMU data of the console handle transmitted by the second inertial sensor, and calculate the first image, the IMU data of the console handle and the spatial positioning information of the smart terminal to generate the spatial positioning information of the console handle.

9. The handheld game console of claim 6, wherein, Further comprising: A connector, which is in communication connection with the console handle and the smart terminal respectively; the console handle is provided with a multifunctional key; And A multifunctional processor, which is connected with the multifunctional key and the master control chip of the smart terminal respectively, and is configured to receive the operation instruction input by the user through the multifunctional key, and send the operation instruction to the master control chip.

10. A virtual system, comprising: Comprise: The smart terminal of claim 1; A virtual head-mounted device, the smart terminal is detachably installed on the virtual head-mounted device body; And a virtual head-mounted handle; Wherein, the virtual head-mounted handle comprises: A control handle; A second infrared sensor arranged on the control handle; and A third inertial sensor arranged on the control handle, which is configured to measure the IMU data of the control handle; Wherein, the control handle and the third inertial sensor are in communication connection with the master control chip respectively.

11. The virtual system of claim 10, wherein, The virtual head-mounted handle further comprises a handle shell, the handle shell comprising a ring-shaped part and a holding part, wherein a recess is arranged at the center of the holding part to accommodate and fix the control handle.

12. The virtual system of claim 10, wherein, The master control chip comprises: A third control unit, which is in communication connection with the function switching module, the first camera and the first inertial sensor respectively, and is used to control the first camera to capture a first image of the surrounding environment where the smart terminal is located, and control the first inertial sensor to detect the IMU data of the smart terminal; A third calculation unit, which is in communication connection with the third control unit, the first camera, the first infrared sensor and the first inertial sensor respectively, and is used to acquire the first image and the IMU data of the smart terminal body, and calculate the IMU data of the smart terminal body and the first image to generate the spatial positioning information of the smart terminal.

13. The virtual system of claim 12, wherein, The master control chip further comprises: A fourth control unit, which is in communication connection with the function switching module, the first camera, the second infrared sensor and the third inertial sensor respectively; the fourth control unit is used to control the first camera to capture the second infrared sensor located on the control handle when the function switching module switches the smart terminal to the virtual head-mounted function, and control the third inertial sensor located on the control handle to detect the IMU data of the control handle; A fourth calculation unit, which is in communication connection with the third calculation unit, the first camera and the third inertial sensor respectively, and is used to acquire the second light spot image of the second infrared sensor transmitted by the first camera and the IMU data of the control handle transmitted by the third inertial sensor, and calculate the spatial positioning information of the smart terminal, the second light spot image and the IMU data of the control handle to generate the spatial positioning information of the control handle.

14. A spatial positioning method of a smart terminal for positioning the smart terminal of claim 1, characterized in that, The spatial positioning method of the smart terminal comprises: The master control chip controls the first camera located on the smart terminal to capture a first image of the surrounding environment where the smart terminal is located; and controls the first inertial sensor to detect the IMU data of the smart terminal; The master control chip acquires the first image of the surrounding environment where the smart terminal is located captured by the first camera; The master control chip acquires the IMU data of the smart terminal body detected by the first inertial sensor; and The master control chip calculates the IMU data of the smart terminal body and the first image to generate the spatial positioning information of the smart terminal; When the smart terminal is in communication connection with the console handle, the spatial positioning method of the smart terminal further comprises: The function switching module switches the smart terminal to the console function; The fourth control unit controls the first camera to capture the second infrared sensor located on the control handle when the function switching module switches the smart terminal to the virtual head-mounted function, and controls the third inertial sensor located on the control handle to detect the IMU data of the control handle; The master chip controls the first camera to shoot a first infrared sensor on the gamepad handle and controls a second inertial sensor on the gamepad handle to detect IMU data of the gamepad handle; The master chip acquires a first light spot image of the first infrared sensor transmitted by the first camera and IMU data of the gamepad handle transmitted by the second inertial sensor, and calculates spatial positioning information of the smart terminal, the first light spot image and the IMU data of the gamepad handle to generate spatial positioning information of the gamepad handle; The smart terminal is installed on a virtual head-mounted device, and when the smart terminal is in communication connection with the control handle, the spatial positioning method of the smart terminal further comprises: The function switching module switches the smart terminal to a virtual head-mounted function; The master chip controls the first camera to shoot a first infrared sensor on the gamepad handle and controls a second inertial sensor on the gamepad handle to detect IMU data of the gamepad handle; The master chip acquires a first light spot image of the first infrared sensor transmitted by the first camera and IMU data of the gamepad handle transmitted by the second inertial sensor, and calculates spatial positioning information of the smart terminal, the first light spot image and the IMU data of the gamepad handle to generate spatial positioning information of the gamepad handle.

Images