Positioning method, system and device of VR glasses

Abstract

The invention provides a VR glasses positioning method, system and device, and relates to the technical field of VR, and the method comprises the steps: receiving the first position information, sent by a GNSS of a vehicle, of the GNSS in a ground coordinate system; second position information, sent by a positioning base station in the vehicle, of the VR glasses in the vehicle body coordinate system relative to the GNSS sensor is received; first position measurement information detected by the IMU and sent by the VR glasses is received, and third position information of the VR glasses in the ground coordinate system relative to the GNSS is determined based on the first position measurement information and the second position information; and determining target position information of the VR glasses in the ground coordinate system based on the first position information and the third position information. According to the invention, the VR glasses can be accurately positioned.

Description

Technical Field

[0001] This application relates to the field of VR technology, and particularly to a positioning method, system and device for a VR glasses. Background Art

[0002] With the continuous development of VR (Virtual Reality) technology, VR glasses have emerged. A VR glasses is a head-mounted VR display device that closes a person's vision of the outside world and guides the user to have a feeling of being in a virtual environment. The display principle is that the left and right eye screens respectively display the images of the left and right eyes. After the human eye obtains this differential information, a three-dimensional sense is generated in the mind.

[0003] Currently, the usage scenarios of VR glasses are mostly static scenarios such as indoors. In static scenarios, the VR glasses can achieve positioning more accurately through the built-in gyroscope. Thus, according to the satellite positioning information obtained from the positioning, VR applications are displayed on the display screen of the VR glasses.

[0004] However, with many automobile companies deploying the metaverse in vehicles, the usage scenarios of VR glasses have changed from static scenarios to dynamic scenarios in vehicles. When a user sits in a vehicle and wears VR glasses, as the vehicle moves, the user can see the moving pictures along with the vehicle through the VR glasses. The pictures are generated based on the positioning results of the VR glasses. In this case, accurately positioning the VR glasses has become the key to meeting the above-mentioned usage requirements of VR glasses. Summary of the Invention

[0005] Embodiments of this application provide a positioning method, system and device for a VR glasses, which can accurately position the VR glasses to provide relatively accurate position information for VR applications. The technical solutions are as follows:

[0006] In a first aspect, a positioning method for a VR glasses is provided. The method is applied to an in-vehicle computing unit, and the method includes:

[0007] Receiving first position information of a Global Navigation Satellite System (GNSS) sensor of a vehicle in a ground coordinate system, where the first position information is detected by the GNSS sensor;

[0008] Receiving second position information of a virtual reality (VR) glasses relative to the GNSS sensor in a vehicle body coordinate system of the vehicle sent by a positioning base station in the vehicle;

[0009] Receive the first position measurement information detected by the inertial measurement unit (IMU) of the VR glasses sent by the VR glasses, and determine the third position information of the VR glasses relative to the GNSS in the ground coordinate system based on the first position measurement information and the second position information;

[0010] Based on the first position information and the third position information, determine the target position information of the VR glasses in the ground coordinate system, where the target position information is used to instruct the vehicle-mounted computing unit to generate a VR image based on the target position information and display the VR image through the VR glasses.

[0011] In a possible implementation, the method further includes:

[0012] Receive the second position measurement information sent by the IMU of the vehicle, where the second position measurement information is detected by the IMU of the vehicle;

[0013] Based on the second position measurement information and the first position information, determine the corrected position information of the GNSS sensor in the ground coordinate system;

[0014] The determining the target position information of the VR glasses in the ground coordinate system based on the first position information and the third position information includes:

[0015] Based on the corrected position information and the third position information, determine the target position information of the VR glasses in the ground coordinate system.

[0016] In a possible implementation, the second position measurement information includes first acceleration information, and the determining the corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information includes:

[0017] Perform a double integral of the first acceleration information with respect to time to obtain the fourth position information;

[0018] Based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration period, determine the fifth position information;

[0019] Based on the first position information and the fifth position information, determine the corrected position information of the GNSS sensor in the ground coordinate system.

[0020] In a possible implementation, the second position measurement information further includes the first rotation angle information of the vehicle body in the ground coordinate system, and the method further includes:

[0021] Determine the seventh position information of the GNSS sensor relative to the IMU of the vehicle in the ground coordinate system based on the sixth position information of the GNSS sensor relative to the IMU of the vehicle in the body coordinate system and the first rotation angle information;

[0022] The determining the fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration period includes:

[0023] Determine the fifth position information based on the fourth position information, the seventh position information, and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration period.

[0024] In a second aspect, a positioning system for a VR glasses is provided. The system includes a GNSS sensor of a vehicle, an IMU of the vehicle, a positioning base station of the vehicle, an in-vehicle computing unit of the vehicle, and the VR glasses, wherein:

[0025] The GNSS sensor is configured to detect the first position information of the GNSS sensor in the ground coordinate system and send the first position information to the in-vehicle computing unit;

[0026] The positioning base station is configured to determine the second position information of the VR glasses relative to the GNSS sensor in the body coordinate system of the vehicle and send the second position information to the in-vehicle computing unit;

[0027] The VR glasses are configured to detect first position measurement information through the IMU of the VR glasses and send the first position measurement information to the in-vehicle computing unit;

[0028] The in-vehicle computing unit is configured to determine the third position information of the VR glasses in the ground coordinate system based on the first position measurement information and the second position information; determine the target position information of the VR glasses in the ground coordinate system based on the first position information and the third position information, wherein the target position information is used to instruct the in-vehicle computing unit to generate a VR image based on the target position information and display the VR image through the VR glasses.

[0029] In a possible implementation, the method further includes:

[0030] Receive the second position measurement information sent by the IMU of the vehicle, where the second position measurement information is detected by the IMU of the vehicle;

[0031] Determine the corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information;

[0032] The determining the target position information of the VR glasses in the ground coordinate system based on the first position information, the second position information, and the third position information includes:

[0033] Determine the target position information of the VR glasses in the ground coordinate system based on the corrected position information, the second position information, and the third position information.

[0034] In a possible implementation, the second position measurement information includes first acceleration information, and the determining the corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information includes:

[0035] Perform a second integral of the first acceleration information with respect to time to obtain fourth position information;

[0036] Determine fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor at the end of the previous iteration cycle;

[0037] Determine the corrected position information of the GNSS sensor in the ground coordinate system based on the first position information and the fifth position information.

[0038] In a possible implementation, the second position measurement information further includes first rotation angle information of the vehicle body in the ground coordinate system, and the method further includes:

[0039] Determine seventh position information of the GNSS sensor relative to the IMU of the vehicle in the ground coordinate system based on sixth position information of the GNSS sensor relative to the IMU of the vehicle in the body coordinate system and the first rotation angle information;

[0040] The determining the fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor at the end of the previous iteration cycle includes:

[0041] Determine fifth position information based on the fourth position information, the seventh position information, and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor at the end of the previous iteration cycle.

[0042] In a third aspect, a positioning device for a VR glasses is provided. The device is applied to an in-vehicle computing unit, and the device includes:

[0043] A receiving module, configured to receive first position information of a global navigation satellite system (GNSS) sensor of a vehicle in a ground coordinate system, where the first position information is detected by the GNSS sensor; receive second position information of the VR glasses relative to the GNSS sensor in a vehicle body coordinate system of the vehicle sent by a positioning base station in the vehicle; receive first position measurement information detected by an inertial measurement unit (IMU) of the VR glasses sent by the VR glasses, and based on the first position measurement information, determine third position information of the VR glasses in the ground coordinate system;

[0044] A positioning module, configured to determine target position information of the VR glasses in the ground coordinate system based on the first position information, the second position information, and the third position information, where the target position information is used to instruct the in-vehicle computing unit to generate a VR image based on the target position information and display the VR image through the VR glasses.

[0045] In a fourth aspect, an in-vehicle computing unit is provided. The in-vehicle computing unit includes a processor and a memory. At least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the operations performed by the positioning method of the VR glasses as described in the first aspect above.

[0046] In a fifth aspect, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the operations performed by the positioning method of the VR glasses as described in the first aspect above.

[0047] In a sixth aspect, a computer program product is provided. At least one instruction is included in the computer program product, and the instruction is loaded and executed by a processor to implement the operations performed by the positioning method of the VR glasses as described in the first aspect above.

[0048] The beneficial effects brought by the technical solutions provided in the embodiments of the present application are as follows:

[0049] In the technical solutions provided in the embodiments of the present application, by using the relatively high computing power of the in-vehicle computing unit and combining the GNSS sensor of the vehicle, the positioning base station of the vehicle, and the IMU of the VR glasses itself to jointly position the VR glasses, the calculation of the position information of the VR glasses is made more efficient and accurate. Description of the Drawings

[0050] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0051] Figure 1 is a flowchart of a positioning method for a VR glasses provided by an embodiment of the present application;

[0052] Figure 2 is a flowchart of a positioning method for a VR glasses provided by an embodiment of the present application;

[0053] Figure 3 is a schematic structural diagram of a positioning device for a VR glasses provided by an embodiment of the present application;

[0054] Figure 4 is a schematic structural diagram of a VR glasses provided by an embodiment of the present application. Specific embodiments

[0055] To make the objectives, technical solutions and advantages of the present application more clear, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0056] The embodiment of the present application provides a positioning method for a VR glasses, and this method can be implemented by a positioning system. The following will give an exemplary introduction to the implementation scenario of the positioning method for the VR glasses provided by the embodiment of the present application.

[0057] When a user is riding in a vehicle and wears a VR glasses in the vehicle cockpit, in this case, the GNSS (Global Navigation Satellite System) sensor of the vehicle, the IMU (Inertial Measurement Unit) of the vehicle, the positioning base station of the vehicle, the in-vehicle computing unit of the vehicle, and the VR glasses together form a positioning system for the VR glasses to position the VR glasses and obtain the position information of the VR glasses in the ground coordinate system.

[0058] Furthermore, the VR glasses can generate a VR image for display on the display screen of the VR glasses according to the position information. In a possible implementation, as the transportation means moves, the user can feel that he has made corresponding movements in the virtual scene shown in the VR image through the VR glasses. For example, when the transportation means turns left, the user will feel that he turns left in the virtual scene shown in the VR image through the VR glasses. Another example is that when the transportation means accelerates, the user will feel that he accelerates in the virtual scene shown in the VR image through the VR glasses.

[0059] See Figure 1 In the technical solution provided in the embodiment of the present application, the GNSS sensor of the vehicle detects the first position information of the vehicle in the ground coordinate system and sends it to the vehicle-mounted computing unit. The positioning base station in the vehicle detects the second position information of the VR glasses relative to the GNSS sensor in the vehicle body coordinate system of the vehicle and sends it to the vehicle-mounted computing unit. Receive the first position measurement information detected by the inertial measurement unit IMU of the VR glasses sent by the VR glasses, and based on the first position measurement information and the second position information, determine the third position information of the VR glasses relative to the GNSS in the ground coordinate system. Furthermore, based on the first position information and the third position information, determine the target position information of the VR glasses in the ground coordinate system. Furthermore, the vehicle-mounted computing unit can generate a VR image according to the target position information and display the VR image on the display screen of the VR glasses.

[0060] The following describes a positioning method for a VR glasses provided in the embodiment of the present application.

[0061] The embodiment of the present application provides a positioning method for a VR glasses. The method is applied to a positioning system. The positioning system includes the GNSS sensor of the vehicle, the IMU of the vehicle, the positioning base station of the vehicle, the vehicle-mounted computing unit of the vehicle, and the IMU of the VR glasses. See Figure 2 The method may include the following processing steps:

[0062] Step 101, the GNSS sensor of the vehicle sends the first position information of the GNSS sensor in the ground coordinate system to the vehicle-mounted computing unit of the vehicle.

[0063] Wherein, the GNSS sensor is a general term for satellite positioning systems, which is a space-based radio navigation positioning system that can provide users with all-weather three-dimensional coordinates, speeds, and time information at any location on the earth's surface or near-earth space. It can be GPS (Global Positioning System), BDS (Beidou Navigation Satellite System), Galileo satellite navigation system, etc. The embodiment of the present application does not limit the specific satellite positioning system.

[0064] In implementation, the GNSS sensor periodically detects its first position information in the ground coordinate system and sends the first position information to the vehicle-mounted computing unit. Among them, the detection period can be configured by relevant personnel according to actual needs, and can be the same as the detection periods of the vehicle's IMU, the IMU of the VR glasses, and the vehicle's positioning base station for position information. The first position information is a three-dimensional coordinate, including the x-axis coordinate, the y-axis coordinate, and the z-axis coordinate. Here, the x-axis, y-axis, and z-axis can refer to the coordinate axes in the ground coordinate system, and the ground coordinate system can also be referred to as the world coordinate system, the geodetic coordinate system, etc. The embodiments of the present application do not limit this. The x-axis coordinate, y-axis coordinate, and z-axis coordinate in the first position information are respectively denoted as X, Y, and Z.

[0065] Step 102: The positioning base station in the vehicle sends the second position information of the VR glasses relative to the GNSS sensor in the vehicle body coordinate system to the vehicle-mounted computing unit.

[0066] In implementation, the positioning base station can be a binocular vision-based positioning system, which can be set on the A-pillar and C-pillar of the vehicle and can capture the interior space of the vehicle. In addition, the positions of the GNSS sensor and the positioning base station are fixed, that is, the position information of the GNSS sensor in the vehicle body coordinate system is fixed and known. The positioning base station can perform SLAM (Simultaneous Localization and Mapping) processing on the interior space of the vehicle to locate the position information of the VR glasses in the body coordinate system (in the case where the user wears the VR glasses, that is, the position information of the human head in the body coordinate system), and further calculate the second position information of the VR glasses relative to the GNSS sensor in the vehicle body coordinate system. Among them, the x-axis coordinate, y-axis coordinate, and z-axis coordinate in the second position information are respectively denoted as xgg, ygg, and zgg.

[0067] In addition, the positioning base station can send the obtained second position information to the vehicle-mounted computing unit.

[0068] Step 103: The IMU of the VR glasses sends the first position measurement information of the IMU of the VR glasses to the vehicle-mounted computing unit.

[0069] In implementation, the IMU built in the VR glasses can perform position measurement periodically to obtain the first position measurement information. Among them, the first position measurement information includes the second acceleration information of the VR glasses in the ground coordinate system and the second rotation angle information of the VR glasses in the ground coordinate system. Among them, the second acceleration information can be denoted as a xg 、a yg and a zg , and the second rotation angle information can be denoted as rgx, rgy, and rgz.

[0070] In addition, the VR glasses can send the obtained first position measurement information to the vehicle-mounted computing unit.

[0071] Step 104: The vehicle-mounted computing unit determines the third position information of the VR glasses relative to the GNSS in the ground coordinate system based on the first position measurement information and the second position information.

[0072] In implementation, according to the second rotation angle information, the vehicle-mounted computing unit converts the second position information of the VR glasses relative to the GNSS sensor in the vehicle body coordinate system into the third position information of the VR glasses relative to the GNSS in the ground coordinate system through a spatial rectangular coordinate system transformation. The specific calculation method is the common method of spatial rectangular coordinate system transformation and will not be elaborated here. Among them, the third position information can be denoted as Xgg, Ygg, and Zgg.

[0073] Step 105: The vehicle-mounted computing unit determines the target position information of the VR glasses in the ground coordinate system based on the first position information and the third position information.

[0074] In implementation, the vehicle-mounted computing unit can subtract the first position information from the third position information to obtain the target position information of the VR glasses in the ground coordinate system.

[0075] In a possible implementation, there are certain errors in the position information measured by the GNSS sensor, and the position measurement information of the vehicle's IMU can be combined to correct it. Accordingly, the processing can be as follows:

[0076] The vehicle's IMU can perform position measurement periodically, obtain the second position measurement information, and send the second position measurement information to the vehicle-mounted computing unit. Among them, the second position measurement information includes the first acceleration information and the first rotation angle information. Among them, the first acceleration information can be denoted as a xi , a yi and a zi , and the first rotation angle information can be denoted as rix, riy, and riz. Furthermore, the vehicle-mounted computing unit can correct the position information measured by the GNSS sensor based on the second position measurement information to obtain the corrected position information of the GNSS sensor in the ground coordinate system. Furthermore, based on the corrected position information and the third position information, the target position information of the VR glasses in the ground coordinate system is determined.

[0077] The method for determining the corrected position information is described below:

[0078] Integrate the first acceleration information twice with respect to time to obtain the fourth position information. Based on the sixth position information and the first rotation information of the GNSS sensor relative to the vehicle's IMU in the body coordinate system, determine the seventh position information of the GNSS sensor relative to the vehicle's IMU in the ground coordinate system. Based on the fourth position information, the seventh position information, and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration cycle, determine the fifth position information. Based on the first position information and the fifth position information, determine the corrected position information of the GNSS sensor in the ground coordinate system. The x-axis coordinate, y-axis coordinate, and z-axis coordinate of the modified position information are denoted as Xgps, Ygps, and Zgps respectively. Specifically, the calculation formula can be as follows:

[0079] Xgps = A1 * X + B1(XgpsT + xi + Xgi)

[0080] Ygps = A1 * Y + B1(YgpsT + yi + Ygi)

[0081] Zgps = A1 * Z + B1(ZgpsT + zi + Zgi)

[0082] Where, Xgi, Ygi, and Zgi represent the seventh position information of the GNSS sensor relative to the vehicle's IMU in the ground coordinate system. Xgi, Ygi, and Zgi are calculated based on the sixth position (xgi, ygi, zgi) of the GNSS sensor relative to the vehicle's IMU in the body coordinate system and the first rotation information according to the transformation relationship of the space rectangular coordinate system. Among them, xgi, ygi, and zgi are fixed and known.

[0083] A1 and B1 are two weight values, and XgpsT, YgpsT, and ZgpsT represent the first position information received from the GNSS sensor at the beginning of the current iteration cycle. The iteration cycle is T, and the value of T can be configured according to actual needs, such as 100 seconds. The fourth position information is xi, yi, and zi, which are obtained by integrating a xi 、a yi and a zi twice respectively. The following is an explanation of the values of A1 and B1:

[0084] Value method one:

[0085] At the beginning of an iteration cycle, set A1 to 0 and B1 to 1. At the end of this iteration cycle, set A1 to 1 and B1 to 0. At the remaining moments within this iteration cycle, A1 is 0 and B1 is 1.

[0086] Value method two:

[0087] At the moment when an iteration cycle starts, set the value of A1 to 0 and the value of B1 to 1. During the process where the time in this iteration cycle changes from 0 to T, the value of A1 increases from 0 to 1. When the time in this iteration cycle becomes T (the end moment of this iteration cycle), the value of A1 just increases to 1. Correspondingly, during the process where the time in this iteration cycle changes from 0 to T, the value of B1 decreases from 1 to 0. When the time in this iteration cycle becomes T (the end moment of this iteration cycle), the value of B1 just decreases to 0. In addition, at any moment, the sum of A1 and A2 is always 1.

[0088] Based on the above method for correcting the first position information, the calculation of the target position information can be as follows:

[0089] Xg = A1(Xgps - Xgg) + B1((XgpsT - Xgg) + xg)

[0090] Yg = A1(Ygps - Ygg) + B1((YgpsT - Ygg) + yg)

[0091] Zg = A1(Zgps - Zgg) + B1((ZgpsT - Zgg) + zg)

[0092] Among them, the x-axis coordinate, y-axis coordinate, and z-axis coordinate in the target position information are respectively denoted as Xg, Yg, and Zg. xg, yg, and zg are obtained by double integrating the second acceleration information with respect to time. The value settings of A1 and B1 have been described in the above embodiments and will not be elaborated here.

[0093] In a possible implementation, the virtual scene displayed in the VR glasses can be a dinosaur world scene, a mountain and jungle scene, a dream fairyland scene, etc. In the VR glasses, the corresponding relationship between the target position information, the viewing direction, and the virtual coordinates and directions in the virtual scene can be preset. When the VR glasses obtain the target position information and the user's viewing direction changes, the position and direction of the virtual scene displayed by the VR glasses on the display screen can also change accordingly. When the user is taking a vehicle, the scene seen by the user through the VR glasses is like moving in the virtual scene, and the moving speed and direction match those of the vehicle.

[0094] In a possible implementation, when the position indicated by the target position information is in a specified area during the user's movement, additional information corresponding to the specified area can be displayed in the VR glasses, and the additional information includes advertisement information. For example, when the target position information indicates that the user is in a certain scenic area, advertisement and scenic spot introduction and other additional information can be displayed in the VR image.

[0095] In the embodiment of the present application, by obtaining the position information of the user (the target position information of the VR glasses) and the viewing direction (the direction detected by the gyroscope), the coordinates and direction corresponding to the current position and the viewing direction in the virtual map are determined. When the actual position of the user changes, the corresponding coordinates of the user in the virtual scene also change. Therefore, when the user is riding in a vehicle, during the driving process of the vehicle, the user is also equivalent to moving through the virtual scene without the need for the user to control. The user obtains a dual enjoyment of vision and feeling during the movement process, enhancing the interest.

[0096] In the technical solution provided by the embodiment of the present application, the GNSS of the vehicle, the positioning base station, and the IMU of the VR glasses are jointly used to position the VR glasses, and the obtained position information of the VR glasses will also be relatively accurate.

[0097] All the above optional technical solutions can be combined arbitrarily to form optional embodiments of the present disclosure, which will not be elaborated one by one here.

[0098] Based on the same technical concept, the embodiment of the present application also provides a positioning device for a VR glasses, which can be applied to an in-vehicle computing unit. Refer to Figure 3 ., the device may include a receiving module 510 and a processing module 520, where:

[0099] The receiving module 510 is configured to receive the first position information of the GNSS sensor in the ground coordinate system sent by the global navigation satellite system GNSS sensor of the vehicle, where the first position information is detected by the GNSS sensor; receive the second position information of the virtual reality VR glasses relative to the GNSS sensor in the vehicle body coordinate system sent by the positioning base station in the vehicle; receive the first position measurement information detected by the inertial measurement unit IMU of the VR glasses sent by the VR glasses, and based on the first position measurement information, determine the third position information of the VR glasses in the ground coordinate system;

[0100] The positioning module 520 is configured to determine the target position information of the VR glasses in the ground coordinate system based on the first position information, the second position information, and the third position information, where the target position information is used to instruct the in-vehicle computing unit to generate a VR image based on the target position information and display the VR image through the VR glasses.

[0101] In a possible implementation, the processing module 520 is configured to:

[0102] Receive the second position measurement information sent by the IMU of the vehicle, where the second position measurement information is detected by the IMU of the vehicle;

[0103] Determine the corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information;

[0104] Determine the target position information of the VR glasses in the ground coordinate system based on the corrected position information and the third position information.

[0105] In a possible implementation, the processing module 520 is configured to:

[0106] Perform a second integration of the first acceleration information with respect to time to obtain the fourth position information;

[0107] Determine the fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration cycle;

[0108] Determine the corrected position information of the GNSS sensor in the ground coordinate system based on the first position information and the fifth position information.

[0109] In a possible implementation, the processing module 520 is configured to:

[0110] Determine the seventh position information of the GNSS sensor relative to the IMU of the vehicle in the ground coordinate system based on the sixth position information of the GNSS sensor relative to the IMU of the vehicle in the body coordinate system and the first rotation angle information;

[0111] The determining the fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration cycle includes:

[0112] Determine the fifth position information based on the fourth position information, the seventh position information, and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration cycle.

[0113] In the technical solution provided by the embodiments of the present application, the GNSS of the vehicle, the positioning base station, and the IMU of the VR glasses are jointly used to position the VR glasses, and the obtained position information of the VR glasses will be relatively accurate.

[0114] It should be noted that when the positioning device of the VR glasses provided in the above embodiments performs VR glasses positioning, only the division of the above functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the VR glasses is divided into different functional modules to complete all or part of the functions described above. In addition, the positioning device of the VR glasses provided in the above embodiments and the embodiments of the VR glasses positioning method belong to the same concept. For the specific implementation process, please refer to the method embodiments and will not be elaborated here.

[0115] Figure 4 FIG. shows a block diagram of the structure of a VR glasses 600 provided by an exemplary embodiment of the present application.

[0116] The VR glasses 600 includes a processor 601 and a memory 602.

[0117] The processor 601 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 601 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 601 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 601 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 601 may further include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0118] The memory 602 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 602 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 601 to implement the VR glasses positioning method provided in the method embodiments of the present application.

[0119] In some embodiments, the VR glasses 600 may further optionally include: a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602, and the peripheral device interface 603 may be connected through a bus or signal lines. Each peripheral device may be connected to the peripheral device interface 603 through a bus, signal lines, or a circuit board. Specifically, the peripheral devices include: a radio frequency circuit 604, a display screen 605, a camera assembly 606 (not shown in the figure), an audio circuit 607, a positioning assembly 608, and a power supply 609, etc.

[0120] The peripheral device interface 603 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 can be implemented on a separate chip or circuit board, and this embodiment does not limit this.

[0121] The radio frequency circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 604 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 604 converts an electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and so on. The radio frequency circuit 604 can communicate with other VR glasses through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 may further include a circuit related to NFC (Near Field Communication), and this application does not limit this.

[0122] The display screen 605 is used to display VR images, UI (User Interface), etc. The UI may include graphics, text, icons, videos, and any combination thereof.

[0123] The camera component 606 is used to collect images or videos. Optionally, the camera component 606 includes a front camera and a rear camera. Generally, the front camera is arranged on the front panel of the VR glasses, and the rear camera is arranged on the back of the VR glasses. In some embodiments, there are at least two rear cameras, which can be any one of a main camera, a depth camera, a wide-angle camera, and a telephoto camera, so as to realize the function of background blurring by fusing the main camera and the depth camera, panoramic shooting by fusing the main camera and the wide-angle camera, and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera component 606 may further include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. The dual-color temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation under different color temperatures.

[0124] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves of the user and the environment, and convert the sound waves into electrical signals and input them to the processor 601 for processing, or input them to the radio frequency circuit 604 to achieve voice communication. For the purpose of stereo collection or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the VR glasses 600. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker can be a traditional thin film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for uses such as ranging. In some embodiments, the audio circuit 607 may further include a headphone jack.

[0125] The positioning component 608 is used to locate the current geographical location of the VR glasses 600 to achieve navigation or LBS (Location Based Service). The positioning component 608 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, or Russia's Galileo system.

[0126] The power supply 609 is used to supply power to each component in the VR glasses 600. The power supply 609 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When the power supply 609 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0127] In some embodiments, the VR glasses 600 further include one or more sensors 610. The one or more sensors 610 include, but are not limited to, an acceleration sensor 611 and a gyroscope sensor 612.

[0128] The acceleration sensor 611 can detect the magnitudes of accelerations on the three coordinate axes of the coordinate system established with the VR glasses 600. For example, the acceleration sensor 611 can be used to detect the components of the gravitational acceleration on the three coordinate axes. The processor 601 can control the display screen 605 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 611. The acceleration sensor 611 can also be used for collecting game or user movement data.

[0129] The gyroscope 612 can detect the body direction, rotation angle, and acceleration of the VR glasses 600. The gyroscope sensor 612 can cooperate with the acceleration sensor 611 to collect the 3D actions of the user on the VR glasses 600. Based on the data collected by the gyroscope 612, the processor 601 can implement the following functions: motion sensing (such as changing the UI according to the user's tilting operation), image stabilization during shooting, game control, and inertial navigation, etc.

[0130] Those skilled in the art can understand that Figure 4 the structure shown in does not limit the VR glasses 600, and may include more or fewer components than shown in the figure, or combine certain components, or adopt different component arrangements.

[0131] The embodiment of the present application further provides a vehicle-mounted computing unit, which may include a processor and a memory. The non-transitory computer-readable storage medium in the memory is used to store at least one instruction, and the at least one instruction is used to be executed by the processor to implement the positioning method of the VR glasses provided in the method embodiment of the present application.

[0132] In an exemplary embodiment, a computer-readable storage medium is further provided, such as a memory including instructions, and the above instructions can be executed by a processor in the VR glasses to complete the positioning method of the VR glasses in the above embodiments. The computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0133] It should be noted that the information involved in this application (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between VR glasses and other devices, etc.) are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the acceleration information, location information, satellite positioning information, etc. involved in this application are all obtained under full authorization.

[0134] In this application, terms such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions. It should be understood that there is no logical or chronological dependence between "first" and "second", nor are the quantity and execution order limited. It should also be understood that although the following description uses terms such as first and second to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. The meaning of the term "at least one" in this application refers to one or more, and the meaning of the term "multiple" in this application refers to two or more.

[0135] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk, an optical disc, etc.

[0136] The above are only optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A positioning method for a VR glasses, characterized in that, The method is applied to an in-vehicle computing unit, and the method includes: Receiving first position information of a Global Navigation Satellite System (GNSS) sensor of a vehicle in a ground coordinate system sent by the GNSS sensor, where the first position information is detected by the GNSS sensor; Receiving second position information of a virtual reality (VR) headset relative to the GNSS sensor in a vehicle body coordinate system of the vehicle sent by a positioning base station in the vehicle; Receiving first position measurement information detected by an Inertial Measurement Unit (IMU) of the VR headset sent by the VR headset, and determining third position information of the VR headset relative to the GNSS in the ground coordinate system based on the first position measurement information and the second position information; Determining target position information of the VR headset in the ground coordinate system based on the first position information and the third position information, where the target position information is used to instruct the in-vehicle computing unit to generate a VR image based on the target position information and display the VR image through the VR headset.

2. The method according to claim 1, characterized in that The method further includes: Receiving second position measurement information sent by an IMU of the vehicle, where the second position measurement information is detected by the IMU of the vehicle; Determining corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information; The determining the target position information of the VR headset in the ground coordinate system based on the first position information and the third position information includes: Determining the target position information of the VR headset in the ground coordinate system based on the corrected position information and the third position information.

3. The method according to claim 2, wherein The second position measurement information includes first acceleration information, and the determining the corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information includes: Performing a second integral of the first acceleration information with respect to time to obtain fourth position information; Determining fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration cycle; Determining the corrected position information of the GNSS sensor in the ground coordinate system based on the first position information and the fifth position information.

4. The method according to claim 3, wherein The second position measurement information further includes first rotation angle information of the vehicle body in the ground coordinate system, and the method further includes: Determining seventh position information of the GNSS sensor relative to the IMU of the vehicle in the ground coordinate system based on sixth position information of the GNSS sensor relative to the IMU of the vehicle in the vehicle body coordinate system and the first rotation angle information; The determining the fifth position information based on the fourth position information and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor received at the end of the previous iteration cycle includes: Determine the fifth position information based on the fourth position information, the seventh position information, and the position information of the GNSS sensor in the ground coordinate system sent by the GNSS sensor at the end of the previous iteration cycle.

5. A positioning system for a VR glasses, characterized in that, The system includes a GNSS sensor of the vehicle, an IMU of the vehicle, a positioning base station of the vehicle, an in-vehicle computing unit of the vehicle, and a VR headset, where: The GNSS sensor is configured to detect first position information of the GNSS sensor in the ground coordinate system and send the first position information to the in-vehicle computing unit; The positioning base station is configured to determine second position information of the VR headset relative to the GNSS sensor in the vehicle body coordinate system of the vehicle and send the second position information to the in-vehicle computing unit; The VR headset is configured to detect first position measurement information through the IMU of the VR headset and send the first position measurement information to the in-vehicle computing unit; The in-vehicle computing unit is configured to determine third position information of the VR headset relative to the GNSS in the ground coordinate system based on the first position measurement information and the second position information; determine target position information of the VR headset in the ground coordinate system based on the first position information and the third position information, where the target position information is used to instruct the in-vehicle computing unit to generate a VR image based on the target position information and display the VR image through the VR headset.

6. The device according to claim 5, characterized in that, The in-vehicle computing unit is further configured to: receive second position measurement information sent by the IMU of the vehicle, where the second position measurement information is detected by the IMU of the vehicle; Determine corrected position information of the GNSS sensor in the ground coordinate system based on the second position measurement information and the first position information; Determine target position information of the VR headset in the ground coordinate system based on the corrected position information and the third position information.

7. A positioning device for a VR glasses, characterized in that, The device is applied to an in-vehicle computing unit, and the device includes: A receiving module, configured to receive first position information of the GNSS sensor in the ground coordinate system sent by a global navigation satellite system (GNSS) sensor of the vehicle, where the first position information is detected by the GNSS sensor; receive second position information of a virtual reality (VR) headset relative to the GNSS sensor in the vehicle body coordinate system of the vehicle sent by a positioning base station in the vehicle; receive first position measurement information detected by an inertial measurement unit (IMU) of the VR headset sent by the VR headset, and determine third position information of the VR headset in the ground coordinate system based on the first position measurement information; A positioning module, configured to determine target position information of the VR headset in the ground coordinate system based on the first position information, the second position information, and the third position information, where the target position information is used to instruct the in-vehicle computing unit to generate a VR image based on the target position information and display the VR image through the VR headset.

8. A vehicle-mounted computing unit, characterized in that, The in-vehicle computing unit includes a processor and a memory, and at least one instruction is stored in the memory, and the instruction is loaded and executed by the processor to implement the operations performed by the positioning method of the VR glasses according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, At least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the operations performed by the positioning method of the VR glasses according to any one of claims 1 to 4.

10. A computer program product, characterized in that, The computer program product includes at least one instruction, and the instruction is loaded and executed by a processor to implement the operations performed by the positioning method of the VR glasses according to any one of claims 1 to 4.

Images