Processing system and method for XR equipment in vehicle
By deploying an infrared transceiver in the vehicle and installing a reflector on the XR device, the three-degree-of-freedom pose of the XR device is determined using the principle of infrared beam reflection. This solves the problem of inaccurate positioning of XR devices in a moving vehicle environment and achieves high-precision, low-cost positioning.
Patent Information
- Application Number
- CN202411014424.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-15
AI Technical Summary
Existing XR devices are not accurate in positioning in moving vehicle environments. In particular, gyroscopes and accelerometers cannot be used effectively when the vehicle's acceleration changes, causing virtual space content to be out of the user's reach.
By deploying an infrared transceiver in the vehicle and installing a reflector on the XR device, the three-degree-of-freedom pose of the XR device can be determined by the reflection principle of the infrared beam. The location information of the light spot sampled by the receiver can be used for positioning, avoiding reliance on gyroscopes and accelerometers.
It achieves high-precision positioning of XR devices in vehicle-moving environments, reduces positioning costs, and decreases the size and weight of the devices.
Smart Images

Figure CN118962707A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and is related to, but not limited to, a processing system and method for an XR device in a vehicle. Background Art
[0002] With the development of Extended Reality (XR) devices and automotive intelligence, using XR devices to display information in the car is a new trend. However, to realize the use of XR devices to display information in the car, the XR devices need to be positioned.
[0003] The positioning of XR devices must rely on data from gyroscopes and accelerometers. However, cars are mobile environments, and gyroscopes will be disturbed in mobile environments, and accelerometers are basically unusable when the car's acceleration changes. For example, when a car slows down, the XR device will recognize the deceleration and mistakenly think that the user is moving backwards. The content of the virtual space will move directly in the direction forward of the user, and may even be out of the user's reach and visibility.
[0004] Simply put, the application of XR devices in mobile environments does not meet actual needs. Summary of the invention
[0005] The present application provides a processing method for an extended reality XR device in a vehicle, which can determine the three-degree-of-freedom position of the XR device on the vehicle and accurately position it, even in an environment where the vehicle is moving.
[0006] The technical solution of this application is implemented as follows:
[0007] In a first aspect, the present application provides a processing system for an extended reality XR device in a vehicle, the system comprising: a transceiver deployed on the vehicle, the transceiver comprising a transmitter and a receiver, an XR device located in the vehicle, a reflector mounted on the XR device, and a processor connected to the receiver; wherein:
[0008] The transmitter is used to: generate an infrared transmission light beam according to a first frequency;
[0009] The reflector is used to: reflect the infrared emission light beam to obtain an infrared reflection light beam;
[0010] The receiver is used to: receive the infrared reflected light beam, sample the light spot position of the infrared reflected light beam received in the current time period, and obtain the light spot position information in the current time period; and send the light spot position information in the current time period to the processor;
[0011] The processor is used to determine the current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period.
[0012] In a second aspect, the present application provides a processing method for an XR device in a vehicle, wherein a transceiver is deployed in the vehicle, the transceiver includes a transmitter and a receiver, the XR device is located in the vehicle, and a reflector is installed on the XR device; wherein an infrared emission beam generated by the transmitter at a first frequency is reflected by the reflector and transmitted to the receiver; the method includes:
[0013] Sampling the light spot position of the infrared reflected light beam received by the receiver in the current time period to obtain the light spot position information in the current time period;
[0014] The three-degree-of-freedom position of the XR device on the vehicle is determined based on the light spot position information within the current time period.
[0015] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program or instructions are stored. When the computer program or instructions are executed by a processor, the method provided in the first aspect above is implemented.
[0016] In a fourth aspect, the present application further provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor in a computer, the method provided in the first aspect is implemented.
[0017] The present application provides a processing system, method, storage medium and computer program product for an XR device in a vehicle. The system includes: a transceiver deployed on the vehicle, the transceiver includes a transmitter and a receiver, an XR device located in the vehicle, a reflector mounted on the XR device, and a processor connected to the receiver; wherein: the transmitter is used to: generate an infrared transmission beam according to a first frequency; the reflector is used to: reflect the infrared transmission beam to obtain an infrared reflection beam; the receiver is used to: receive the infrared reflection beam, sample the light spot position of the infrared reflection beam received in the current time period, and obtain the light spot position information in the current time period; send the light spot position information in the current time period to the processor; the processor is used to: determine the current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period.
[0018] First, the solution of the present application is to determine the current three-degree-of-freedom posture of the XR device on the vehicle through an infrared transceiver deployed in the vehicle and a reflector installed on the XR device, based on the reflection principle of the infrared light beam on the reflector. The three-degree-of-freedom posture of the XR device on the vehicle is determined based on the light spot position information received by the receiver in the current time period. Since the light spot position based on the light beam reflection principle has the characteristics of high precision, the positioning accuracy of the XR device is also high, and it has the characteristics of accurate positioning. Secondly, when determining the three-degree-of-freedom posture of the XR device, it does not rely on gyroscopes and accelerometers. Even if the vehicle is in a moving state, it has no effect on the transmission of the infrared light beam, so even in an environment where the vehicle is moving, the positioning is still accurate. Thirdly, this solution realizes the positioning of the three-degree-of-freedom posture through transceivers and reflectors, and has the characteristics of low implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 A schematic diagram of a first optional structure of a processing system for an XR device in a vehicle provided in an embodiment of the present application;
[0020] Figure 2 A second optional structural diagram of a processing system for an XR device in a vehicle provided in an embodiment of the present application;
[0021] Figure 3 A third optional structural diagram of a processing system for an XR device in a vehicle provided in an embodiment of the present application;
[0022] Figure 4 A schematic diagram of a first optional structure of a processing method for an XR device in a vehicle provided in an embodiment of the present application;
[0023] Figure 5 A second optional structural diagram of the processing method of the XR device in the vehicle provided in the embodiment of the present application;
[0024] Figure 6 A third optional structural diagram of the processing method of the XR device in the vehicle provided in the embodiment of the present application;
[0025] Figure 7 A fourth optional structural diagram of the processing method of the XR device in the vehicle provided in the embodiment of the present application;
[0026] Figure 8 A fifth optional structural diagram of the processing method of the XR device in the vehicle provided in the embodiment of the present application;
[0027] Fig. 9 A sixth optional structural diagram of the processing method of the XR device in the vehicle provided in the embodiment of the present application;
[0028] Fig.10 An optional schematic diagram of 3dof provided in an embodiment of the present application;
[0029] Fig.11 An optional schematic diagram of 6dof provided in an embodiment of the present application;
[0030] Fig.12 An optional structural diagram of the basic principle of the processing solution of the XR device in the vehicle provided in the embodiment of the present application;
[0031] Fig.13 An optional structural diagram of the functional architecture of the whole vehicle provided in the embodiment of the present application;
[0032] Fig.14 An optional structural diagram of a processing scenario of an XR device in a vehicle provided in an embodiment of the present application;
[0033] Fig.15 Another optional structural diagram of a processing scenario of an XR device in a vehicle provided in an embodiment of the present application;
[0034] Fig.16 A schematic diagram of an optional structure of a reflector provided in an embodiment of the present application;
[0035] Fig.17 An optional structural schematic diagram of the angle range corresponding to the transmitter and the receiver in the vehicle provided in the embodiment of the present application;
[0036] Fig.18 An optional structural schematic diagram of the deployment positions of a transmitter and a receiver in a vehicle provided in an embodiment of the present application;
[0037] Fig.19 An optional structural schematic diagram of the displacement between two positions on the receiver provided in an embodiment of the present application;
[0038] Fig. 20 Another optional structural schematic diagram of the displacement between two positions on the receiver provided in the embodiment of the present application;
[0039] Fig.21 A schematic diagram of another optional structure of the displacement between two positions on the receiver provided in the embodiment of the present application;
[0040] Fig. 22 An optional flowchart of the function area state switching process provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.
[0042] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0043] In the following description, the terms "first\second\third" are used only as examples to distinguish different objects, and do not represent a specific order for the objects, nor do they have a limitation on the order of precedence. It is understandable that "first\second\third" can be interchanged with a specific order or order of precedence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.
[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.
[0045] Below, various embodiments of the processing system, method, storage medium, and program product of the XR device in the vehicle provided by the embodiments of the present application are described.
[0046] In a first aspect, this embodiment provides a processing system for an XR device in a vehicle, referring to Figure 1 As shown, the processing system of the XR device includes: a transceiver 10 deployed on the vehicle, the transceiver 10 includes a transmitter 101 and a receiver 102, an XR device 20 located in the vehicle, a reflector 201 installed on the XR device 20, and a processor 30 connected to the receiver 102; wherein:
[0047] The transmitter 101 is used to: generate an infrared transmission light beam 101A according to a first frequency;
[0048] The reflector 201 is used to: reflect the infrared emission light beam to obtain an infrared reflection light beam 101B;
[0049] The receiver 102 is used to: receive the infrared reflected light beam 101B, sample the light spot position of the infrared reflected light beam 101B received in the current time period, and obtain the light spot position information in the current time period; and send the light spot position information in the current time period to the processor 30;
[0050] The processor 30 is used to determine the current three-degree-of-freedom posture of the XR device 20 on the vehicle based on the light spot position information in the current time period.
[0051] in:
[0052] XR devices may include, but are not limited to, virtual reality (VR) devices, augmented reality (AR) devices, and mixed reality (MR) devices. Exemplarily, XR devices may be AR glasses, AR helmets, and the like.
[0053] Generally, the posture of an XR device can include a three-degree-of-freedom posture and a six-degree-of-freedom posture. Among them, the three-degree-of-freedom posture includes: the rotation angle of the object on the X, Y, and Z axes. The six-degree-of-freedom posture includes: the rotation angle on the X, Y, and Z axes, and the coordinate position on the X, Y, and Z axes.
[0054] The embodiments of the present application are applied to a three-degree-of-freedom posture scenario, that is, the position of the XR device is considered to be fixed and known, and when the angle of the XR device changes, the three rotation angles of the XR device are located.
[0055] The embodiments of the present application do not limit the specific position of the XR device in the vehicle, which can be determined according to actual needs. For example, in the case where the XR device is AR glasses, the XR device can be located at the driver's eyes. Generally, during the process, the driver sits in the driving seat, and the position of the XR device can be considered to be fixed, and only the angle is adjusted and changed.
[0056] The embodiments of the present application do not limit the deployment location of the transceiver in the vehicle and the number of transceivers, and can be configured according to actual needs.
[0057] A reflector refers to a mirror that has the ability to reflect infrared light.
[0058] In some embodiments, the reflector is a plane reflector.
[0059] In other embodiments, the reflector is a multi-faceted reflector.
[0060] The embodiment of the present application does not limit the installation position of the reflector on the XR device, and can be configured according to actual needs. For example, the reflector can be installed at the nose pad position of the AR glasses.
[0061] In practice, the reflector can be installed in a fixed or detachable manner on the XR device, and the specific implementation is configured according to actual needs.
[0062] The processor refers to a chip or chip system with relevant data processing capabilities. The processor is connected to the receiver, and the processor can receive the light spot position information within the current time period sent by the receiver.
[0063] In some embodiments, the processor includes: a cockpit processor of the vehicle, or a processor of the XR device. That is, the processor can be integrated into the cockpit processor or the processor of the XR device.
[0064] In other embodiments, the processor may be deployed separately from the vehicle's cockpit processor and the XR device's processor.
[0065] For the transmitter 101: it is used to generate an infrared transmission light beam 101A according to a first frequency.
[0066] The embodiment of the present application does not limit the specific value of the first frequency, and can be configured according to actual needs. The transmitter 101 generates an infrared emission beam 101A according to the first frequency, and the generated infrared emission beam 101A is transmitted to the reflector 201 .
[0067] Regarding the reflector 201: it is used to reflect the infrared emission light beam to obtain an infrared reflection light beam 101B.
[0068] The reflector 201 can reflect the infrared light beam transmitted to the surface of the reflector 201 to obtain an infrared reflected light beam 101B. In practice, the infrared light beam transmitted to the surface of the reflector 201 may include, in addition to the infrared emission light beam 101A of the first frequency, infrared emission light beams of other frequencies transmitted by other transmitters, and infrared emission light beams transmitted to the surface of the reflector 201 in sunlight.
[0069] For the receiver 102: it is used to receive the infrared reflected light beam 101B, sample the light spot position of the infrared reflected light beam 101B received in the current time period, and obtain the light spot position information in the current time period; and send the light spot position information in the current time period to the processor 30.
[0070] The embodiment of the present application does not limit the specific duration of the current time period, and can be configured according to actual needs. For example, the duration of the current time period can be 0.1 seconds.
[0071] Light spot position, used to indicate the position of the light spot on the receiver.
[0072] The light spot position information may include: the light spot position and the acquisition time of the light spot position.
[0073] Since the receiver 102 is connected to the processor, the receiver 102 can send the light spot position information in the current time period to the processor 30 .
[0074] For the processor 30: used to determine the current three-degree-of-freedom position of the XR device 20 on the vehicle based on the light point position information in the current time period.
[0075] In practice, the positions of the transceiver 10 and the XR device 20 are fixed, that is, the distance between the transceiver 10 and the XR device 20 is known. In this way, after obtaining the light spot position information in the current time period, the light spot displacement can be further determined, and the current three-degree-of-freedom posture of the XR device 20 on the vehicle can be determined based on the light spot displacement and the distance between the transceiver 10 and the XR device 20.
[0076] The processing system of an XR device in a vehicle provided in an embodiment of the present application includes: a transceiver deployed on the vehicle, the transceiver includes a transmitter and a receiver, an XR device located in the vehicle, a reflector installed on the XR device, and a processor connected to the receiver; wherein: the transmitter is used to: generate an infrared transmission beam according to a first frequency; the reflector is used to: reflect the infrared transmission beam to obtain an infrared reflection beam; the receiver is used to: receive the infrared reflection beam, sample the light spot position of the infrared reflection beam received in a current time period, and obtain the light spot position information in the current time period; send the light spot position information in the current time period to the processor; the processor is used to: determine the current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period.
[0077] First, the solution of the present application is to determine the current three-degree-of-freedom posture of the XR device on the vehicle through an infrared transceiver deployed in the vehicle and a reflector installed on the XR device, based on the reflection principle of the infrared light beam on the reflector. The three-degree-of-freedom posture of the XR device on the vehicle is determined based on the light spot position information received by the receiver in the current time period. Since the light spot position based on the light beam reflection principle has the characteristics of high precision, the positioning accuracy of the XR device is also high, and it has the characteristics of accurate positioning. Secondly, when determining the three-degree-of-freedom posture of the XR device, it does not rely on gyroscopes and accelerometers. Even if the vehicle is in a moving state, it has no effect on the transmission of the infrared light beam, so even in an environment where the vehicle is moving, the positioning is still accurate. Thirdly, this solution realizes the positioning of the three-degree-of-freedom posture through transceivers and reflectors, and has the characteristics of low implementation cost.
[0078] In some embodiments, reference Figure 2 As shown in the figure, the processor 30 is also connected to the XR device 20; the processor 30 is also used for:
[0079] Based on the current three-degree-of-freedom posture of the XR device 20 on the vehicle, determine whether the XR device 20 satisfies a first condition; when it is determined that the XR device 20 satisfies the first condition, determine a target rendering content and a display reference center point; transmit the target rendering content and the display reference center point to the XR device, so as to display the target rendering content on a display screen of the XR device with the display reference center point as the center.
[0080] The first condition includes: the rotation angle within the first time period is less than a first angle threshold.
[0081] The duration of the first time period and the first angle threshold may be determined according to actual needs. For example, the first condition may include: the total head turning angle within 0.5 seconds (S) is less than 5 degrees.
[0082] The target rendering content refers to the content to be displayed by the XR device. The display reference center point refers to the center point of the display screen.
[0083] Determining the target rendering content may refer to a process of generating the target rendering content. The embodiment of the present application does not limit the specific rendering process and may be configured according to actual needs.
[0084] Here, the display screen of the XR device may include but is not limited to: a physical electronic screen or a virtual screen.
[0085] In this way, the processor can also determine and process the target rendering content. When the processor 30 is not deployed in the XR device, the rendering load of the XR device can be reduced, and the processing efficiency and response efficiency of the XR device can be improved. Deploying the processor in the cockpit processor has the characteristic of simple implementation.
[0086] In some embodiments, reference Figure 3 The content shown is in the case where multiple groups of transceivers 10 are deployed on the vehicle, and a group of transceivers 10 includes a transmitter 101 and a receiver 102;
[0087] Each group of transceivers 10 in the plurality of groups of transceivers is deployed at a different position on the vehicle;
[0088] The rendering content corresponding to each group of transceivers 10 in the multiple groups of transceivers is different;
[0089] The first frequency corresponding to each group of transceivers 10 in the multiple groups of transceivers has a different value;
[0090] The processor is also used to: determine that the target rendering content is the rendering content corresponding to the target transceiver in the multiple groups of transceivers; the target transceiver includes: a transceiver corresponding to the receiver that receives the infrared reflected light beam; or a transceiver corresponding to the receiver that receives the infrared reflected light beam of the first frequency.
[0091] In practice, the deployment positions of multiple sets of transceivers can be configured according to actual needs. In practice, different transceivers can be configured according to different functional areas. For example, a set of transceivers is configured for the driver's driving information display functional area; a set of transceivers is configured for the driver's vehicle control information display functional area; and a set of transceivers can also be configured for the co-pilot's entertainment information display functional area.
[0092] When multiple transceivers are deployed on a vehicle, the operating frequencies (first frequencies) corresponding to each transceiver group may be the same or different. If the first frequencies corresponding to each transceiver group are different, a receiver may receive and process only the infrared reflected light beam sent by the transmitter corresponding to the receiver according to the filtering function. In this way, the receiver may filter out infrared light beams of other frequencies sent by other transmitters, further improving the accuracy of the received light spot position, thereby improving the accuracy of positioning.
[0093] When multiple transceivers are deployed on the vehicle, one set of transceivers corresponds to one rendering content, but the three-degree-of-freedom posture of the XR device at a time is fixed, that is, it can only correspond to one set of transceivers. Therefore, the target rendering content should be the rendering content of the target transceiver. The target transceiver here can be the transceiver corresponding to the receiver that receives the infrared reflected beam; or the transceiver corresponding to the receiver that receives the infrared reflected beam of the first frequency.
[0094] It can be seen that the solution of displaying different rendering content in different functional areas through multiple groups of transceivers further improves the application experience of XR devices.
[0095] For example, the XR device is AR glasses, the driver wears AR glasses, and multiple transceivers are deployed in the vehicle. If the driver looks straight ahead, the transceiver in front of him is triggered, and the rendering content (driving information) corresponding to the transceiver in front of him is displayed accordingly.
[0096] In some embodiments, only one set of transceivers may be deployed on the vehicle.
[0097] In some embodiments, when two sets of transceivers are deployed on the vehicle, a first set of transceivers of the two sets of transceivers is deployed in front of the driving position in the vehicle;
[0098] Correspondingly, the rendering content corresponding to the first group of transceivers includes: driving information of the vehicle;
[0099] The second set of transceivers of the two sets of transceivers is disposed above or below the central rearview mirror of the vehicle;
[0100] Correspondingly, the target rendering content corresponding to the second group of transceivers includes: vehicle control information of the vehicle.
[0101] Driving information may include, but is not limited to, vehicle speed, vehicle lights, fuel level or battery level, and other information that requires attention during driving.
[0102] Vehicle control information may include, but is not limited to, air conditioning control, heating control and other control information.
[0103] In practice, a third group of transceivers may be deployed, which will not be described in detail here.
[0104] In a second aspect, this embodiment provides a processing method for an XR device in a vehicle.
[0105] A transceiver is deployed in the vehicle, the transceiver includes a transmitter and a receiver, the XR device is located in the vehicle, and a reflector is installed on the XR device; wherein the infrared emission light beam generated by the transmitter at a first frequency is reflected by the reflector and transmitted to the receiver.
[0106] The method provided in the embodiment of the present application is described below. Figure 4 The processing method of the XR device in the vehicle may include but is not limited to Figure 4 S401 and S402 shown.
[0107] S401. Sample the light spot position of the infrared reflection light beam received by the receiver in the current time period to obtain the light spot position information in the current time period.
[0108] The processing system of the XR device in the vehicle samples the light spot position of the infrared reflected light beam received by the receiver in the current time period of the receiver to obtain the light spot position information in the current time period.
[0109] The receiver can convert the light spot where the infrared reflected light beam falls on the receiver into an electrical signal, sense the position of the light spot, and record the acquisition time of the light spot position in real time to obtain the light spot position information of multiple light spots in the current time period.
[0110] S402: Determine a current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period.
[0111] The processing system of the XR device in the vehicle determines the current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period through a processor.
[0112] The specific implementation of S402 may refer to the detailed description of the processor side, which will not be described in detail here.
[0113] An embodiment of the present application provides a method for processing an XR device in a vehicle, wherein a transceiver is deployed in the vehicle, the transceiver includes a transmitter and a receiver, the XR device is located in the vehicle, and a reflector is installed on the XR device; wherein an infrared emission light beam generated by the transmitter at a first frequency is transmitted to the receiver after being reflected by the reflector; the method comprises: sampling the light spot position of the infrared reflection light beam received by the receiver in a current time period to obtain the light spot position information in the current time period; based on the light spot position information in the current time period, determining the current three-degree-of-freedom posture of the XR device on the vehicle.
[0114] Regarding the processing method of the XR device in the vehicle, first of all, this method is to use the infrared transceiver deployed in the vehicle and the reflector installed on the XR device, based on the reflection principle of the infrared light beam on the reflector, to determine the current three-degree-of-freedom posture of the XR device on the vehicle. The three-degree-of-freedom posture of the XR device on the vehicle is determined based on the light spot position information received by the receiver in the current time period. Since the light spot position based on the light beam reflection principle has the characteristics of high precision, the positioning accuracy of the XR device is also high, and it has the characteristics of accurate positioning. Secondly, when determining the three-degree-of-freedom posture of the XR device, it does not rely on the gyroscope or accelerometer. Therefore, even if the vehicle is in a moving state, it has no effect on the transmission of the infrared light beam, so even in an environment where the vehicle is moving, the positioning is still accurate. Thirdly, this solution realizes the positioning of the three-degree-of-freedom posture through transceivers and reflectors, and has the characteristics of low implementation cost.
[0115] Next, the execution timing of sampling the light spot position of the infrared reflection light beam received by the receiver in the current time period in S401 to obtain the light spot position information in the current time period is described.
[0116] refer to Figure 5 As shown in the content, the process may include but is not limited to the following S4011 and S4012.
[0117] S4011. Determine whether the infrared reflected light beam received by the receiver within the current time period satisfies a second condition.
[0118] The second condition is used to indicate that the time for the receiver to receive the infrared reflected light beam exceeds a first time threshold within the current time period.
[0119] The embodiment of the present application does not specifically limit the value of the first time threshold, and can be configured according to actual needs. For example, the first time threshold can be 0.1 seconds.
[0120] When the receiver receives the red dot for more than 0.1s in the unactivated state, it means that the user begins to move roughly toward the functional area and there may be a need to activate the functional area, so the functional area enters the standby state from the unactivated state.
[0121] S4012. When the infrared reflected light beam received by the receiver in the current time period satisfies the second condition, sampling the light spot position of the infrared reflected light beam received by the receiver in the current time period is performed to obtain the light spot position information in the current time period.
[0122] The second condition is set to activate the sampling of the light spot position. In this way, when the second condition is not met, the receiver can reduce the sampling frequency and enter the power saving mode; when the second condition is met, the receiver enters the standby state and the rated working state. This can reduce power consumption.
[0123] In some embodiments, the processing method of the XR device in the vehicle provided by the embodiments of the present application can also filter the received infrared reflected light beam.
[0124] refer to Figure 6 The process may include but is not limited to the following S403.
[0125] S403, filtering the infrared light beam received by the receiver in the current time period; and obtaining the infrared reflected light beam of the first frequency in the current time period.
[0126] The receiver can filter out interference from infrared sunlight through high-pass filtering, and can then filter out interference from infrared beams of other transceivers based on band-pass filtering, thereby obtaining an infrared reflection beam of the first frequency of the receiver.
[0127] Correspondingly, S401 may be implemented as follows: sampling the light spot position of the infrared reflection light beam of the first frequency received by the receiver in the current time period to obtain the light spot position information in the current time period.
[0128] In this way, filtering can reduce the interference of other infrared beams, improve the accuracy of the light spot position, and further improve the accuracy of XR device positioning.
[0129] Next, the process of determining the current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period in S402 is described.
[0130] refer to Figure 7 As shown in the content, the process may include but is not limited to the following S701 to S704.
[0131] S701. Determine a total displacement within the current time period based on the light spot position information within the current time period.
[0132] There are multiple light spot positions in the current time period, and the sum of the displacements of the multiple light spot positions is the total displacement in the current time period.
[0133] S702: Project the total displacement in the current time period to the horizontal axis direction, the vertical axis direction and the vertical axis direction respectively to obtain the horizontal total displacement, the vertical total displacement and the vertical total displacement in the current time period.
[0134] The horizontal axis, vertical axis and vertical axis here are equivalent to the X-axis, Y-axis and Z-axis.
[0135] The total displacement is projected onto the three axes to obtain the total displacement in three directions, namely, the horizontal total displacement, the vertical total displacement, and the vertical total displacement in the current time period.
[0136] S703: Divide the total horizontal displacement, the total vertical displacement, and the total vertical displacement in the current time period by the reference radius respectively to obtain quotients to be determined as three rotation angles of the XR device on the vehicle.
[0137] The reference radius is the distance between the XR device and the transceiver.
[0138] The quotient of the total horizontal displacement in the current time period and the reference radius is determined as the horizontal rotation angle of the XR device on the vehicle; the quotient of the total vertical displacement in the current time period and the reference radius is determined as the vertical rotation angle of the XR device on the vehicle; the quotient of the total vertical displacement in the current time period and the reference radius is determined as the vertical rotation angle of the XR device on the vehicle.
[0139] S704: Determine that the current three-degree-of-freedom posture of the XR device on the vehicle is the three rotation angles.
[0140] It can be seen that determining the three-degree-of-freedom posture of the XR device by means of total displacement is simple to implement and has high accuracy.
[0141] Next, the process of determining the total displacement in the current time period based on the light spot position information in the current time period in S701 is described.
[0142] refer to Figure 8 As shown in the content, when the light spot position information includes M light spot positions and the acquisition time corresponding to each of the multiple light spot positions, the process may include but is not limited to the following S7011 to S7014.
[0143] S7011. Based on the acquisition time corresponding to each of the M light spot positions, the M light spots are sorted in the order of the acquisition time to obtain a light spot sequence.
[0144] The light spots in the light spot sequence are arranged in the order of acquisition time.
[0145] S7012. For every two adjacent light spots in the light spot sequence, determine the displacement between each two adjacent light spots based on the light spot positions of each two adjacent light spots to obtain N displacements.
[0146] Among them, the M is greater than 1; the N is greater than the M.
[0147] In the receiver, for the middle light spot, the displacement between every two adjacent light spots is directly from the light spot at the previous moment to the light spot at the next moment. For the light spots on both sides of the receiver, two corresponding displacements may be obtained, one is from the light spot at the previous moment to the light spot at the next moment, and the other is from the light spot at the previous moment to the edge of the receiver, and then from the edge of the other side of the receiver to the light spot at the next moment.
[0148] In practice, there may be many light spots. Since there may be two displacements of the light spots on both sides of the receiver, the number of N displacements obtained is greater than the number of M light spots.
[0149] S7013. Delete the displacements greater than the displacement threshold among the N displacements to obtain M-1 target displacements.
[0150] The present application embodiment does not specifically limit the displacement threshold, and can be configured according to actual needs. For example, the displacement threshold can be the sum of the mode value of N displacements and the first floating value, or the displacement threshold can be the sum of the average value of N displacements and the first floating value.
[0151] In simple terms, the obviously larger displacements among the N displacements are deleted, thereby obtaining M-1 target displacements. This is equivalent to deleting the redundant erroneous displacements in the light spots on both sides of the receiver, thereby obtaining M-1 target displacements (correct displacements) between each two adjacent light spots.
[0152] S7014. Determine that the total displacement in the current time period is the sum of the M-1 target displacements.
[0153] The total displacement in the current time period can be obtained by summing the displacements of M-1 targets.
[0154] In this way, when determining the total displacement, the directionality of the displacement of the light spots on both sides of the receiver is taken into account, and the erroneous displacement is deleted by means of the displacement threshold, which improves the accuracy of the total displacement and further improves the accuracy of XR device positioning.
[0155] The processing method of the XR device in the vehicle provided in the embodiment of the present application may also include displaying the rendered content. Fig. 9 As shown in the content, the process may include but is not limited to the following S404 to S406.
[0156] S404: Determine whether the XR device meets a first condition based on the current three-degree-of-freedom posture of the XR device on the vehicle.
[0157] The first condition is used to indicate that the rotation angle within the first time period is less than a first angle threshold.
[0158] The implementation of S404 may refer to the detailed description of the processor side above, which will not be described in detail here.
[0159] S405: When it is determined that the XR device satisfies the first condition, determine a target rendering content and a display reference center point.
[0160] The implementation of S405 may refer to the detailed description of the processor side above, which will not be described in detail here.
[0161] S406: Transmit the target rendering content and the display reference center point to the XR device, so as to display the target rendering content on a display screen of the XR device with the display reference center point as the center.
[0162] The implementation of S406 may refer to the detailed description of the processor side above, which will not be described in detail here.
[0163] Next, the process of determining the target rendering content in S405 is described.
[0164] If the number of the transceivers is one group, the target rendering content is determined to be the rendering content corresponding to the transceiver.
[0165] If there are multiple groups of transceivers, the target rendering content is determined to be the rendering content corresponding to the target transceiver in the multiple groups of transceivers; the target transceiver includes: a transceiver corresponding to the receiver that receives the infrared reflected light beam; or a transceiver corresponding to the receiver that receives the infrared reflected light beam of the first frequency.
[0166] The following describes a processing solution for the XR device in a vehicle provided in an embodiment of the present application through a specific scenario.
[0167] With the development of XR (a general term for VR, AR, and MR) devices and automotive intelligence, using XR devices to display information in the car is a foreseeable trend in the future. However, the problem that must be avoided when using XR devices is the positioning of the device in the virtual space.
[0168] Traditional XR devices are designed for users to use in a static space relative to the ground. They use external laser positioning, external / internal image recognition and other technologies, supplemented by the combination of accelerometers and gyroscopes to provide three degrees of freedom (3dof) or 6dof positioning for the device.
[0169] 3dof refers to the rotation angle of the 3-dimensional rectangular coordinate axis formed by xyz. Fig.10 As shown in the figure, 3dof can be understood as the ability to detect plane rotation angle 1001, vertical pitch angle 1002, and oblique tilt angle 1003. 6dof refers to the addition of the three rotation directions of the above 3dof and the coordinate positions in the three directions of xyz. For example, refer to Fig.11 The content shown in Fig.10 In addition to the three displayed angles, there is also the ability to translate forwards and backwards 1004, left and right 1005, and up and down 1006.
[0170] Whether it is 3dof or 6dof, the assistance of gyroscopes and accelerometers is indispensable, but the car is a mobile environment, the gyroscope will be disturbed in it, and the accelerometer is basically unusable due to the change in the car's acceleration. The specific effect is that when deceleration occurs, the device will recognize the deceleration and mistakenly think that the user is moving backwards. The objects in the virtual space will move directly in the direction forward of the user and quickly leave the user's reach and visibility.
[0171] In view of the above problems, the solutions of related technologies can include two solutions. Solution 1 is to link the accelerometer of the VR device with the accelerometer of the vehicle to eliminate the acceleration effect caused by the movement of the vehicle, thereby achieving 6dof positioning on the vehicle. Solution 2 is to directly use AR without positioning capability to link with the cockpit.
[0172] However, the above two types still use traditional XR devices to adapt to the vehicle environment and are not devices developed specifically for the vehicle environment. The former's headset is large and heavy because it is responsible for rendering the virtual space, and the latter's AR glasses do not have positioning capabilities and are actually just a head-mounted display.
[0173] Considering that the car is a relatively small environment and the user's range of activities is limited, the relatively backward 3dof on traditional XR devices becomes economical and usable. This embodiment proposes a lightweight positioning solution that does not use a gyroscope or accelerometer and leaves positioning and rendering to the cockpit to achieve 3dof positioning for XR devices in the car.
[0174] This embodiment relies on an infrared pulse light transmitter mounted on the vehicle to send an infrared pulse light beam to the position of the XR device at a certain frequency (equivalent to the above-mentioned first frequency), which is received by a receiver after being reflected by a multi-faceted reflector mounted on the XR device. The receiver converts the reflected pulse light (equivalent to the above-mentioned infrared reflected light beam) into an electrical signal, and determines the rotation angle of the XR device based on the change in the position where the signal falls on the surface of the receiver (equivalent to the above-mentioned light spot position information).
[0175] The basic principle of the processing scheme of the XR device in the vehicle includes: the position of the infrared light receiving point received on the receiver (hereinafter referred to as the red dot equivalent to the above light point) depends on the relative angle between the normal of the reflecting surface and the incident infrared light, as well as the distance between the reflecting point and the red dot. Therefore, the displacement distance is determined by the rotation angle of the reflecting surface and the distance from the red dot to the reflecting point (hereinafter referred to as the distance between the two points). When the distance between the two points is relatively fixed, the rotation angle of the reflecting surface can be calculated by changing the position of the receiving point.
[0176] refer to Fig.12 As shown, the pulse infrared beam transmitter 1201 emits infrared light, which is transmitted to the receiver 1203 after being emitted by the reflective surface 1202. It can be seen that the normal of the reflective surface is 1204, the incident angle is α, and the exit angle is β.
[0177] The functional architecture of the entire vehicle is described below.
[0178] refer to Fig.13 As shown in the figure, the functional architecture of the whole vehicle includes: a vehicle cabin 1301, a vehicle chip 1302 (equivalent to the above-mentioned processor) and an XR device 1303. Among them, N infrared beam transmitters 13011 and N infrared beam receivers 13012 are deployed in the vehicle cabin 1301.
[0179] The infrared beam transmitter 13011 is used to transmit a light beam, and the transmitted light beam is reflected by the multi-faceted reflector 13031 of the XR device 1303 to form a reflected light beam; the infrared beam receiver 13012 receives the reflected light beam and records the light spot movement information; the infrared beam receiver 13012 sends the light spot movement information to the vehicle chip 1302. The vehicle chip 1302 performs positioning calculation 13021, virtual space rendering 13022, and video signal output 13023. The vehicle chip 1302 transmits the video signal to the video signal receiving module 13032 of the XR device 1303 for display.
[0180] The general idea of this embodiment is to determine the rotation of the XR device by the displacement of the red dot on the receiver. To realize this idea, the following technologies are mainly included.
[0181] Technology 1. The multi-faceted reflector on the XR device enables the receiver to always receive the red dot.
[0182] Since the distance between the two points is relatively long, when the reflective surface undergoes a very small rotation, the displacement distance of the projected red dot will be very long, which means that the receiver needs to have a large area to always receive the red dot (equivalent to the above-mentioned light spot), which is obviously unrealistic in the space inside the car. Therefore, in this embodiment, many tiny reflective surfaces will be used to form a spherical multi-faceted reflector to ensure that the red dot is always within the receiver range.
[0183] refer to Fig.14 indivual Fig.15 The content shown, Fig.14 It includes scene 1 and scene 2. Fig.15 The scene 3 and scene 4 are included. A scene includes: receiver 1401, reflection surface 1 1402, reflection surface 2 1403, normal 1404, emission beam 1405 and reflection beam 1406. From scene 1 to scene 4, the reflection surface rotates counterclockwise. From scene 1 to scene 2, the receiving point moves from the center of the receiver to the edge of the receiver. When it reaches scene 3, the red dot exceeds the receiver and continues to rotate to scene 4. The reflection point moves from reflection surface 1 to reflection surface 2. The special reflection angle of reflection surface 2 makes the red dot return to the edge of the receiver.
[0184] As can be seen from the above, multiple tiny reflective surfaces are placed at a certain angle to form a reflective sphere and placed on the XR device (such as Fig.16 As shown in the figure, the more reflective surfaces there are, the shorter the existence time of scene 3 is, which can ensure that the red dot is still on the receiver most of the time when the XR device rotates at a large angle, and then the total movement distance of the red dot can be calculated, and then the rotation angle and speed of the XR device can be calculated.
[0185] The deployment of reflectors can refer to Fig.16A multi-faceted reflector 1602 is deployed on the AR glasses 1601. The magnification structure of the multi-faceted reflector 1602 can be referred to Fig.16 The multi-faceted mirror magnification structure 1603 is shown.
[0186] In this way, the business center of XR device positioning can be transferred from the XR device to the more powerful vehicle intelligent cockpit. The XR device is only responsible for displaying the scene rendered by the cockpit without calculating the virtual space, which greatly reduces the size and weight of the device.
[0187] Technology 2: Multiple receivers and transmitters are configured in the car so that the position of the XR device can still be identified when a large angle rotation occurs, and different functional areas can be distinguished.
[0188] Despite the use of a reflective spherical structure, if the XR device is rotated too far, the light beam emitted by the transmitter may still not be reflected back to the receiver. Therefore, a pair of transmitters and receivers only has a specific working range, and can only be recognized when the user is roughly facing the transmitter, that is, when the light beam from the transmitter enters the reflective angle.
[0189] For the angular range of the transmitter and receiver in the vehicle, refer to Fig.17 As shown, transmitter 1 1701, receiver 1 1702, transmitter 2 1703, receiver 2 1704, and XR device 1705 can be deployed. The reflective angle range corresponding to transmitter 1 1701 and receiver 1 1702 is range 1 1706; the reflective angle range corresponding to transmitter 2 1703 and receiver 2 1704 is range 2 1707.
[0190] Multiple sets of transmitters and receivers can also be installed in the car, so that functional areas that can display more different information can be defined, and different logic can be performed in each functional area by using the changes in the rotation angle of the XR device.
[0191] For the deployment locations of the transmitter and receiver in the vehicle, refer to Fig.18 The content shown in the figure shows that the car is equipped with a transmitter 1 1801, a receiver 1 1802, a transmitter 2 1803 and a receiver 2 1804. Among them, the transmitter 1 1801 and the receiver 1 1802 are installed on the left A-pillar in the car, and the transmitter 2 1803 and the receiver 2 1804 are deployed above the central rearview mirror. The working area facing the receiver 1 1802 is defined as the driving information display area. When the driver wears the XR device and faces straight ahead, the XR device plays driving-related HUD, etc.; the working area facing the receiver 2 1804 is defined as the vehicle control information display area. When the driver wears the XR device and faces to the right, the XR device plays vehicle control information.
[0192] Similarly, transmitters and receivers may be arranged in pairs at other locations of the vehicle, which will not be described in detail here.
[0193] Technology 3: Using infrared beams with a certain pulse frequency instead of continuous infrared light to solve the problems of sunlight interference and interference between different transmitters.
[0194] Since infrared light is also a component of sunlight, in order to reduce the interference caused by sunlight hitting the reflective spherical surface and then reflecting onto the receiver, the infrared beam transmitter is configured to emit in the form of a pulse beam of a certain frequency when working. This pulse frequency can be regarded as the operating frequency of the transmitter. The receiver can filter out the interference of sunlight infrared with the help of high-pass filtering.
[0195] Different transmitters can be configured with different operating frequencies, and the content of the area will be displayed only when the receiver receives a pulse infrared photoelectric light flashing at a specific frequency. For example, receiver 2 1804 will not activate the vehicle control information display when it receives the light beam emitted by transmitter 1 1801 but does not receive the light beam emitted by transmitter 2 1803.
[0196] In this way, the interference of sunlight can be eliminated and the reception efficiency can be enhanced, while reducing the burden on the user's senses caused by displaying too much content at the same time and reducing the load on the vehicle to render too much XR content.
[0197] Technology 4: Principle of positioning algorithm of receiver.
[0198] From a bird's-eye view, since the user's position is fixed relative to the receiver, and the length / width of the receiver (a few centimeters) is extremely small relative to the distance between the user and the receiver (tens of centimeters), the plane receiver can be approximately considered as a curved surface on a sphere with the position of the reflective surface worn by the user as the center and the distance between the reflective surface and the receiver as the radius. To further improve the accuracy, the curvature of the receiver surface can also be made to conform to the curved surface, and when configuring, the curved surface can be made perpendicular to the line from its center point to the driver's direction. Furthermore, a motor system can be added for alignment.
[0199] The displacement between two positions on the receiver, since the distance from the reflective surface to the red dot of the receiver can be considered fixed, refer to Fig.19 The content shown, that is, the length of distance one 1901 and distance two 1902 are the same, both are R. The displacement X between position one 1903 and position two 1904 of the red dot reflected by the infrared ray on the surface of the receiver = the user's head rotation angle α (calculated in radians) × R. Among them, the infrared ray also has the function of measuring distance, and the R can be calculated in real time or calculated and saved when used for the first time.
[0200] Since the infrared ray is a pulse beam with a frequency of V, that is, the time interval between two consecutive red dots is 1 / V=T, the receiver can calculate the angle α=X / R of the user's head rotation within the time T.
[0201] Since the frequency V is generally in the order of tens of kilohertz (KHz) (such as 40KHz), the time between each two red dots is T = 0.025ms, which is much shorter than the human reaction time (the maximum reaction time is 100ms). Therefore, it can be considered that the direction of the user's head rotation during this period is a unidirectional movement. This is the theoretical basis for the accuracy of the calculated α, and this α is also the minimum unit rotation angle.
[0202] A user's head turn is often composed of millions of such tiny displacements. During this process, the red dot may move out of the receiver position, and then return to the receiver due to the characteristics of the multi-faceted reflective surface. If the smallest unit α is directly calculated, it will be found that the displacement direction is unknown. For example, refer to Fig. 20 As shown, the receiver 2001 detects position 1 2011 and position 2022, but does not know the displacement direction between position 1 2011 and position 2 2022, that is, it is not known whether the red dot moves from the upper left to the lower right, or continues to move to the upper left.
[0203] Therefore, the calculation of α cannot be based on the displacement of two positions alone. Since the scale of T is extremely small, X not only has the unidirectional characteristics within the period of time, but also the length and direction of adjacent displacements within a period of time are almost constant. Therefore, when calculating α, it is necessary to first collect dozens of position points, and take the mode of the length of X as the unidirectional displacement length. The displacement that is significantly different from this value indicates that it is a situation that exceeds the receiver, and then the displacement direction of these dozens of position points can be determined. For example, refer to Fig.21 As shown in the content, multiple positions are detected in the receiver 2101, including position 1 21011, position 2 21012, position 3 21013, position 4 21014 and position 5 21015 in chronological order. Among them, displacement X1 is the displacement between position 1 21011 and position 2 21012, displacement X2 is the displacement between position 2 21012 and position 3 21013, and displacement X4 is the displacement between position 4 21014 and position 5 21015. However, displacement X3 may have two situations. Situation 1 is that position 3 21013 points to the left edge of the receiver 2101, and then points to position 4 21014 from the right edge of the receiver 2101; situation 2 is that position 3 21013 directly points to position 4 21014.
[0204] The red dot's displacement at position three at time 1 exceeds the receiver range, and at time 2 it returns to position four from the lower right corner of the receiver. If X3 in case 1 from position three to position four is calculated directly, the length of displacement three is too different from displacement X1, displacement X2, and displacement X4 (which can be considered the distance that can be moved in one time unit). Therefore, it can be considered that case 2 of displacement X3 calculated in this way is wrong, and the direction can be determined.
[0205] Since dozens of position points need to be collected before the displacement is calculated, a delay of 0.1ms will be generated, but it is still far less than the limit of human reaction time, so the delay can be ignored.
[0206] In summary, the positioning scheme of the receiver is to collect the sum Xn of the displacement of the red dot on the receiver during this period of time with a period of 0.1s each time, decompose it into horizontal displacement Xh and vertical displacement Xv, and use this to calculate the angle α=Xn / R of the user's head rotation during this period of time, so as to obtain the user's horizontal head turning angle and vertical head turning angle, and then determine the movement of the user's head direction.
[0207] Technology 5: Functional area status switching process.
[0208] refer to Fig. 22 The contents shown include but are not limited to the following S2201 to S2203.
[0209] Initial; after power-on, the function area is inactive by default and the following S2201 is executed;
[0210] S2201, entering the function area inactive state; the receiver enters the power saving mode, and the cockpit does not render the scene;
[0211] When the receiver receives the red dot for more than 0.1s, it enters the function area standby state and executes the following S2202;
[0212] S2202, entering the function area standby state; the receiver is started, the cockpit does not render the scene, but calculates the user's head turning angle;
[0213] When the user turns his head to an angle less than 5° within 0.5s, the function area is activated and the following S2203 is executed;
[0214] S2203, enter the function area activation state; reset the center point position, use the direction of the user's head when entering this state as the new center point, render the scene in the cockpit, and calculate the user's head turning angle. When the positioning is lost, maintain the activation state for 10 seconds.
[0215] When the receiver loses the red dot for more than 10 seconds, it enters the function inactive state and executes the above S2201.
[0216] in:
[0217] For the inactive state of the function area, when the vehicle is powered on, it enters the inactive state by default. In this state, in order to reduce power consumption, the receiver reduces the sampling frequency and enters the power saving mode; the cockpit does not render the scene corresponding to the function area.
[0218] For the function area standby state, when the receiver receives the red dot for more than 0.1s in the inactive state, it means that the user begins to roughly face the function area and may need to activate the function area, so the function area enters the standby state from the inactive state. In this state, the receiver enters the rated working state, the sampling frequency is restored to its rated working frequency, and the displacement information of the red dot is continuously fed back to the cockpit system. The cockpit does not render the scene of the function area at this time, but calculates and monitors the user's head turning angle.
[0219] When the function area is in standby state, if the system monitors that the total head turning angle of the user within 0.5s is less than 5° (the time and angle can be calibrated), it means that the user is clearly looking at the function area. At this time, it is determined that there is a need to activate the function area, so the function area enters the function area activation state from the standby state.
[0220] For the function area activation state, when entering this state, the user's head direction is repositioned to the display reference center point (which can be repositioned later by means of steering wheel buttons, etc.), and the scene rendering of the function area is performed based on the reference center point. When positioning is lost (the receiver does not receive the red dot), the activation state is maintained for up to 10 seconds (calibration is possible), and after 10 seconds, it exits to the function area inactive state.
[0221] The main contents of this embodiment may include: an algorithm for calculating the rotation angle of the XR device based on the translation distance of the red dot reflected on the receiver; positioning logic and rendering capabilities are defined in the cockpit rather than the XR device; a spherical reflective surface is provided on the XR device; a pulsed light beam is used to filter out sunlight interference; and different functional areas are defined according to the working range of different transmitters to reduce the cockpit rendering load.
[0222] The technical effects achieved by this embodiment may include: since the computing center is transferred to the cockpit, compared with the VR solution in the related art, the head-mounted device can be reduced from the 500g level of VR to the 100g level of AR, reducing the weight by 80%. Compared with the AR solution in the related art, the positioning capability is increased, which greatly enriches the usage scenarios. Since the computing center is transferred to the cockpit, compared with AR glasses with built-in 3dof positioning, the head-mounted device does not need to have its own XR chip, and the glasses do not need to be equipped with various sensors, which is cheaper and the weight is further reduced. Compared with external laser positioning and built-in / external visual positioning, the infrared positioning of this embodiment has lower cost and simpler algorithm.
[0223] In a third aspect, the present application also provides a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction on the storage medium is executed, the processing method of the XR device in the vehicle provided in the embodiment of the present application is implemented.
[0224] In a fourth aspect, the present application also provides a computer program product, which includes a computer program or an instruction. When the computer program or the instruction is executed, the processing method of the XR device in the vehicle provided in the embodiment of the present application is implemented.
[0225] It should be noted that the description of the electronic device has the same beneficial effects as the processing method of the XR device in the vehicle, and will not be repeated one by one. In addition, for technical details not disclosed in the device embodiment of this application, please refer to the description of the processing system embodiment of the XR device in the vehicle.
[0226] The description of the storage medium is similar to the description of the processing method of the XR device in the above-mentioned vehicle, and has the same beneficial effect description as the processing method of the XR device in the vehicle, which will not be repeated one by one. In addition, for technical details not disclosed in the device embodiment of this application, please refer to the description of the processing method embodiment of the XR device in the vehicle of the vehicle.
[0227] The description of the computer program product is similar to the description of the processing method of the XR device in the vehicle, and has the same beneficial effects as the processing method of the XR device in the vehicle, which will not be repeated here. In addition, for technical details not disclosed in the device embodiment of this application, please refer to the description of the processing method embodiment of the XR device in the vehicle.
[0228] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.
[0229] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0230] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units. A person of ordinary skill in the art may understand that all or part of the steps of implementing the above method embodiments may be completed by hardware related to program instructions, and the above program may be stored in a computer-readable storage medium, which, when executed, executes the steps of the above method embodiments; and the above storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.
[0231] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0232] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0233] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0234] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A processing system for an extended reality (XR) device in a vehicle, characterized in that: The system comprises: a transceiver deployed on the vehicle, the transceiver comprising a transmitter and a receiver, an XR device located in the vehicle, a reflector mounted on the XR device, and a processor connected to the receiver; wherein: The transmitter is used to: generate an infrared transmission light beam according to a first frequency; The reflector is used to: reflect the infrared emission light beam to obtain an infrared reflection light beam; The receiver is used to: receive the infrared reflected light beam, sample the light spot position of the infrared reflected light beam received in the current time period, and obtain the light spot position information in the current time period; and send the light spot position information in the current time period to the processor; The processor is used to determine the current three-degree-of-freedom posture of the XR device on the vehicle based on the light spot position information in the current time period.
2. The system according to claim 1, characterized in that The processor is also connected to the XR device; the processor is also used for: Determining whether the XR device satisfies a first condition based on the current three-degree-of-freedom posture of the XR device on the vehicle; the first condition includes that a rotation angle within a first time period is less than a first angle threshold; In a case where it is determined that the XR device satisfies the first condition, determining a target rendering content and a display reference center point; The target rendering content and the display reference center point are transmitted to the XR device, so that the target rendering content is displayed on a display screen of the XR device with the display reference center point as the center.
3. The system according to claim 2, characterized in that In the case where a plurality of sets of transceivers are deployed on the vehicle, a set of the transceivers includes one of the transmitters and one of the receivers; Each of the plurality of groups of transceivers is deployed at a different position on the vehicle; The rendering content corresponding to each group of transceivers in the multiple groups of transceivers is different; The first frequency corresponding to each group of transceivers in the multiple groups of transceivers has a different value; The processor is also used to: determine that the target rendering content is the rendering content corresponding to the target transceiver in the multiple groups of transceivers; the target transceiver includes: a transceiver corresponding to the receiver that receives the infrared reflected light beam; or a transceiver corresponding to the receiver that receives the infrared reflected light beam of the first frequency.
4. The system according to claim 3, characterized in that In the case where two sets of transceivers are deployed on the vehicle, a first set of transceivers of the two sets of transceivers is deployed in front of the driving position in the vehicle; Correspondingly, the rendering content corresponding to the first group of transceivers includes: driving information of the vehicle; The second set of transceivers of the two sets of transceivers is disposed above or below the central rearview mirror of the vehicle; Correspondingly, the target rendering content corresponding to the second group of transceivers includes: vehicle control information of the vehicle.
5. The system according to any one of claims 1 to 4, characterized in that: The processor includes: a cockpit processor of the vehicle, or a processor of the XR device.
6. The system according to any one of claims 1 to 4, characterized in that: The reflector is a multi-faceted reflector.
7. A processing method for an XR device in a vehicle, characterized in that: A transceiver is deployed in the vehicle, the transceiver includes a transmitter and a receiver, the XR device is located in the vehicle, and a reflector is installed on the XR device; wherein the infrared emission light beam generated by the transmitter at a first frequency is reflected by the reflector and transmitted to the receiver; the method includes: Sampling the light spot position of the infrared reflected light beam received by the receiver in the current time period to obtain the light spot position information in the current time period; Based on the light spot position information in the current time period, determine the current three-degree-of-freedom position of the XR device on the vehicle.
8. The method according to claim 7, characterized in that The sampling of the light spot position of the infrared reflected light beam received by the receiver in the current time period to obtain the light spot position information in the current time period includes: Determine whether the infrared reflected light beam received by the receiver in the current time period satisfies a second condition, where the second condition is used to indicate that the time for the receiver to receive the infrared reflected light beam in the current time period exceeds a first time threshold; When the infrared reflected light beam received by the receiver in the current time period satisfies the second condition, the light spot position of the infrared reflected light beam received by the receiver in the current time period is sampled to obtain the light spot position information in the current time period.
9. The method according to claim 7, characterized in that: The method further comprises: Filtering the infrared light beam received by the receiver in the current time period; obtaining the infrared reflected light beam of the first frequency in the current time period; Correspondingly, sampling the light spot position of the infrared reflected light beam received by the receiver in the current time period to obtain the light spot position information in the current time period includes: The light spot position of the infrared reflection light beam of the first frequency received by the receiver in the current time period is sampled to obtain the light spot position information in the current time period.
10. The method according to any one of claims 7 to 9, characterized in that: The determining, based on the light spot position information within the current time period, a current three-degree-of-freedom posture of the XR device on the vehicle includes: Determine the total displacement within the current time period based on the light spot position information within the current time period; Projecting the total displacement in the current time period to the horizontal axis direction, the vertical axis direction and the vertical axis direction respectively to obtain the horizontal total displacement, the vertical total displacement and the vertical total displacement in the current time period; Dividing the total horizontal displacement, the total vertical displacement, and the total vertical displacement in the current time period by the reference radius respectively, to determine the three rotation angles of the XR device on the vehicle; the reference radius is the distance between the XR device and the transceiver; Determine the three-degree-of-freedom posture of the XR device currently on the vehicle as the three rotation angles.
11. The method according to claim 10, characterized in that In a case where the light spot position information includes M light spot positions and a collection time corresponding to each of the multiple light spot positions, determining the total displacement in the current time period based on the light spot position information in the current time period includes: Based on the acquisition time corresponding to each of the M light spot positions, the M light spots are sorted in the order of the acquisition time to obtain a light spot sequence; For each two adjacent light spots in the light spot sequence, determine the displacement between each two adjacent light spots based on the light spot positions of each two adjacent light spots to obtain N displacements; wherein M is greater than 1; and N is greater than M; Deleting the displacements greater than the displacement threshold among the N displacements to obtain M-1 target displacements; The total displacement in the current time period is determined to be the sum of the M-1 target displacements.
12. The method according to any one of claims 7 to 9, characterized in that: The method further comprises: Determining whether the XR device satisfies a first condition based on the current three-degree-of-freedom posture of the XR device on the vehicle; the first condition is used to indicate that the rotation angle within a first time period is less than a first angle threshold; In a case where it is determined that the XR device satisfies the first condition, determining a target rendering content and a display reference center point; The target rendering content and the display reference center point are transmitted to the XR device, so that the target rendering content is displayed on a display screen of the XR device with the display reference center point as the center.
13. The method according to claim 12, characterized in that Determining the target rendering content includes: If the number of the transceivers is one group, determining that the target rendering content is the rendering content corresponding to the transceiver; If there are multiple groups of transceivers, the target rendering content is determined to be the rendering content corresponding to the target transceiver in the multiple groups of transceivers; the target transceiver includes: a transceiver corresponding to the receiver that receives the infrared reflected light beam; or a transceiver corresponding to the receiver that receives the infrared reflected light beam of the first frequency.
