Control method and device, vehicle-mounted equipment and medium
By controlling the adaptation of the coordinate system of the vehicle equipment in the on-board AR glasses, the coordinate system of the vehicle or the world remains unchanged according to the mode and scene of the passenger or driver, the problem of poor interaction experience between the passenger and the driver is solved, and the problem of poor passenger and driver interaction experience is achieved 6Dof's high-quality interaction.
Patent Information
- Application Number
- CN202310548119.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
Vehicle AR glasses need to be adapted according to the different position coordinate systems of passengers and drivers. It is difficult for the existing technology to effectively determine and switch the coordinate system between the two, resulting in poor interactive experience.
By obtaining the current working mode of the vehicle equipment, the coordinate system of the vehicle equipment is controlled to remain unchanged in passenger mode relative to the vehicle coordinate system, and in driving mode, the coordinate system relative to the world coordinate system or vehicle coordinate system is selected according to the operating scene, realizing virtualization Anchoring of an object or image.
The 6Dof interactive experience between passengers and drivers is realized in different coordinate systems, ensuring the correct anchoring of virtual objects or images with the vehicle or external environment, improving the user experience.
Smart Images

Figure CN118939109A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present disclosure relate to the technical field of wearable devices, and more specifically, to a control method, a control device, a vehicle-mounted device, and a computer-readable storage medium. Background Art
[0002] With the development of augmented reality technology, AR products such as AR glasses have appeared in more and more application scenarios, such as car-mounted AR glasses.
[0003] In the related art, in-vehicle AR glasses can be used by two types of users: passengers and drivers. However, the posture coordinate systems required by passengers and drivers are different. Based on this, there is a problem of how to determine the posture coordinate systems required by these two types of users: passengers and drivers. Summary of the invention
[0004] The purpose of the embodiments of the present disclosure is to provide a control method, device, vehicle-mounted equipment and medium.
[0005] According to a first aspect of an embodiment of the present disclosure, a control method is provided, the method comprising:
[0006] Acquire a current working mode of the vehicle-mounted device; wherein the current working mode includes one of a driving mode and a passenger mode;
[0007] When the current working mode is the passenger mode, controlling the coordinate system of the onboard device to remain unchanged relative to the vehicle coordinate system;
[0008] When the current working mode is the driving mode, the coordinate system of the on-board device is controlled to remain unchanged relative to the world coordinate system according to the current operating scenario of the on-board device, or the coordinate system of the on-board device is controlled to remain unchanged relative to the vehicle coordinate system.
[0009] Optionally, when the current working mode is the driving mode, controlling the coordinate system of the on-board device to remain unchanged relative to the world coordinate system according to the current operating scenario of the on-board device, or controlling the coordinate system of the on-board device to remain unchanged relative to the vehicle coordinate system, includes:
[0010] In the case where the current working mode is the driving mode, if the current operating scenario is the first operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the world coordinate system;
[0011] In the case where the current working mode is the driving mode, if the current operating scenario is the second operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the vehicle coordinate system;
[0012] The first operation scene at least includes a navigation scene, and the second operation scene at least includes a display scene.
[0013] Optionally, controlling the on-board device coordinate system to remain unchanged relative to the vehicle coordinate system includes:
[0014] Acquire the rotation quaternion of the first pose data of the vehicle; wherein the rotation quaternion of the first pose data is the rotation quaternion of the current pose data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts to run;
[0015] Obtaining the rotation quaternion of the real-time position data of the vehicle;
[0016] Obtaining the rotation quaternion of the real-time position and posture data of the vehicle-mounted device;
[0017] Obtaining the rotation quaternion of the converted real-time pose data of the on-board device according to the rotation quaternion of the first pose data of the vehicle, the rotation quaternion of the real-time pose data of the vehicle and the rotation quaternion of the real-time pose data of the on-board device;
[0018] According to the rotation quaternion of the converted real-time position and posture data of the vehicle-mounted device, it is determined that the coordinate system of the vehicle-mounted device remains unchanged relative to the vehicle coordinate system.
[0019] Optionally, obtaining the current working mode of the vehicle-mounted device includes:
[0020] In response to an operation of setting the working mode of the vehicle-mounted device, providing a setting entry;
[0021] The operating mode set through the first setting entry is obtained as the current operating mode of the vehicle-mounted device.
[0022] Optionally, the method further comprises:
[0023] Establish a short-range communication connection with the vehicle;
[0024] Based on the short-distance communication connection, a rotation quaternion of the first pose data of the vehicle and a rotation quaternion of the real-time pose data of the vehicle transmitted by the vehicle are received.
[0025] According to a second aspect of an embodiment of the present disclosure, a control device is provided, the device comprising:
[0026] An acquisition module, used to acquire a current working mode of the vehicle-mounted device; wherein the current working mode includes one of a driving mode and a passenger mode;
[0027] A control module, configured to control the coordinate system of the onboard device to remain unchanged relative to the vehicle coordinate system when the current working mode is the passenger mode;
[0028] The control module is also used to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the world coordinate system, or to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the vehicle coordinate system according to the current operating scenario of the vehicle-mounted device when the current working mode is the driving mode.
[0029] Optionally, the control module is specifically used to:
[0030] In the case where the current working mode is the driving mode, if the current operating scenario is the first operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the world coordinate system;
[0031] In the case where the current working mode is the driving mode, if the current operating scenario is the second operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the vehicle coordinate system;
[0032] The first operation scene at least includes a navigation scene, and the second operation scene at least includes a display scene.
[0033] Optionally, the control module is specifically used to:
[0034] Acquire the rotation quaternion of the first pose data of the vehicle; wherein the rotation quaternion of the first pose data is the rotation quaternion of the current pose data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts to run;
[0035] Obtaining the rotation quaternion of the real-time position data of the vehicle;
[0036] Obtaining the rotation quaternion of the real-time position and posture data of the vehicle-mounted device;
[0037] Obtaining the rotation quaternion of the converted real-time pose data of the on-board device according to the rotation quaternion of the first pose data of the vehicle, the rotation quaternion of the real-time pose data of the vehicle and the rotation quaternion of the real-time pose data of the on-board device;
[0038] According to the rotation quaternion of the converted real-time position and posture data of the vehicle-mounted device, it is determined that the coordinate system of the vehicle-mounted device remains unchanged relative to the vehicle coordinate system.
[0039] According to a third aspect of an embodiment of the present disclosure, there is provided a vehicle-mounted device, the vehicle-mounted device comprising:
[0040] Memory for storing executable computer instructions;
[0041] A processor is used to execute the control method according to the first aspect above under the control of the executable computer instructions.
[0042] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the control method described in the first aspect above is executed.
[0043] One beneficial effect of the disclosed embodiment is that, when the current working mode of the vehicle-mounted device is the passenger mode, it indicates that the passenger is currently using the vehicle-mounted device. At this time, the vehicle-mounted device coordinate system will be controlled to remain unchanged relative to the vehicle coordinate system, that is, the vehicle-mounted device coordinate system and the vehicle coordinate system are consistent, thereby realizing the anchoring of the virtual object or image displayed by the vehicle-mounted device relative to the vehicle. When the current working mode of the vehicle-mounted device is the driving mode, it indicates that the driver is currently using the vehicle-mounted device. At this time, the vehicle-mounted device coordinate system will be controlled to remain unchanged relative to the world coordinate system according to different selections of the current operating scene of the vehicle-mounted device, or the vehicle-mounted device coordinate system will be controlled to remain unchanged relative to the vehicle coordinate system. For example, when it is necessary to interact with the external environment, the vehicle-mounted device coordinate system can be controlled to remain unchanged relative to the world coordinate system. At this time, the virtual object or image displayed by the vehicle-mounted device is anchored relative to the external environment. For example, when it is not necessary to interact with the external environment, the vehicle-mounted device coordinate system can be controlled to remain unchanged relative to the vehicle coordinate system, thereby realizing the anchoring of the virtual object or image displayed by the vehicle-mounted device relative to the vehicle. In other words, through the embodiments of the present application, passengers and drivers can get a good 6Dof experience when interacting with objects in different coordinate systems.
[0044] Other features and advantages of the present specification will become apparent from the following detailed description of exemplary embodiments of the present specification with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the specification and, together with the description, serve to explain the principles of the specification.
[0046] Figure 1 is a schematic diagram of the hardware configuration of a control system according to an embodiment of the present disclosure;
[0047] Figure 2 is a flow chart of a control method according to an embodiment of the present disclosure;
[0048] Figure 3 is a principle block diagram of a control device according to an embodiment of the present disclosure;
[0049] Figure 4 It is a principle block diagram of a vehicle-mounted device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0050] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the embodiments of the present disclosure.
[0051] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0052] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0053] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0054] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0055] <Hardware Configuration>
[0056] Figure 1 The figure is a schematic diagram of the hardware configuration of a control system that can be used to implement a control method of an embodiment. Figure 1 The in-vehicle device 100, the vehicle 200 and the network 300 are shown. The in-vehicle device 100 can be connected to the network 300, and can also be connected to the vehicle 200 through a communication method such as Bluetooth. In one embodiment, the in-vehicle device 100 is connected to the vehicle 200 only through a communication method such as Bluetooth. Among them, multiple servers 301 and 302 can be set in the network 300. The network 300 can be a wireless communication network or a wired communication network. The network 300 can be a local area network or a wide area network. The network 300 can be a short-distance communication or a long-distance communication.
[0057] In one embodiment, Figure 1 As shown, the vehicle-mounted device 100 may include a processor 101 and a memory 102. The vehicle-mounted device 100 also includes a communication device 103, a display device 104, a user interface 105, a camera device 106, an audio / video interface 107, a sensor 108, etc. In addition, the vehicle-mounted device 100 may also include a power management chip 109 and a battery 110, etc.
[0058] Among them, the processor 101 can be various processors. The memory 102 can store the underlying software, system software, application software, data, etc. required for the operation of the vehicle-mounted device 100. The memory 102 can include various forms of memory, such as ROM, RAM, Flash, etc. The communication device 103 can include, for example, a WiFi communication device, a Bluetooth communication device, a 3G, 4G and 5G communication device, etc. Through the communication device 103, the vehicle-mounted device 100 can be arranged in the network. The display device 104 can be a liquid crystal display, an OLED display, etc. In one example, the display device 104 can be a touch screen. The user can perform input operations through the display device 104. In addition, the user can also perform fingerprint recognition through the touch screen. The user interface 105 may include a USB interface, a lightning interface, a keyboard, etc. The camera device 106 can be a single camera or multiple cameras. The audio / video interface 107 can include, for example, a speaker interface, a microphone interface, a video transmission interface such as HDMI, etc. The sensor 108 can include, for example, a gyroscope, an accelerometer, a temperature sensor, a humidity sensor, a pressure sensor, etc. For example, the posture information of the vehicle-mounted device can be determined by the sensor. The power management chip 109 can be used to manage the power input to the vehicle-mounted device 100 and can also manage the battery 110 to ensure greater utilization efficiency. The battery 110 is, for example, a lithium-ion battery.
[0059] The vehicle-mounted device 100 may be a vehicle-mounted head-mounted display device, for example, vehicle-mounted AR (Augmented Reality) glasses and vehicle-mounted MR (Mixed Reality) glasses, etc., which are not limited in the embodiments of the present disclosure. Figure 1 The various components shown are only schematic. The vehicle-mounted device 100 may include Figure 1 One or more of the components shown, but not necessarily including Figure 1 All parts in. Figure 1 The illustrated in-vehicle device 100 is illustrative only and is in no way intended to limit the embodiments herein, their application, or uses.
[0060] In this embodiment, the memory 102 of the vehicle-mounted device 100 is used to store program instructions, which are used to control the processor 101 to operate to execute the control method. The technician can design instructions according to the scheme disclosed in the present invention. How the instructions control the processor to operate is well known in the art, so it will not be described in detail here.
[0061] In one embodiment, Figure 1As shown, the vehicle 200 may include a processor 201 and a memory 202. The vehicle 200 also includes a communication device 203, a display device 204, a user interface 205, a camera device 206, an audio / video interface 207, a sensor 208, etc. In addition, the second device 200 may also include a power management chip 209 and a battery 210, etc.
[0062] The vehicle 200 may be a mobile phone, a portable computer, a tablet computer, a PDA, a wearable device, etc., which is not limited in the embodiments of the present disclosure. Figure 1 The various components shown are for illustration only. Vehicle 200 may include Figure 1 One or more of the components shown, but not necessarily including Figure 1 All parts in. Figure 1 The illustrated vehicle 200 is for illustrative purposes only and is in no way intended to limit the embodiments herein, their application, or uses.
[0063] It should be understood that although Figure 1 Only one in-vehicle device 100 and one vehicle 200 are shown, but this does not mean that the number of each is limited, and the control system may include multiple in-vehicle devices 100 and multiple vehicles 200.
[0064] In the above description, the technician can design instructions according to the solution provided by the present disclosure. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.
[0065] <Method Example>
[0066] Figure 2 A control method according to an embodiment of the present disclosure is shown. The control method can be performed by Figure 1 The vehicle-mounted device shown in the figure can be a vehicle-mounted AR glasses and a vehicle-mounted MR glasses, etc. Figure 2 As shown, the control method of this embodiment may include the following steps S2100 to S2300:
[0067] Step S2100, obtaining the current working mode of the vehicle-mounted device.
[0068] The current working mode includes one of a driving mode and a passenger mode. When the current working mode of the vehicle-mounted device is the passenger mode, it usually indicates that the user currently using the vehicle-mounted device is a passenger. When the current working mode of the vehicle-mounted device is the driving mode, it usually indicates that the user currently using the vehicle-mounted device is a driver, that is, the vehicle-mounted device of the embodiment of the present disclosure can provide different working modes for the user to choose.
[0069] In this embodiment, after the user turns on the vehicle-mounted device, on the one hand, the vehicle-mounted device will establish a short-distance communication connection with the vehicle, so that the vehicle transmits the vehicle's position data to the vehicle-mounted device based on the short-distance communication connection. In other words, the vehicle-mounted device receives the vehicle's position data transmitted by the vehicle based on the short-distance communication connection.
[0070] After the user turns on the vehicle-mounted device, on the other hand, the desktop launcher of the vehicle-mounted device is started, and the desktop environment of the vehicle-mounted device starts running. When the desktop environment of the vehicle-mounted device starts running, the vehicle can send the first position data of the vehicle to the vehicle-mounted device based on the short-distance connection, that is, the vehicle-mounted device can receive the first position data of the vehicle transmitted by the vehicle based on the short-distance communication connection. Among them, the first position data is the current position data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts running.
[0071] More specifically, when the desktop environment of the vehicle-mounted device starts running, the vehicle can send the rotation quaternion of the first position data of the vehicle to the vehicle-mounted device based on the short-distance connection. In other words, the vehicle-mounted device can receive the rotation quaternion Q of the first position data of the vehicle transmitted by the vehicle. carBegin Among them, the rotation quaternion Q of the first pose data is carBegin It is the rotation quaternion of the current position data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts running.
[0072] The posture data includes rotation quaternion. The vehicle includes an inertial measurement unit (IMU). The inertial measurement unit can collect the posture data of the vehicle at a set frequency. The posture data of the vehicle collected by the inertial measurement unit at the current moment can reflect the posture change of the inertial measurement unit at the current moment relative to the previous moment.
[0073] It is understandable that the number and type of inertial measurement units installed in the vehicle can be set by the designer according to the usage scenario and requirements to ensure the accuracy of the detected vehicle posture data.
[0074] In this embodiment, after the desktop environment of the vehicle-mounted device starts running, the vehicle-mounted device displays the desktop environment within the user's field of vision. In an optional embodiment, this step S2100 of obtaining the current working mode of the vehicle-mounted device may further include: providing a setting entry; obtaining the working mode set through the setting entry as the current working mode of the vehicle-mounted device.
[0075] Specifically, the vehicle-mounted device displays a desktop environment within the user's field of vision, and the desktop environment provides a setting entry. The setting entry can be any entry for the user to input information, such as an input box, a drop-down list, a voice input entry, etc., which is not limited here. The user can set the required working mode through the setting entry. Exemplarily, the desktop environment of the vehicle-mounted device provides a drop-down list, and two drop-down information are displayed in the drop-down list, one of which is "passenger mode" and the other is "driving mode". For example, the user can click on the drop-down information "passenger mode" and the vehicle-mounted device can enter the passenger mode. For another example, the user can click on the drop-down information "driving mode" and the vehicle-mounted device can enter the driving mode. In this way, the user can set the current working mode of the vehicle-mounted device according to his own needs to meet the user's personalized needs.
[0076] Step S2200, when the current working mode is the passenger mode, control the coordinate system of the on-board device to remain unchanged relative to the vehicle coordinate system.
[0077] In this embodiment, when the current working mode of the vehicle-mounted device is the passenger mode, it indicates that the passenger is currently using the vehicle-mounted device. Since the passenger only needs to anchor the virtual objects or images displayed in the vehicle-mounted device relative to the vehicle, the vehicle-mounted device can control the vehicle-mounted device coordinate system and the vehicle coordinate system to remain consistent, so that the vehicle-mounted device changes with the movement of the vehicle, thereby achieving the anchoring of the virtual objects or images displayed in the vehicle-mounted device relative to the vehicle.
[0078] It is understandable that since both the vehicle steering and the on-board device steering will cause the on-board device's posture data to change, if you want to keep the on-board device coordinate system relatively unchanged relative to the vehicle coordinate system, you need to offset the effect of the vehicle steering on the on-board device's posture data. In an optional embodiment, when the current working mode is the passenger mode, this step S2200 controls the on-board device coordinate system to remain unchanged relative to the vehicle coordinate system and can further include the following steps S2210 to S2250:
[0079] Step S2210, obtaining the rotation quaternion of the first position data of the vehicle.
[0080] The rotation quaternion of the first pose data is the rotation quaternion Q of the current pose data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts running. carBegin It can be understood that, referring to the above steps, when the desktop environment of the vehicle-mounted device starts running, the vehicle can send the rotation quaternion of the first position data of the vehicle to the vehicle-mounted device based on the short-distance connection, and the vehicle-mounted device can receive the rotation quaternion Q of the first position data of the vehicle transmitted by the vehicle. carBegin .
[0081] Step S2220, obtaining the rotation quaternion Q of the vehicle's real-time posture data car .
[0082] The real-time position and posture data of the vehicle refers to the position and posture data recorded in real time during the driving process of the vehicle.
[0083] In this step S2220, during the driving process of the vehicle, the vehicle can send the real-time position and posture data of the vehicle to the vehicle device based on the short-distance communication connection, that is, the vehicle-mounted device can receive the real-time position and posture data of the vehicle transmitted by the vehicle based on the short-distance communication connection. Specifically, during the driving process of the vehicle, the vehicle can send the rotation quaternion of the real-time position and posture data of the vehicle to the vehicle-mounted device based on the short-distance communication connection, that is, the vehicle-mounted device can receive the rotation quaternion of the real-time position and posture data of the vehicle transmitted by the vehicle based on the short-distance communication connection.
[0084] Step S2230, obtaining the rotation quaternion Q of the real-time posture data of the vehicle-mounted device xr .
[0085] In this step S2230, the vehicle-mounted equipment includes an inertial measurement unit, which can collect the posture data of the vehicle-mounted equipment at a set frequency. The posture data of the vehicle-mounted equipment collected by the inertial measurement unit at the current moment can reflect the posture change of the inertial measurement unit at the current moment relative to the previous moment.
[0086] The number and type of inertial measurement units installed in the vehicle-mounted device can be set by the designer according to the usage scenario and requirements to ensure the accuracy of the detected position and posture data of the vehicle-mounted device.
[0087] The real-time position and posture data of the vehicle-mounted device is the real-time position and posture data of the user recorded by the vehicle-mounted device when the user is wearing the vehicle-mounted device.
[0088] Step S2240, obtaining the rotation quaternion Q of the converted real-time pose data of the on-board device according to the rotation quaternion of the first pose data of the vehicle, the rotation quaternion of the real-time pose data of the vehicle and the rotation quaternion of the real-time pose data of the on-board device passenger .
[0089] In this step S2240, the rotation quaternion Q of the converted real-time posture data of the vehicle-mounted device passenger Satisfies the following formula:
[0090] Q passenger =Q xr (Q car Q carBegin -1 ) -1 (1)
[0091] Among them, Q car is the rotation quaternion of the vehicle’s real-time posture data, Q xr is the rotation quaternion of the real-time posture data of the vehicle-mounted equipment, Q carBegin The rotation quaternion of the first position data of the vehicle.
[0092] Step S2250, determining that the coordinate system of the on-board device remains unchanged relative to the vehicle coordinate system based on the rotation quaternion of the converted real-time posture data of the on-board device.
[0093] According to the above steps S2210 to S2250, the influence of vehicle steering on the posture data of the vehicle-mounted device can be eliminated, so that the coordinate system of the vehicle-mounted device and the vehicle coordinate system remain consistent, and the virtual object or image displayed by the vehicle-mounted device is anchored relative to the vehicle.
[0094] Step S2300, when the current working mode is the driving mode, the coordinate system of the vehicle-mounted device is controlled to remain unchanged relative to the world coordinate system according to the current operating scenario of the vehicle-mounted device, or the coordinate system of the vehicle-mounted device is controlled to remain unchanged relative to the vehicle coordinate system.
[0095] In this embodiment, when the current working mode is the driving mode, this step S2300 controls the coordinate system of the vehicle-mounted device to remain unchanged relative to the world coordinate system according to the current operation scene of the vehicle-mounted device, or controls the coordinate system of the vehicle-mounted device to remain unchanged relative to the vehicle coordinate system, which may further include the following steps S2310 to S2320:
[0096] Step S2310, when the current working mode is the driving mode, if the current operating scenario is the first operating scenario, control the vehicle-mounted device coordinate system to remain unchanged relative to the world coordinate system.
[0097] The first operation scenario at least includes a navigation scenario. It can be understood that the first operation scenario can be an interaction scenario in which a user interacts with an external environment outside the vehicle, which is not limited in this embodiment.
[0098] In this step S2310, when the current working mode of the vehicle-mounted device is the driving mode, it indicates that the driver is using the vehicle-mounted device. If the vehicle-mounted device is a scene in which the user interacts with the external environment outside the vehicle, such as a navigation scene, it indicates that the virtual content or image displayed by the vehicle-mounted device needs to interact with the external environment outside the vehicle. The virtual content or image is anchored relative to the external environment. The vehicle-mounted device can control the vehicle-mounted device coordinate system and the world coordinate system to remain consistent, so that the vehicle-mounted device will not change with the movement of the vehicle, thereby realizing the anchoring of virtual objects or images relative to the external environment.
[0099] Step S2320, when the current working mode is the driving mode, if the current operating scenario is the second operating scenario, control the vehicle-mounted device coordinate system to remain unchanged relative to the vehicle coordinate system.
[0100] The second operation scene at least includes a display scene, which can be, for example, at least one of a speed display scene, a rotation speed display scene, a fuel level display scene, and a power level display scene. It can be understood that the second operation scene can be other scenes except the first operation scene, that is, the second operation scene is other scenes except the interaction scene, which is not limited in this embodiment.
[0101] In this step S2320, when the current working mode of the vehicle-mounted device is the driving mode, it indicates that the driver is using the vehicle-mounted device. If the vehicle-mounted device is displaying a scene at this time, it indicates that the virtual content or image displayed by the vehicle-mounted device does not need to interact with the external environment outside the vehicle. At this time, the vehicle-mounted device can control the vehicle-mounted device coordinate system and the vehicle coordinate system to remain consistent, so that the vehicle-mounted device changes with the movement of the vehicle, thereby realizing the anchoring of virtual objects or images relative to the vehicle.
[0102] Regarding this step 2320 of controlling the coordinate system of the on-board device to remain unchanged relative to the vehicle coordinate system, you can refer to the above-mentioned embodiments, for example, obtaining the rotation quaternion of the vehicle's first pose data; obtaining the rotation quaternion of the vehicle's real-time pose data; obtaining the rotation quaternion of the real-time pose data of the on-board device; obtaining the rotation quaternion of the converted real-time pose data of the on-board device based on the rotation quaternion of the vehicle's first pose data, the rotation quaternion of the vehicle's real-time pose data and the rotation quaternion of the real-time pose data of the on-board device; determining that the coordinate system of the on-board device remains unchanged relative to the vehicle coordinate system based on the rotation quaternion of the converted real-time pose data of the on-board device.
[0103] According to the embodiment of the present disclosure, when the current working mode of the vehicle-mounted device is the passenger mode, it indicates that the passenger is currently using the vehicle-mounted device. At this time, the coordinate system of the vehicle-mounted device will be controlled to remain unchanged relative to the vehicle coordinate system, that is, the coordinate system of the vehicle-mounted device is consistent with the vehicle coordinate system, thereby realizing the anchoring of the virtual object or image displayed by the vehicle-mounted device relative to the vehicle. When the current working mode of the vehicle-mounted device is the driving mode, it indicates that the driver is currently using the vehicle-mounted device. At this time, the coordinate system of the vehicle-mounted device will be controlled to remain unchanged relative to the world coordinate system according to different selections of the current operation scene of the vehicle-mounted device, or the coordinate system of the vehicle-mounted device will be controlled to remain unchanged relative to the vehicle coordinate system. For example, when it is necessary to interact with the external environment, the coordinate system of the vehicle-mounted device can be controlled to remain unchanged relative to the world coordinate system. At this time, the virtual object or image displayed by the vehicle-mounted device is anchored relative to the external environment. For example, when it is not necessary to interact with the external environment, the coordinate system of the vehicle-mounted device can be controlled to remain unchanged relative to the vehicle coordinate system, thereby realizing the anchoring of the virtual object or image displayed by the vehicle-mounted device relative to the vehicle. In other words, through the embodiment of the present application, passengers and drivers can get a good 6Dof experience when interacting with objects in different coordinate systems.
[0104] <Device Example>
[0105] Figure 3 is a schematic diagram of a control device according to an embodiment, referring to Figure 3 As shown, the control device 300 includes an acquisition module 310 and a control module 320 .
[0106] The acquisition module 310 is used to acquire the current working mode of the vehicle-mounted device; wherein the current working mode includes one of a driving mode and a passenger mode;
[0107] A control module 320, configured to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the vehicle coordinate system when the current working mode is the passenger mode;
[0108] The control module 320 is also used to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the world coordinate system, or to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the vehicle coordinate system according to the current operating scenario of the vehicle-mounted device when the current working mode is the driving mode.
[0109] In one embodiment, the control module 320 is specifically used to: when the current working mode is the driving mode, if the current operating scene is the first operating scene, control the coordinate system of the on-board device to remain unchanged relative to the world coordinate system; when the current working mode is the driving mode, if the current operating scene is the second operating scene, control the coordinate system of the on-board device to remain unchanged relative to the vehicle coordinate system; wherein, the first operating scene includes at least a navigation scene, and the second operating scene includes at least a display scene.
[0110] In one embodiment, the control module 320 is specifically used to: obtain the rotation quaternion of the first pose data of the vehicle; wherein the rotation quaternion of the first pose data is the rotation quaternion of the current pose data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts to run; obtain the rotation quaternion of the real-time pose data of the vehicle; obtain the rotation quaternion of the real-time pose data of the vehicle-mounted device; obtain the rotation quaternion of the converted real-time pose data of the vehicle-mounted device based on the rotation quaternion of the first pose data of the vehicle, the rotation quaternion of the real-time pose data of the vehicle and the rotation quaternion of the real-time pose data of the vehicle-mounted device; determine that the coordinate system of the vehicle-mounted device remains unchanged relative to the vehicle coordinate system based on the rotation quaternion of the converted real-time pose data of the vehicle-mounted device.
[0111] In one embodiment, the acquisition module 310 is specifically used to provide a setting entry; and acquire the working mode set through the setting entry as the current working mode of the vehicle-mounted device.
[0112] In one embodiment, the device 300 also includes a communication module (not shown in the figure) for establishing a short-distance communication connection with the vehicle; based on the short-distance communication connection, receiving the rotation quaternion of the first pose data of the vehicle and the rotation quaternion of the real-time pose data of the vehicle transmitted by the vehicle.
[0113] According to the embodiment of the present disclosure, when the current working mode of the vehicle-mounted device is the passenger mode, it indicates that the passenger is currently using the vehicle-mounted device. At this time, the coordinate system of the vehicle-mounted device will be controlled to remain unchanged relative to the vehicle coordinate system, that is, the coordinate system of the vehicle-mounted device is consistent with the vehicle coordinate system, thereby realizing the anchoring of the virtual object or image displayed by the vehicle-mounted device relative to the vehicle. When the current working mode of the vehicle-mounted device is the driving mode, it indicates that the driver is currently using the vehicle-mounted device. At this time, the coordinate system of the vehicle-mounted device will be controlled to remain unchanged relative to the world coordinate system according to different selections of the current operation scene of the vehicle-mounted device, or the coordinate system of the vehicle-mounted device will be controlled to remain unchanged relative to the vehicle coordinate system. For example, when it is necessary to interact with the external environment, the coordinate system of the vehicle-mounted device can be controlled to remain unchanged relative to the world coordinate system. At this time, the virtual object or image displayed by the vehicle-mounted device is anchored relative to the external environment. For example, when it is not necessary to interact with the external environment, the coordinate system of the vehicle-mounted device can be controlled to remain unchanged relative to the vehicle coordinate system, thereby realizing the anchoring of the virtual object or image displayed by the vehicle-mounted device relative to the vehicle. In other words, through the embodiment of the present application, passengers and drivers can get a good 6Dof experience when interacting with objects in different coordinate systems.
[0114] <Equipment Embodiment>
[0115] Figure 4 FIG. 1 is a schematic diagram of the hardware structure of a vehicle-mounted device according to an embodiment. Figure 4 As shown, the vehicle-mounted device 400 includes a processor 410 and a memory 420 .
[0116] The memory 420 may be used to store executable computer instructions.
[0117] The processor 410 can be used to execute the control method described in the embodiment of the method of the present disclosure according to the control of the executable computer instructions.
[0118] The vehicle-mounted device 400 may be Figure 1 The vehicle-mounted device 100 shown may also be a device with other hardware structures, which is not limited here.
[0119] In another embodiment, the vehicle-mounted device 400 may include the above control device 300 .
[0120] In one embodiment, each module of the above control device 300 can be implemented by the processor 410 running computer instructions stored in the memory 420 .
[0121] <Computer Readable Storage Medium>
[0122] The embodiment of the present disclosure further provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the control method provided by the embodiment of the present disclosure is executed.
[0123] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.
[0124] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.
[0125] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0126] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0127] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0128] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0129] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0130] The flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of an instruction, and the module, a program segment or a part of an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.
[0131] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.
Claims
1. A control method, characterized in that: The method comprises: Acquire a current working mode of the vehicle-mounted device; wherein the current working mode includes one of a driving mode and a passenger mode; When the current working mode is the passenger mode, controlling the coordinate system of the onboard device to remain unchanged relative to the vehicle coordinate system; When the current working mode is the driving mode, the coordinate system of the on-board device is controlled to remain unchanged relative to the world coordinate system according to the current operating scenario of the on-board device, or the coordinate system of the on-board device is controlled to remain unchanged relative to the vehicle coordinate system.
2. The method according to claim 1, characterized in that When the current working mode is the driving mode, controlling the coordinate system of the vehicle-mounted device to remain unchanged relative to the world coordinate system according to the current operating scenario of the vehicle-mounted device, or controlling the coordinate system of the vehicle-mounted device to remain unchanged relative to the vehicle coordinate system, includes: In the case where the current working mode is the driving mode, if the current operating scenario is the first operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the world coordinate system; In the case where the current working mode is the driving mode, if the current operating scenario is the second operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the vehicle coordinate system; The first operation scene at least includes a navigation scene, and the second operation scene at least includes a display scene.
3. The method according to claim 1 or 2, characterized in that: The controlling the on-board device coordinate system to remain unchanged relative to the vehicle coordinate system comprises: Acquire the rotation quaternion of the first pose data of the vehicle; wherein the rotation quaternion of the first pose data is the rotation quaternion of the current pose data of the vehicle recorded by the vehicle-mounted device when the desktop environment of the vehicle-mounted device starts to run; Obtaining the rotation quaternion of the real-time position data of the vehicle; Obtaining the rotation quaternion of the real-time position and posture data of the vehicle-mounted device; Obtaining the rotation quaternion of the converted real-time pose data of the on-board device according to the rotation quaternion of the first pose data of the vehicle, the rotation quaternion of the real-time pose data of the vehicle and the rotation quaternion of the real-time pose data of the on-board device; According to the rotation quaternion of the converted real-time position and posture data of the vehicle-mounted device, it is determined that the coordinate system of the vehicle-mounted device remains unchanged relative to the vehicle coordinate system.
4. The method according to claim 1 or 2, characterized in that: The obtaining of the current working mode of the vehicle-mounted device includes: Provide setting entry; The working mode set through the setting entry is obtained as the current working mode of the vehicle-mounted device.
5. The method according to claim 3, characterized in that: The method further comprises: Establish a short-range communication connection with the vehicle; Based on the short-distance communication connection, a rotation quaternion of the first pose data of the vehicle and a rotation quaternion of the real-time pose data of the vehicle transmitted by the vehicle are received.
6. A control device, characterized in that: The device comprises: An acquisition module, used to acquire a current working mode of the vehicle-mounted device; wherein the current working mode includes one of a driving mode and a passenger mode; A control module, configured to control the coordinate system of the onboard device to remain unchanged relative to the vehicle coordinate system when the current working mode is the passenger mode; The control module is also used to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the world coordinate system, or to control the coordinate system of the vehicle-mounted device to remain unchanged relative to the vehicle coordinate system according to the current operating scenario of the vehicle-mounted device when the current working mode is the driving mode.
7. The device according to claim 6, characterized in that The control module is specifically used for: In the case where the current working mode is the driving mode, if the current operating scenario is the first operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the world coordinate system; In the case where the current working mode is the driving mode, if the current operating scenario is the second operating scenario, controlling the vehicle-mounted device coordinate system to remain unchanged relative to the vehicle coordinate system; The first operation scene at least includes a navigation scene, and the second operation scene at least includes a display scene.
8. The device according to claim 7, wherein the control module is specifically configured to: Obtain the rotation quaternion of the first position data of the vehicle; wherein, The rotation quaternion of the first posture data is the rotation quaternion of the current posture data of the vehicle recorded by the in-vehicle device when the desktop environment of the in-vehicle device starts to run; Obtaining the rotation quaternion of the real-time position data of the vehicle; Obtaining the rotation quaternion of the real-time position and posture data of the vehicle-mounted device; Obtaining the rotation quaternion of the converted real-time pose data of the on-board device according to the rotation quaternion of the first pose data of the vehicle, the rotation quaternion of the real-time pose data of the vehicle and the rotation quaternion of the real-time pose data of the on-board device; According to the rotation quaternion of the converted real-time position and posture data of the vehicle-mounted device, it is determined that the coordinate system of the vehicle-mounted device remains unchanged relative to the vehicle coordinate system.
9. A vehicle-mounted device, characterized in that: The vehicle-mounted equipment includes: Memory for storing executable computer instructions; A processor, configured to execute the control method according to any one of claims 1 to 5 under the control of the executable computer instructions.
10. The vehicle-mounted device according to claim 9, characterized in that: The vehicle-mounted device is a head-mounted display device.
11. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions are executed by a processor to execute the control method according to any one of claims 1 to 5.
