In-vehicle infotainment system, method for implementing extended display, and storage medium

Abstract

The present application relates to the field of car machine systems, and more particularly to a car machine system, a method for implementing extended display, and a computer storage medium. The car machine system comprises: a communication module configured to establish a communication connection with a mobile terminal and receive original control data from the mobile terminal, wherein the original control data comprises sensor data collected by a sensor of the mobile terminal; an adaptation module configured to convert the original control data into pose control data for an extended display device; and a routing module configured to distribute the pose control data to a corresponding extended display device in one or more extended display devices for the corresponding extended display device to update a display picture.

Description

Technical Field

[0001] This invention relates to the field of vehicle infotainment systems, and more specifically to a vehicle infotainment system, a method for implementing extended display, and a computer storage medium. Background Technology

[0002] Extended Reality (XR) devices are used to provide users with an altered reality. The term encompasses Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) devices. Currently, XR devices commonly utilize dedicated XR controllers as handheld control devices for human-computer interaction. However, existing XR controllers are mostly one-to-one controllers, unable to control multiple XR devices simultaneously. Furthermore, in scenarios where multiple XR devices operate concurrently, the need for simultaneous control by multiple controllers inevitably leads to signal delays. Summary of the Invention

[0003] To address or at least mitigate one or more of the above problems, the following technical solutions are provided. Embodiments of this application provide a vehicle infotainment system, a method for implementing extended display, and a computer storage medium, thereby enabling the simultaneous control of multiple in-vehicle extended reality devices using a single external mobile terminal.

[0004] According to a first aspect of the present invention, a vehicle infotainment system is provided, the system comprising: a communication module configured to establish a communication connection with a mobile terminal and receive raw control data from the mobile terminal, wherein the raw control data includes sensor data collected by sensors of the mobile terminal; an adaptation module configured to convert the raw control data into attitude control data for extended display devices; and a routing module configured to distribute the attitude control data to a corresponding extended display device among one or more extended display devices, so that the corresponding extended display device can update its display screen.

[0005] As an alternative or supplement to the above solutions, in a system according to an embodiment of the present invention, the communication module is further configured to: realize wired data transmission with the one or more extended display devices through at least one USB interface; or realize wireless data transmission with the one or more extended display devices based on nested word communication or Bluetooth communication.

[0006] As an alternative or supplement to the above solutions, in a system according to an embodiment of the present invention, the communication module is further configured to: identify the type and model of the mobile terminal when establishing a communication connection with the mobile terminal based on the Bluetooth proprietary communication protocol; determine the communication parameters for data transmission using the Bluetooth proprietary communication protocol based on the type and model of the mobile terminal; and receive the original control data based on the communication parameters.

[0007] As an alternative or supplement to the above solutions, in a system according to an embodiment of the present invention, the communication module is further configured to: establish a communication connection with one or more types of mobile terminals based on the Bluetooth proprietary communication protocol, wherein the types of mobile terminals include at least one of the following: smartphones, tablet computers, wearable smart bracelets, wearable smart headphones, and control handles.

[0008] As an alternative or supplement to the above solutions, in a system according to an embodiment of the present invention, the raw control data includes at least one of the following: angular velocity data collected by the gyroscope sensor of the mobile terminal; key data collected by the key sensor of the mobile terminal; image data collected by the image sensor of the mobile terminal; acceleration data collected by the accelerometer sensor of the mobile terminal; and direction data collected by the geomagnetic sensor of the mobile terminal.

[0009] As an alternative or supplement to the above solution, in a system according to an embodiment of the present invention, the adaptation module is further configured to: perform attitude calculation on the original control data and generate quaternion data of the mobile terminal, so as to be used as the attitude control data for the extended display device, wherein the quaternion data indicates the rotation angle of the mobile terminal in the four axes of the spatial coordinate system.

[0010] As an alternative or supplement to the above solutions, in a system according to an embodiment of the present invention, the communication module includes a clock synchronization unit, which sends a synchronization signal to an extended display device in an enabled state at a preset frequency to achieve clock synchronization between the vehicle system and the extended display device.

[0011] According to a second aspect of the present invention, a method for implementing an extended display is provided, the method comprising the following steps performed by a vehicle infotainment system: A) establishing a communication connection with a mobile terminal and receiving raw control data from the mobile terminal, wherein the raw control data includes sensor data collected by sensors of the mobile terminal; B) converting the raw control data into attitude control data for the extended display device; and C) distributing the attitude control data to a corresponding extended display device among one or more extended display devices, so that the corresponding extended display device can update the display screen.

[0012] As an alternative or supplement to the above solutions, a method according to an embodiment of the present invention further includes: implementing wired data transmission with the one or more extended display devices through at least one USB interface; or implementing wireless data transmission with the one or more extended display devices based on nested word communication or Bluetooth communication.

[0013] As an alternative or supplement to the above solutions, in a method according to an embodiment of the present invention, step A includes: when establishing a communication connection with the mobile terminal based on the Bluetooth proprietary communication protocol, identifying the type and model of the mobile terminal; determining communication parameters for data transmission using the Bluetooth proprietary communication protocol based on the type and model of the mobile terminal; and receiving the original control data based on the communication parameters.

[0014] As an alternative or supplement to the above solution, in a method according to an embodiment of the present invention, step B includes: performing attitude calculation on the original control data and generating quaternion data of the mobile terminal for use as the attitude control data for the extended display device, wherein the quaternion data indicates the rotation angle of the mobile terminal in the four axes of the spatial coordinate system.

[0015] As an alternative or supplement to the above solution, a method according to an embodiment of the present invention further includes: sending a synchronization signal to an extended display device in an enabled state at a preset frequency to achieve clock synchronization between the vehicle system and the extended display device.

[0016] According to a third aspect of the present invention, a computer storage medium is provided, the computer storage medium including instructions that, when executed, perform any one of the methods described according to a second aspect of the present invention.

[0017] The vehicle infotainment system according to one or more embodiments of the present invention can simultaneously access multiple extended display devices and utilize a mobile terminal as a controller for multiple extended display devices. By distributing attitude control data to the corresponding extended display devices, it achieves one-to-many control of in-vehicle XR devices. Furthermore, the vehicle infotainment system according to one or more embodiments of the present invention can establish communication connections with various types of mobile terminals (e.g., smartphones, tablets, wearable smart bracelets, wearable smart headphones) via Bluetooth proprietary communication protocols and uniformly convert sensor data received from the mobile terminals into attitude control data for the extended display devices. This breaks the limitation of prior art requiring dedicated XR controllers and saves on the hardware implementation cost of in-vehicle extended displays. Attached Figure Description

[0018] The above and / or other aspects and advantages of the present invention will become clearer and more readily understood from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are denoted by the same reference numerals. In the drawings: Figure 1 A schematic block diagram of a vehicle infotainment system 10 according to one or more embodiments of the present invention; and Figure 2 This is a schematic flowchart of a method 20 for implementing an extended display according to one or more embodiments of the present invention. Detailed Implementation

[0019] The following detailed description is merely exemplary in nature and is not intended to limit the disclosed technology or its application and use. Furthermore, it is not intended to be bound by any express or implied theory presented in the foregoing technical fields, background art, or the following detailed description.

[0020] In the following detailed description of the embodiments, numerous specific details are set forth in order to provide a more thorough understanding of the disclosed technology. However, it will be apparent to those skilled in the art that the disclosed technology can be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.

[0021] Terms such as "comprising" and "including" indicate that, in addition to the units and steps that are directly and explicitly stated in the specification, the technical solution of the present invention does not exclude the presence of other units and steps that are not directly or explicitly stated. Terms such as "first" and "second" do not indicate the order of the units in terms of time, space, size, etc., but are merely used to distinguish the units.

[0022] In the following, various exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0023] Figure 1 This is a schematic block diagram of a vehicle infotainment system 10 according to one or more embodiments of the present invention. To clearly illustrate the principles of the invention, Figure 1 Other devices that work in conjunction with the vehicle infotainment system 10 are also shown.

[0024] Similar to existing in-vehicle infotainment systems, Figure 1The vehicle infotainment system 10 shown can provide a connection channel for external devices (e.g., mobile terminal 30) to achieve functional complementarity. For example, mobile terminal 30 can be connected to vehicle infotainment system 10 via wired or wireless means, thereby enabling vehicle infotainment system 10 to control mobile terminal, or mobile terminal 30 to control vehicle infotainment system 10. Mobile terminal 30 can be a smartphone. In some scenarios, vehicle infotainment system 10 can use mobile terminal 30 to make calls and play music, while vehicle infotainment system 10 can act as audio output device for mobile terminal 30. Exemplarily, mobile terminal 30 can also be other forms of portable devices, such as laptops, tablets, e-book readers, wearable smart bracelets, smart bands, etc.

[0025] Unlike traditional in-vehicle infotainment systems, Figure 1 The vehicle infotainment system 10 shown can also provide at least one interface for one or more external extended display devices 40 (e.g., VR devices, AR devices, and MR devices), so that the external extended display device 40 can use the mobile terminal 30 as an XR controller for the extended display device 40 via the connection channel between the vehicle infotainment system 10 and the mobile terminal 30. Currently, most integrated or separate extended display devices on the market use dedicated XR controllers, which not only fail to achieve one-to-many control but also increase the hardware implementation cost of in-vehicle extended displays. With the vehicle infotainment system 10 provided by this invention, the external extended display device 40 can use the mobile terminal 30 as an XR controller for the extended display device 40 via the connection channel between the vehicle infotainment system 10 and the mobile terminal 30 (e.g., a Bluetooth connection based on the Bluetooth proprietary communication protocol). This simple functional expansion gives existing mobile terminals XR controller functionality, thereby breaking the limitations of dedicated XR controllers in the prior art and saving on the hardware implementation cost of in-vehicle extended displays.

[0026] like Figure 1 As shown, the vehicle infotainment system 10 includes a communication module 110, an adapter module 120, and a routing module 130.

[0027] Optionally, the communication module 110 of the vehicle infotainment system 10 establishes a communication connection with one or more extended display devices 40 via wired or wireless means. In one example, the communication module 110 implements wired data transmission with the extended display device 40 through at least one interface, which may be an interface conforming to the USB Type-C specification, and may also include a path operating based on the Display Port (DP) protocol and a path operating based on the USB protocol. In another example, the communication module 110 implements wireless data transmission with the extended display device 40 based on nested word communication or Bluetooth (Bluetooth Low Energy, BLE) communication.

[0028] Optionally, the communication module 110 includes a clock synchronization unit, which can send synchronization signals to each of the extended display devices 40 in the enabled state at a preset frequency to achieve clock synchronization between the vehicle system and each of the extended display devices 40. Optionally, the communication module 110 may also include a heartbeat detection unit, which is configured to initiate a heartbeat and continuously detect the heartbeat signal after the extended display device 40 is connected. The heartbeat signal refers to a signal that the communication module 110 sends or receives during communication with the extended display device 40 to determine whether the communication is in a normal or interrupted state. For example, the heartbeat detection unit can send a first data packet to the extended display device 40 at regular intervals. If the extended display device 40 successfully receives the first data packet, it will send a second data packet to the heartbeat detection unit. If the heartbeat detection unit successfully receives the second data packet from the extended display device 40, it indicates that the communication connection between the communication module 110 and the extended display device 40 is normal; if the heartbeat detection unit fails to receive the second data packet from the extended display device 40, it indicates that the communication connection between the communication module 110 and the extended display device 40 is interrupted. If the second data packet from the extended display device 40 is not successfully received, the communication module 110 can also initiate a reconnection mechanism to prevent the extended display device 40 from disconnecting.

[0029] The communication module 110 is further configured to establish a communication connection with the mobile terminal 30 and receive raw control data from the mobile terminal 30 in response to the access of the extended display device 40. Exemplarily, the aforementioned communication connection may include a Bluetooth connection established based on the Bluetooth proprietary communication protocol and other wireless connection methods. The Bluetooth proprietary communication protocol refers to a protocol improved by the device manufacturer based on the classic Bluetooth protocol. Exemplarily, after receiving the access command from the extended display device 40, the communication module 110 of the vehicle system 10 actively initiates a Bluetooth scan, that is, broadcasts preset information of the Bluetooth proprietary communication protocol. When the mobile terminal 30 matches the preset information of the Bluetooth proprietary communication protocol, it initiates Bluetooth authentication with the vehicle system 10. After successful authentication, a communication connection is established between the communication module 110 and the mobile terminal 30.

[0030] Optionally, the communication module 110 is further configured to: identify the type and model of the mobile terminal 30 when establishing a communication connection with the mobile terminal 30 based on the Bluetooth proprietary communication protocol; determine the communication parameters for data transmission using the Bluetooth proprietary communication protocol based on the type and model of the mobile terminal 30; and receive raw control data based on the communication parameters. For example, when the mobile terminal 30 is a smartphone, after successfully establishing a communication connection with the communication module 110, the mobile terminal 30 can obtain the International Mobile Equipment Identity (IMSI) by calling the interface functions provided by the software development kit of its own operating system (e.g., Android operating system), and then obtain and send type and model information to the communication module 110. For example, the type of the mobile terminal may include one or more of the following: smartphone, tablet computer, wearable smart bracelet, wearable smart headset, control handle. For example, the model of the mobile terminal may include the manufacturer information, specific model name, etc. After receiving the type and model of the mobile terminal 30, the communication module 110 can determine the communication parameters for data transmission based on the corresponding information preset in the Bluetooth proprietary communication protocol (e.g., a correspondence table between communication parameters and the type and / or model of the mobile terminal). For example, communication parameters may include transmission time interval, maximum transmission unit, minimum transmission unit, etc.

[0031] The mobile terminal 30 is equipped with one or more sensors, and the sensor data collected by the sensors of the mobile terminal 30 can be used as raw control data for the extended display device 40. Optionally, the raw control data includes at least one of the following: angular velocity data collected by the gyroscope sensor of the mobile terminal; key data collected by the key sensor of the mobile terminal; image data collected by the image sensor of the mobile terminal; acceleration data collected by the accelerometer sensor of the mobile terminal; and direction data collected by the geomagnetic sensor of the mobile terminal.

[0032] The adaptation module 120 of the vehicle infotainment system 10 is configured to uniformly convert the raw control data received from the mobile terminal 30 into attitude control data for the extended display device 40. Optionally, this data conversion process may include using the adaptation module 120 to perform attitude calculations on various types of raw control data (e.g., angular velocity data, acceleration data, and orientation data) and generate quaternion data for the mobile terminal 30 to be used as attitude control data for the extended display device 40, wherein the quaternion data indicates the rotation angle of the mobile terminal 30 in the four axes of the spatial coordinate system.

[0033] Optionally, the adapter module 120 can be further configured to fuse sensor data collected by vehicle sensors (e.g., vehicle-mounted gyroscope sensor, vehicle-mounted accelerometer sensor, vehicle-mounted geomagnetic sensor) with raw control data received from the mobile terminal 30, so as to enhance the accuracy of the generated attitude control data by utilizing vehicle-mounted information.

[0034] Optionally, the adapter module 120 can also be configured to use the Mann filter algorithm to perform anti-shake processing on the attitude control data, so as to avoid the small-range jitter of the attitude control data when the vehicle is bumped up and down, which would cause the target control in the extended display device 40 to be unable to be accurately controlled.

[0035] The routing module 130 of the vehicle infotainment system 10 is configured to distribute attitude control data to the corresponding extended display devices 40, so that the corresponding extended display devices 40 can update their display screens. Optionally, the routing module 130 can be configured to create a local area network (LAN) for the vehicle infotainment system to assign IP addresses to each extended display device 40 within the LAN created for the vehicle infotainment system. For example, the routing module 130 can negotiate the allocation of IP addresses between the vehicle infotainment system and the extended display devices 40 based on a proprietary protocol, enabling the vehicle infotainment system and the extended display devices 40 to communicate within the same network segment within the created LAN. For example, the routing module 130 distributes attitude control data to the corresponding extended display devices 40 based on their IP addresses. With the help of the routing module 130, both unified control of the extended display devices connected to the vehicle infotainment system 10 and individual control of a single extended display device or some of these extended display devices can be achieved.

[0036] The in-vehicle infotainment system 10 according to the present invention can simultaneously access multiple extended display devices and utilize a mobile terminal (e.g., a smartphone) as a controller for multiple extended display devices. By distributing attitude control data to the corresponding extended display devices, it achieves one-to-many control of in-vehicle XR devices. Furthermore, the in-vehicle infotainment system 10 according to the present invention can establish communication connections with various types of mobile terminals (e.g., smartphones, tablets, wearable smart bracelets, wearable smart headphones) via a Bluetooth proprietary communication protocol and uniformly convert sensor data received from the mobile terminals into attitude control data for the extended display devices. This breaks the limitation of prior art requiring dedicated XR controllers and saves on the hardware implementation cost of in-vehicle extended displays.

[0037] Figure 2 A schematic flowchart illustrating a method 20 for implementing an extended display according to an embodiment of the present invention is shown. Figure 2 As shown, method 20 includes a vehicle infotainment system (e.g., Figure 1The vehicle infotainment system 10 shown executes the following steps: Step S210, establishing a communication connection with the mobile terminal and receiving raw control data from the mobile terminal, wherein the raw control data includes sensor data collected by the sensors of the mobile terminal; Step S220, converting the raw control data into attitude control data for the extended display device; and Step S230, distributing the attitude control data to the corresponding extended display device for updating the display screen. The steps of method 20 can be carried out with reference to the specific workflow of the vehicle infotainment system described above, and the relevant content is also cited here. Due to space limitations, they will not be elaborated here.

[0038] Optionally, method 20 may further include: implementing wired data transmission with the extended display device via at least one USB interface; or implementing wireless data transmission with the extended display device based on nested word communication or Bluetooth communication.

[0039] Optionally, step S210 includes: identifying the type and model of the mobile terminal when establishing a communication connection with the mobile terminal based on the Bluetooth proprietary communication protocol; determining the communication parameters for data transmission using the Bluetooth proprietary communication protocol based on the type and model of the mobile terminal; and receiving raw control data based on the communication parameters.

[0040] Optionally, step S220 includes: performing attitude calculation on the original control data and generating quaternion data of the mobile terminal for use as attitude control data for the extended display device, wherein the quaternion data indicates the rotation angle of the mobile terminal in the four axes of the spatial coordinate system.

[0041] Optionally, method 20 may further include: sending a synchronization signal to an extended display device that is in an enabled state at a preset frequency to achieve clock synchronization between the vehicle system and the extended display device.

[0042] Furthermore, as described above, the present invention can also be implemented as a computer storage medium storing information for causing a computer to perform, such as Figure 2 The procedure for the method shown is as follows. Here, various types of computer storage media can be used, such as disks (e.g., magnetic disks, optical disks, etc.), cards (e.g., memory cards, optical cards, etc.), semiconductor memory (e.g., ROM, non-volatile memory, etc.), and tapes (e.g., magnetic tape, cassette tape, etc.).

[0043] Where applicable, the various embodiments provided by the present invention may be implemented using hardware, software, or a combination of hardware and software. Furthermore, where applicable, without departing from the scope of the invention, the various hardware and / or software components described herein may be combined into composite components comprising software, hardware, and / or both. Where applicable, without departing from the scope of the invention, the various hardware and / or software components described herein may be divided into sub-components comprising software, hardware, or both. Additionally, where applicable, it is contemplated that software components may be implemented as hardware components, and vice versa.

[0044] The software (such as program code and / or data) according to the invention can be stored on one or more computer storage media. It is also contemplated that the software identified herein can be implemented using one or more networked and / or otherwise general-purpose or special-purpose computers and / or computer systems. Where applicable, the order of the various steps described herein can be changed, combined into compound steps, and / or divided into sub-steps to provide the features described herein.

[0045] The embodiments and examples presented herein are provided to best illustrate embodiments of the invention and its particular applications, thereby enabling those skilled in the art to practice and use the invention. However, those skilled in the art will understand that the above description and examples are provided merely for ease of illustration and example. The descriptions presented are not intended to cover all aspects of the invention or to limit the invention to the precise forms disclosed.

Claims

1. A vehicle infotainment system, characterized in that, The system includes: A communication module configured to establish a communication connection with a mobile terminal and receive raw control data from the mobile terminal, wherein the raw control data includes sensor data collected by sensors of the mobile terminal; An adaptation module is configured to perform attitude calculation on the original control data and generate quaternion data for the mobile terminal, which is used as attitude control data for the extended display device. The quaternion data indicates the rotation angle of the mobile terminal along the four axes of the spatial coordinate system. The adaptation module is further configured to fuse sensor data collected by the vehicle's sensors with the original control data to enhance the accuracy of the attitude control data. The routing module is configured to enable the vehicle-mounted system to create a local area network (LAN) and assign IP addresses of one or more extended display devices to the LAN, allowing the vehicle-mounted system and the one or more extended display devices to communicate within the same network segment. The routing module also distributes the attitude control data to the corresponding extended display devices among the one or more extended display devices, so that the corresponding extended display devices can update the display screen.

2. The system according to claim 1, wherein the communication module is further configured to: Wired data transmission with the one or more extended display devices is achieved via at least one USB interface; or Wireless data transmission with the one or more extended display devices is achieved through nested word communication or Bluetooth communication.

3. The system according to claim 1, wherein the communication module is further configured to: When establishing a communication connection with the mobile terminal based on the Bluetooth proprietary communication protocol, the type and model of the mobile terminal are identified; Based on the type and model of the mobile terminal, the communication parameters for data transmission are determined using the Bluetooth proprietary communication protocol; and The original control data is received based on the communication parameters.

4. The system according to claim 1, wherein the communication module is further configured to: Establish a communication connection with one or more types of mobile terminals based on the Bluetooth proprietary communication protocol, wherein the types of mobile terminals include at least one of the following: smartphone, tablet computer, wearable smart bracelet, wearable smart headset, and control handle.

5. The system of claim 1, wherein the raw control data includes at least one of the following: Angular velocity data collected by the gyroscope sensor of the mobile terminal; Key press data collected by the key sensor of the mobile terminal; Image data acquired by the image sensor of the mobile terminal; Acceleration data collected by the accelerometer sensor of the mobile terminal; as well as Directional data collected by the geomagnetic sensor of the mobile terminal.

6. The system according to claim 1, wherein the communication module includes a clock synchronization unit, the clock synchronization unit sending a synchronization signal to the extended display device in an enabled state at a preset frequency to achieve clock synchronization between the vehicle system and the extended display device.

7. A method for implementing extended display, characterized in that, The method includes the following steps performed by the vehicle infotainment system: A. Establish a communication connection with the mobile terminal and receive raw control data from the mobile terminal, wherein the raw control data includes sensor data collected by the sensors of the mobile terminal; B. Perform attitude calculation on the original control data and generate quaternion data of the mobile terminal to be used as attitude control data for the extended display device, wherein the quaternion data indicates the rotation angle of the mobile terminal in the four axes of the spatial coordinate system, and perform data fusion of sensor data collected by the vehicle sensor with the original control data to enhance the accuracy of the attitude control data. as well as C. Create a local area network for the vehicle infotainment system and assign IP addresses of one or more extended display devices to the local area network, so that the vehicle infotainment system and the one or more extended display devices can communicate in the same network segment, and distribute the attitude control data to the corresponding extended display devices among the one or more extended display devices so that the corresponding extended display devices can update the display screen.

8. The method according to claim 7, wherein the method further comprises: Wired data transmission with the one or more extended display devices is achieved through at least one USB interface; or Wireless data transmission with the one or more extended display devices is achieved through nested word communication or Bluetooth communication.

9. The method of claim 7, wherein step A comprises: When establishing a communication connection with the mobile terminal based on the Bluetooth proprietary communication protocol, the type and model of the mobile terminal are identified; Based on the type and model of the mobile terminal, the communication parameters for data transmission are determined using the Bluetooth proprietary communication protocol; and The original control data is received based on the communication parameters.

10. The method of claim 7, wherein the method further comprises: A synchronization signal is sent at a preset frequency to the extended display device that is in the enabled state to achieve clock synchronization between the vehicle system and the extended display device.

11. A computer storage medium, characterized in that, The computer storage medium includes instructions that, when executed, perform the method according to any one of claims 7-10.

Images