Gaming in vehicle environment

Abstract

There is provided a gaming input provider (500) for an automotive operation system, AOS, system (100), which operates according to an AOS, the AOS including a AOS standard gaming input application programming interfaces, APIs, block (502) requiring, in input, a gaming input signal (512) of a pre-defined format and providing, to a higher layer gaming engine (204), a standardized version of the gaming input signal (212) of the pre-defined format, the gaming communication controller comprising: a communication service (102) to wirelessly receive a gaming input signal (302) from a mobile device (300); a hardware abstraction layer, HAL, block (202) to convert the gaming input signal (302, 112) onto the pre-defined format.

Description

[0001] Gaming in vehicle environment

[0002] There are disclosed techniques for gaming communications in an automotive operating system (AOS). An application regards providing input to a gaming engine.

[0003] Vehicles often make use of applications operating according to AOSs, e.g. for entertainment, gaming, etc. A well-known AOS is Android AOS (AAOS).

[0004] Many AOSs (and in particular AAOS) provide application programming interfaces (APIs), which allow unified access to peripheral devices. An APIs block may be a HID (Human Interface Device) input (which may include game controllers as a subset), which provides standard APIs, output a standardized output value, which may be a sensed value from a joystick. The standardized APIs block is meant to receive a standardized gaming input, in a pre-defined, standard format. However, the standardized input APIs block has been proven particularly difficult to be adapted or customized to several applications. Adaptation would need to happen inside the gaming engine. The gaming engine, e.g. as developed by the authors of the gaming engine, expects standard HID APIs. If we would like the gaming engine to directly talk to the communication interface 102, then each gaming engine would need to be changed. Indeed, the APIs are resident, and cannot be easily changed. Or, the APIs could be changed, to deviate from the AOS, which would increase the burden for the compatibility with the application layer. APIs could be received from a remote server, for example, but in that case time delay should be necessary. Moreover, a continuous changing of these APIs and routines may endanger the operability of the application. In particular, several AOSs (and in particular AAOS) are quite irksome when concerning the adaptation of the transmission and processing of signals in formats different from the required one.

[0005] Some kinds of inputs are therefore impossible without APIs. Not all the possible inputs are provided, and this suboptimal, because there would be, in theory, several possible techniques for providing an input, and not only through a wired joystick.

[0006] Summary

[0007] There is provided a gaming input provider for an automotive operation system, AOS, system, which operates according to an AOS, the AOS including a AOS standard gaming input application programming interfaces, APIs, block requiring, in input, a gaming input signal of a pre-defined format and providing, to a higher layer gaming engine, a standardized version of the gaming input signal of the pre-defined format, the gaming communication controller comprising: a communication service to wirelessly receive a gaming input signal from a mobile device (300); a hardware abstraction layer, HAL, block to convert the gaming input signal onto the pre-defined format.

[0008] There is also provided a gaming method for an automotive operation system, AOS, system, which operates according to an AOS, the AOS including a AOS standard gaming input application programming interfaces, APIs, block requiring, in input, a gaming input signal of a pre-defined format and providing, to a higher layer gaming engine, a standardized version of the gaming input signal of the pre-defined format, the method comprising: wirelessly receiving a gaming input signal from a mobile device; converting the gaming input signal onto the pre-defined format.

[0009] Figures

[0010] Figs. 1 and 2 show examples.

[0011] Description

[0012] Here below there is an example of a gaming input provider 500, which is mostly considered to include a communication service 102 and a HAL block 202.

[0013] Here below there is also discussed an example (e.g. Fig. 1) of a gaming engine 204 for a gaming activity operating at an application layer 105, a standardized APIs block 502 (HID) for providing a APIs gaming input signal 512 according to a pre-defined format, and a communication service 102 providing the input signal.. The gaming input 302 may be provided, for example, from movement of a mobile device 300, and may be processed to emulate the input of another physical input device, such as a joystick.

[0014] In particular, there are examples of tricking the standardized APIs block (HID) 502 by inputting a gaming input signal 212 converted onto a format accepted by the APIs block (HID) 502. The APIs block 502 "believes" to be inputted with a standard HAL signal 212 which provides, as could be understood from its format, an input signal from a joystick. Therefore, the standardized APIs block 502 is "tricked" and provides a standardized gaming input signal 512 to the gaming engine 204 without necessity of reinstalling the gaming engine 204.

[0015] An output 512 of the standardized APIs block 502 (here indicated as processed version 212 of the gaming input signal 112) may be provided to the higher application layer 105 (e.g. to the gaming engine 204). The gaming engine 204 may, for example, provide gaming content 262 to be displayed in a display 252 (e.g. through a standardized video output APIs block 263 and, downstream, a video output HAL 564)

[0016] Fig. 1 shows an example of the environment in which the present technique operates. Reference numeral Reference numeral 200 refers to a vehicle environment (e.g., a car or another vehicle). The vehicle environment 200 includes a head unit 250. The head unit 250 may be part of a dashboard of the vehicle 200, or anyhow connected to it (e.g., fixedly mounted, e.g. integrally part of it). The head unit 250 be or include a hardware component onto which software components operate, and therefore includes processing units (e.g. CPUs, GPUs, processors, microcontrollers, FPGAs, etc.).

[0017] The head unit 250 may include an automotive operating system (AOS) system 100 and an application layer 105 with a media communication application 260 operating on the AOS system 100. The applications of the application layer 105. The AOS system 100 may be implemented in software and hardware. The AOS system 100 may provide specific services to the applications 260. The AOS system 100 may operate according to an AOS. The AOS may provide specific services (e.g. APIs) to particular higher-layer applications (e.g. 260). An example of the AOS may be Android Automotive Operating System (AAOS). Other examples include a Linux-based system and QNX (QNX).

[0018] The AOS system 100 may include, as a software element, a standardized gaming I HID input APIs block 502. The standardized gaming I input APIs block 502 may be understood, for example, as a virtual sensor (e.g., a virtual joystick) which receives a gaming input signal 212 according to a format specified by the AOS. Application program interfaces (APIs) and routines through which the standardized gaming I HID input APIs block 502 operate are in principle standardized and not modifiable if not with a new release or a reinstallation of the AOS. The standardized gaming I HID input APIs block 502 may have been conceived to be inputted with the gaming input signal 212 having a pre-defined format which is the format of a signal provided by a wired joystick (e.g. a value which increases with the increase of the orientation of the bar of the joystick). Accordingly, the standardized gaming I HID input APIs block 502 is meant to be inputted with a gaming input signal formatted according to a straightforward format without too many possibilities for customization or formatting or adaptation. The HAL 202 may be inputted with an input gaming signal 112 which has been obtained from the movements of the mobile device 300. Fig. 1 shows that media content 262 may be provided to a output media APIs block 263, to a output media HAL block 564, and it loudspeakers and / or a display 252. Otherwise, the media content 262 (or at least a portion of it) can be provided to the mobile device 300. Fig. 1 shows that the processed version 512 of the API's standardized gaming input signal 512 as outputted by the APIs block 502 is provided to the gaming engine 204. The gaming engine 204 may provide gaming content and / or media content to a remote provider (not shown in Fig. 1). The gaming engine 204 may define new gaming content to be provided in feedback (e.g. to display device(s) and / or loudspeaker(s) of the vehicle and / or to the mobile device 300, which in turn will activate its own display unit(s) and / or loud- speaker(s)).

[0019] It is now explained how the gaming input signal 212, inputted to the APIs (gaming I HID) block 502, is obtained. In an internal space (e.g., cockpit or vehicle cockpit) 210 there is a mobile device 300, e.g. non-physically connected to the head unit 250. The mobile device 300 is capable of obtaining movement signal regarding its own movement e.g. through inertial sensors (accelerometers, gyroscopic sensors, etc.). The mobile device 300 may be connected, for example, to the head unit 250 (and more in particular to the communication service 102) a local communication network (e.g. WLAN, Wi-Fi, Bluetooth, Zig Bee).

[0020] As shown in Fig. 1, the head unit 250 may include the communication service 102 which receives the video input signals from the mobile device 300.

[0021] Fig. 1 shows that the communication service 102 is part of the AOS system 100. However, this is not strictly necessary: the communication service 102 may be an external to the AOS system 100 in some examples. In particular, the communication service 102 may operate with its own routines and / or APIs, which are not necessarily based on the routines and APIs of the HAL 202.

[0022] The communication service 102 may receive the gaming input signal 302, e.g. through a wireless local communication connection, from the mobile device 300, and provided it as 112 (302 and 112 may indicate the same information, in some examples, without important processing). The gaming input signal 302 (e.g. including the movement input signal indicative of the movement of the mobile device 300) may be in a particular format which is indicative of the movement, but which is not the format that the APIs block 502 shall use. For example, the format of the gaming input signal 302 may not emulate that of a joystick, and / or it may have a different number of bits, a different representation, etc.. However, the HAL block 202 may convert the gaming input signal 302 (112) onto the predefined format 212. The APIs block 202 may provide the gaming input signal 302 (in its APIs version 512) to the gaming engine 204. It is possible (like in Fig. 2) to apply a different strategy according to which the gaming input signal 302 is transmitted to the communication service 102 not through a local connection (such as WiFi, WLAN, Bluetooth, etch.) but through a remote connection (e.g., TCP / IP, internet, mobile communication, 5G, etc.). The gaming input signal 302 may be provided, in remote, to a remote gaming provider 400 which, in turn, can send, through another remote connection (e.g., TCP / IP, internet, mobile communication, 5G, etc.), to the communication service 102. Once the communication service 102 has received the gaming input signal 302, the communication service 102 may provide the gaming input signal 302to the HAL block 202, and then the process could continue like in Fig. 1.. This solution is easier (because it is not necessary to insert the WLAN password, for example) but is also less reliable and causes a higher latency than that of Fig. 1. In particular, the remote connection through the provider 400 is particularly advantageous in the case of using the authenticating information 251 (see below).

[0023] Therefore, while in some examples there is only the structure of Fig. 1 and in some other examples there is only the structure of Fig. 2, in some examples, there is choice between:

[0024] 1) a local connection (like in Fig. 1) for transmitting the gaming input signal 302 to the APIs block 502 (e.g. through the communication service 102 and the HAL block 202);

[0025] 2) a remote connection (like in Fig. 2) for transmitting the gaming input signal 302 to the APIs block 502 (e.g. through the communication service 102 and the HAL block 202) through a remote gaming engine provider.

[0026] In other terms, the gaming input provider 500, in case the pre-defined format does not present a latency lower than a predetermined latency threshold required for the game, may choose a local connection between the communication service (102) and the mobile device (300) instead of a remote connection.

[0027] In other examples, the chosen connection is the fasted, whether it is remote or wireless.

[0028] The game may be at least partially controlled by the mobile device 300. In general examples, however, it may normally occur: the gaming engine 204 may process at least part of the game; the remote gaming provider 400 may process at least part of the game (e.g., at least a connection with other players); the mobile device 300 may process only the user interface, e.g. to provide the gaming input signal 302 (in particular the movement input signal indicative of the movement of the mobile device 300).

[0029] An installation procedure may be performed, e.g. downloading code from the remote gaming provider 400 to the AOS system 100. The gaming engine 204 (or at least a part therefore) may be setup, for example, during an installation procedure, e.g. from the remote gaming provider 400. The operations of the communication controller may be setup during the installation procedure. The APIs block 502 may be setup during the installation procedure. Notably, the APIs block (HID) 502, which is already according to the particular operating system (e.g. AAOS), can be maintained unchanged during the installation procedure. The communication service 102 may be setup during the installation procedure. Importantly, the installation procedure may be performed without APIs for processing the gaming input signal (either in its version 112 inputted to the APIs block, or in its version outputted by the APIs block). Therefore, the APIs block can be maintained unchanged during the installation procedure. Hence, compatibility issues are avoided.

[0030] It is to be noted that the input to the gaming APIs block (HID) 502 was natively imagined to be that from a joystick, i.e. relating to a relative position between a movable bar and a fixed base of the joystick. However, the movement of the mobile device 300 is in general detected through accelerometric (inertial) measurements. Hence, it may be that the values to be provided to the APIs block 502 shall be subjected to a physical conversion. For example, the HAL block 202 may convert the received gaming input signal 112 (302) from acceleration to speed (e.g. through mathematical integration), or from acceleration to position (e.g. through double mathematical integration), or from position to acceleration (e.g. through second derivative), or from position to speed (e.g., through first derivative). In alternative, it may be the mobile device 300 which provides the already-converted values. This is not

[0031] Therefore, there may be two different types of conversions (and the HAL block 202 may perform both or one of them):

[0032] 1) A data-structure conversion (i.e. a format conversion), i.e. from a data structure provided by the mobile device 300 to the data structure requested by the APIs block 502, and from one format of the received gaming input signal 302 onto the format requested by the APIs block.

[0033] 2) A physical-value conversion, i.e. from a physical magnitude to a different physical magnitude (e.g. from acceleration to speed, etc. as discussed above). The communication service 102 may provide an identification sign 251 to an output unit (e.g. a display) 252 of the head unit 250 or more in general of the vehicle 200. The output unit 252 may output (e.g. visualize) authenticating information 251 (the identification sign). The authenticating information 251 may be unique information, such as a QR code, a barcode, or another code. The authenticating information 251 may be displayed as a static image, in some examples. The authenticating information 251 may be acquired by the mobile device 300. For example, the mobile device 300 may have an embedded camera 305 for acquiring the authentication information 251 (the QR code, a bar-code, or another code), e.g. under the user's command. The mobile device 300 may therefore decode the authenticating information 251 (e.g., as code obtained from the QR code, or other identification sign) and route the mobile device's authenticating information 251 to the remote gaming provider 400 (in some examples based on Fig. 1, the remote gaming provider 400 could have the only aim of allowing the authorization). The authenticating information 251 (e.g. identification sign) may provide at least partially encrypted information, but the mobile device 300 may notwithstanding send the encrypted information without decrypting it. A password may be encompassed in the authenticating information 251. An example of the mobile device's authenticating information may be a URL or an-other type of address of a web page. The authenticating information 251 may therefore open a particular webpage from the remote gaming provider 400. The webpage may provide an executable code for providing a user interface for providing gaming content, for example. The authenticating information 251 is in principle uniquely associated with the vehicle 200. The remote gaming provider 400 may therefore include a database coupling the mobile device's authenticating information as received from the mobile device 300 with the authenticating information as expected and, in case of a positive comparison (i.e., in case the mobile device's authenticating information is the same of the expected authenticating information or differs for a negligible amount, e.g. for less than a predetermined threshold), then the authentication information is positive, so that the mobile device 300 can be considered authenticated. In case of negative result of the comparison (i.e., in case the mobile device's authenticating information differs from the expected authenticating information, or differs for a non-neg- ligible amount, e.g. for more than a predetermined threshold), then a negative authentication information may be provided to the communication service 102 or no authentication information at all. The authenticating information 251 (e.g. QR code or other identification sign) may in principle constantly be the same for the same vehicle 200 and in theory never change. In theory, the authenticating information 251 can be pre-stored in a read only memory (ROM) of the AOS system 100. In some examples, however, the authenticating information 251 may be subsequently updated. Accordingly for example, the remote gaming provider 400 may provide an update authenticating information to the communication service 102. The update authenticating information may be for example a new authenticating information 251 (e.g. a new Q code or new identification information, distinct from the previous one). In other examples, the authenticating information 251 is modified only partially (e.g., only part of the QR code is modified, or some other kind of incremental modification may be performed). After the update the authenticating information 251 information has been provided from the head unit remote gaming provider 400, the authenticating information 251 changes (and the authenticating information as expected at the head unit remote gaming provider 400 changes, as well). Therefore, if the mobile device 300 tries to use an old authenticating information 251 (e.g., a QR code of the day before), it may be that the mobile device's authenticating information is not recognized anymore as authentic, and the mobile device 300 is not given the access to the game.

[0034] Notably, at least some of the following advantages can be attained examples above:

[0035] 1) It is not strictly necessary, for the mobile device 300, to have an application preinstalled

[0036] 2) The mobile device 300 does not necessarily need to receive APIs

[0037] 3) The communication service 102 does not need to exchange any password with the mobile device 300

[0038] 4) The mobile device 300 does not need log in a WiFi or other local network (at least for the authentication)

[0039] 5) The remote gaming provider 400 does not need to provide APIs to the communication service 102.

[0040] There is disclosed a media communication method ....

[0041] Discussion

[0042] Remote Game Controller in a car

[0043] Android Automotive Operating System (AAOS) is available for the head-units of cars: https: / / developer.android.com / ca rs?hl=en

[0044] AAOS also contains sensor and input device HALs (HAL= Hardware Abstraction Layer), which is an abstraction to integrate e.g., game controllers to be recognizable by Android:

[0045] Our invention includes using a mobile device (phone, tablet, etc.) 300 to generate such input data (e.g., by moving the mobile device through the air like air console https: / / www.airconsole.com / ), and to connect such device via a remote connection with the head-unit (client / server). Then, the mobile device 300 will generate sensor and user input data 302 and will send such data to a software component 102 running on the AAOS head unit 250. This component will then feed the data to the AAOS HALs 202, emulating built-in sensors and Bluetooth- or USB-connected game controllers, such that the input data complies with the standard Android APIs.

[0046] Hence, any game running on AAOS can use the mobile phone 300 for remote control e.g. via the cloud and does not require adaptations for this, and no other connection to the car 200 is required.

[0047] Additional Embodiments: The type of input data generated by the mobile device game controller may exceed the standard interfaces provided by the AAOS HALs. In this case (Fig. 2) the mobile device game controller (102) will use a side channel to directly communicate with a dedicated Android Service, which allows bidirectional communication with the AAOS games. In this case, AAOS games are adapted for the interpretation of such additional data and the provisioning of additional meta data to the mobile device game controller to allow for game-specific, dynamic user interfaces.

[0048] Additional Embodiments: Combination of the embodiment with a QR Code 251. Connect any mobile device 300 with a head-unit, and exchange data bi-directionally with the head-unit, and let the mobile device control aspects of the head-unit. This is done by a software running on a head-unit and logging into a cloud service. The car is required to be uniquely identified, this will be done with e.g., a unique device ID. Then on request of the head-unit UX, a QR code will be displayed on the screen of the head-unit. This will allow any passenger to scan the QR code with their mobile device, which will open a web app connecting with the cloud service. Now the software on the head-unit can exchange (bidirectional) data with the mobile device.

[0049] In an embodiment, the mobile devices / phones are used to generate sensor data to remotely control games, and the mobile device or phone is - via a combination with the AAOS HALs - exposed as a generic game controller.

[0050] It is to be mentioned here that all alternatives or aspects as discussed before and all aspects as defined by independent claims in the following claims can be used individually, i.e., without any other alternative or object than the contemplated alternative, object or independent claim. However, in other embodiments, two or more of the alternatives or the aspects or the independent claims can be combined with each other and, in other embodiments, all aspects, or alternatives and all independent claims can be combined to each other.

[0051] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0052] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed.

[0053] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0054] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0055] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

[0056] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0057] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

[0058] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

[0059] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0060] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0061] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0062] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

[0063] 1. Mobile device or method of operating a mobile device as described above.

[0064] 2. Head unit or method of operating a head unit as described above.

[0065] 3. Cloud service or method of operating a cloud service as described above.

[0066] 4. Communication method as described above.

[0067] 5. Computer program configured for performing any one of the methods in aspects 1

[0068] - 3 when running on a computer or a processor.

Claims

Claims1. A gaming input provider (500) for an automotive operation system, AOS, system (100), which operates according to an AOS, the AOS including a AOS standard gaming input application programming interfaces, APIs, block (502) requiring, in input, a gaming input signal (512) of a pre-defined format and providing, to a higher layer gaming engine (204), a standardized version of the gaming input signal (212) of the pre-defined format, the gaming communication controller comprising: a communication service (102) to wirelessly receive a gaming input signal (302) from a mobile device (300); a hardware abstraction layer, HAL, block (202) to convert the gaming input signal (302, 112) onto the pre-defined format.

2. The gaming input provider claim 1, configured to wirelessly receive, from the mobile device (300), the gaming input signal (302) as including a movement signal indicative of movement of the mobile device (300), the HAL block (202) being configured to convert the movement signal (302) onto a HAL-specific format indicative of a movement (212).

3. The gaming input provider of any of the preceding claims, configured to transmit gaming content (262) to the mobile device (300).

4. The gaming input provider of any of the preceding claims, configured to have a local wireless connection with the mobile device (300).

5. The gaming input provider of any of the preceding claims, configured to have a remote wireless connection with the mobile device (300) through a remote gaming provider (400).

6. The gaming input provider of any of the preceding claims, configured to choose between a remote wireless connection with the mobile device (300) and a local wireless connection with the mobile device (300).

7. The gaming input provider of any of the preceding claims, the gaming input provider (500) being configured to choose a connection with lower latency between a remote connection and a local connection between the communication service (102) and the mobile device (300)8. The gaming input provider of any of the preceding claims, further configured to perform a physical-value conversion of the gaming input signal (302) from a first physical magnitude to a second physical magnitude, the second physical magnitude being the magnitude considered by the gaming engine.

9. The gaming input provider of any of the preceding claims, configured to provide an identification sign (251) recognizable by the mobile device (300), so that the mobile device (300) independently authenticates with the remote gaming provider, so that the game controller receives gaming media streams son after the independent authentication of the mobile device with the remote gaming provider.

10. The gaming input provider of claim 9, configured to receive the identification sign, or unique information for generating the information sign, from a remote gaming provider, the game controller being configured to render the identification sign according to the unique information received from the remote gaming provider.

11. The gaming input provider of claim 9 or 10, wherein the identification sign is a QR code.

12. The gaming input provider of any of the preceding claims, wherein the AOS is Android AOS, AAOS.

13. The gaming input provider of any of the preceding claims, wherein the APIs block is a HID block.

14. A gaming method for an automotive operation system, AOS, system (100), which operates according to an AOS, the AOS including a AOS standard gaming input application programming interfaces, APIs, block (502) requiring, in input, a gaming input signal (512)of a pre-defined format and providing, to a higher layer gaming engine (204), a standardized version of the gaming input signal (212) of the pre-defined format, the method comprising: wirelessly receiving a gaming input signal (302) from a mobile device (300); converting the gaming input signal (302, 112) onto the pre-defined format.

15. A non-transitory storage unit storing instructions which, when executed by a processor, cause the processor to perform the method of claim 14.

Images