An electronic device, a vehicle and a method for communicating control information

Abstract

An electronic device is disclosed. The electronic device comprises memory circuitry, processing circuitry and an interface. The electronic device is configured to obtain sensor data from one or more sensors. The electronic device is configured to determine one or more physical inputs. The electronic device is configured to determine user data indicative of a user behaviour such as by applying a machine learning model to the sensor data and the one or more physical inputs. The electronic device is configured to communicate control information based on the user data.

Description

[0001] AN ELECTRONIC DEVICE, A VEHICLE AND A METHOD FOR COMMUNICATING CONTROL INFORMATION

[0002] The present disclosure pertains to the field of vehicles, such as partially or fully autonomous vehicles. The present disclosure relates to an electronic device, a vehicle, and a related method for communicating control information.

[0003] BACKGROUND

[0004] As vehicles are getting more autonomous, there is uncertainty in the autonomous systems operating the vehicles due to the lack of transparency. Regardless of the level of autonomy, it is envisaged that there will be human occupants in vehicles either as a driver and / or passenger(s).

[0005] For example, Autonomous Vehicles, AVs, are foreseen to have a potential for improving the safety and comfort of travel. However, there is uncertainty about the intent of the system supporting autonomous operations.

[0006] Further, there is a variety of different driving styles and / or preferences that, if not adhered to, may result in discomfort of passenger(s) and potentially a distrust of the autonomous system.

[0007] SUMMARY

[0008] Accordingly, there is a need for vehicles, electronic devices and methods for communicating control information of a vehicle system, which may mitigate, alleviate or address the existing shortcomings and may allow determining a user behaviour (such as a driver and / or passenger behaviour reaction), adapting a control operation of the vehicle system accordingly, and communicating information regarding the control operation, for example, to the user.

[0009] An electronic device is disclosed. The electronic device comprises memory circuitry, processor circuitry and an interface. The electronic device is configured to obtain sensor data from one or more sensors. The electronic device is configured to determine one or more physical inputs. The electronic device is configured to determine user data indicative of a user behaviour such as by applying a machine learning model to the sensor data and the one or more physical inputs. The electronic device is configured to communicate control information based on the user data.

[0010] A method performed by an electronic device is disclosed. The method comprises comprising obtaining sensor data from one or more sensors. The method comprises determining one or more physical inputs. The method comprises determining user data indicative of a user behaviour such as by applying a machine learning model to the sensor data and the one or more physical inputs. The method comprises communicating control information based on the user data.

[0011] A vehicle is disclosed. The vehicle comprises memory circuitry, processor circuitry and an interface. The vehicle comprises an electronic device according to this disclosure.

[0012] It is an advantage of the present disclosure that the disclosed electronic device and the disclosed method may assist the man-machine interaction between a user and the electronic device, thereby enabling an improvement of potential comfort, confidence and trust in a vehicle equipped with an autonomous capability (such as the autonomous driving systems of the autonomous vehicle, such as a partially or fully AV).

[0013] The disclosed electronic device and the disclosed method may reduce the number of unnecessary manual overrides, such as carried out by a driver of the vehicle, by communicating the control information based on the user data.

[0014] Further, the disclosed electronic device and the disclosed method may provide an evaluation of a driver / passenger vigilance and / or alertness to ensure the driver of the vehicle is ready to take control if necessary (such as in case avoiding action is required).

[0015] It may be appreciated that the disclosed technique allows for an automated adjustment of operations of the vehicle (such as a personalisation of the operations and / or parameters thereof).

[0016] BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features and advantages of the present disclosure will become readily apparent to those skilled in the art by the following detailed description of examples thereof with reference to the attached drawings, in which:

[0018] Fig. 1 is a diagram illustrating an example vehicle and an example electronic device according to this disclosure,

[0019] Fig. 2 is a diagram illustrating an example vehicle according to this disclosure, and

[0020] Fig. 3 is a flow-chart illustrating an example method, performed in an electronic device according to this disclosure. DETAILED DESCRIPTION

[0021] Various examples and details are described hereinafter, with reference to the figures when relevant. It should be noted that the figures may or may not be drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures. It should also be noted that the figures are only intended to facilitate the description of the examples. They are not intended as an exhaustive description of the disclosure or as a limitation on the scope of the disclosure. In addition, an illustrated example needs not have all the aspects or advantages shown. An aspect or an advantage described in conjunction with a particular example is not necessarily limited to that example and can be practiced in any other examples even if not so illustrated, or if not so explicitly described.

[0022] The figures are schematic and simplified for clarity, and they merely show details which aid understanding the disclosure, while other details have been left out. Throughout, the same reference numerals are used for identical or corresponding parts.

[0023] A vehicle can be seen as a means of transportation. Examples of vehicles include a car, a truck, a van, a motorcycle, a train, an airplane, and / or a vessel. The vehicle may have an autonomous capability providing various levels of autonomous driving, for example ranging from lane assist to fully autonomous. An autonomous capability may be seen as a function, feature, or system that enables a vehicle to be partially or fully operated without real-time human intervention.

[0024] The Society of Automotive Engineers (SAE) International's standard J3016 defines six levels of autonomous driving ranging from level 0 (fully manual) to level 5 (fully autonomous). For example, in level 0 (so called No Driving Automation), the vehicle is manually controlled by a driver which provides the "dynamic driving task" although there may be systems in place to help the driver, (such as emergency braking system which technically doesn’t "drive" the vehicle, and thereby does not qualify as automation or autonomy). For example, in level 1 (so called Driver Assistance), the vehicle is controlled by the driver which has drivers support features for driver assistance, such as steering or accelerating (such as cruise control) while a driver monitors the other aspects of driving. Examples of level 1 element of autonomous capability include adaptive cruise control (ACC) where the vehicle can be kept at a safe distance behind the vehicle in front. This may be seen as the lowest level of automation. For example, in level 2 (so called Partial Driving Automation), the vehicle can control both steering and accelerating / decelerating, such as in advanced driver assistance systems (ADAS). For example, in level 2, a human sits in the driver’s seat and can take control of the car at any time. For example, in level 3 (so called Conditional Driving Automation), a vehicle has “environmental detection” capabilities and can make informed decisions autonomously, such as accelerating past a slow-moving vehicle, while still providing the possibility of a human override. For example, in level 3, the driver is to remain alert and ready to take control if the system is unable to execute the task. For example, in level 4 (so called High Driving Automation), a vehicle can intervene if things go wrong or there is a system failure. For example, in other words, level 4 vehicles do not require human interaction in most circumstances, however, a human still has the option to manually override. For example, level 4 vehicles can operate in self-driving mode, but in areas limited by geofencing. For example, level 4 vehicles can be used for ride sharing.

[0025] For example, in level 5 (so called Full Driving Automation), a vehicle does not require human attention and is not limited to certain areas and / or conditions.

[0026] In the present disclosure, the vehicle may for example be partially autonomous (such as SAE levels 1 , 2 or 3, such as semi-autonomous). In some examples, the vehicle is fully autonomous (such as SAE level 4 or 5, such as a driver-less vehicle).

[0027] Fig. 1 is a diagram illustrating an example vehicle 1 and an example electronic device 300 according to this disclosure.

[0028] Vehicle 1 can be seen as an equipment for transporting one or more items (such as one or more users, and / or one or more inert objects) from a first location to a second location. The vehicle 1 may be associated with a user in that the vehicle may be transporting the user. In some examples, the user may be a driver of the vehicle 1 , (such as driver 10 shown in Fig. 2). In some examples, the user may be a passenger of the vehicle 1 , (such as passenger 12 shown in Fig. 2).

[0029] The vehicle 1 may have some autonomous capability. Vehicle 1 is for example partially autonomous or fully autonomous (such as SAE levels 1-5).

[0030] The vehicle comprises one or more sensors 100. In some examples, the one or more sensors comprise an optical sensor 101 , a heart rate sensor 102, and / or a microphone 103. In some examples, the vehicle comprises a display 201.

[0031] In one or more examples, the vehicle 1 comprises an electronic device 300 disclosed herein.

[0032] In one or more examples, the electronic device 300 is separate from the vehicle 1. The electronic device 300 comprises memory circuitry 301 , processor circuitry 302, and an interface 303. In some examples, the processor 302 can be seen as an autonomous vehicle (AV) computer. The interface 303 may be configured for wired and / or wireless communications via a wired and / or wireless communication system.

[0033] The electronic device 300 is configured to obtain (such as receive and / or retrieve) sensor data from one or more sensors. The electronic device 300 is configured to obtain the sensor data from the one or more sensors, for example via interface 303. The sensor data can be seen as data obtained using the one or more sensors 100. The sensor data is for example associated with the vehicle 1 and / or with the user. In some examples, the sensor data may be associated with a user (such as passenger and / or driver) of the vehicle 1 . In one or more example electronic devices, the sensor data comprises one or more of: optical sensor data, head position data, gaze sensor data, movement data, audio data, and heart rate data. In one or more example electronic devices, the one or more sensors comprise one or more of: an optical sensor, a heart rate sensor, and a microphone. An example of sensor data associated with the user includes heart rate data, and / or other health data. In some examples, other suitable sensors can be considered.

[0034] The optical sensor 101 can for example be seen as a visual sensor. In other words, the optical sensor 101 may be configured to detect light (such as visible, ultraviolet, infrared, etc.). Example of optical sensors include a camera and / or other suitable optical sensors. The optical sensor 101 is for example configured to obtain optical data. For example, the optical sensor 101 may be a camera. In some examples, the optical data comprises data indicative of the visual activity of a user, such as gaze information (such as gaze direction) and / or facial expression and / or pupil size. The electronic device 300 is for example configured to obtain, such as using the optical sensor 101 , optical sensor data, head position data, gaze sensor data, and / or movement data. The optical sensor data can be seen as optical data obtained by the optical sensor 101. For example, the optical sensor data comprises information associated with the head and / or face of a user, such as a facial expression and / or gaze information of the user. The electronic device 300 can for example be configured to detect the facial expression of a user based on the optical sensor data. In one or more examples, the optical sensor data may comprise data indicative of gaze and / or change in skin colour (such as skin becoming more and more red) and / or data indicative of a facial expression of a user, such as fear, anger, relaxed, etc.

[0035] Head position data for example comprises information indicative of the head position of a user.

[0036] The head position data may for example be obtained by tracking the head position of a user, such as using the optical sensor 101. In some examples, obtaining the head position data comprises tracking the head position of a user by aligning viewpoints in 3D space such that a correct perspective is obtained. Gaze sensor data for example comprises information indicative of the gaze of a user, such as gaze direction. In other words, the gaze sensor data may comprise information indicative of the direction which a user is looking in (as shown in by driver gaze 11 and passenger gaze 13 of Fig. 2). The gaze sensor data may for example be based on the head position of a user, the eye direction of a user, etc. Gaze sensor data can for example be seen as gaze detection data. The gaze sensor data may for example comprise a point at which a user is looking (such as a focus, such as a point of focus). The gaze sensor data for example comprises a field of view of the user. For example, the gaze sensor data may comprise information associated with the peripheral vision of the user.

[0037] The movement data is for example associated with the user. The movement data for example comprises information indicative of the movement of the user (such as the driver and / or the passenger). For example, the movement data may comprise information indicative of the movement of the limbs (such as arms, legs, hands, and / or feet).

[0038] The heart rate sensor 102 is for example configured to detect the heart rate of the user. In some examples, the heart rate sensor 102 may be worn by the user, and thereby associated with the user. For example, the user may wear a wearable electronic device (such as smart watch) comprising a heart rate sensor 102 configured to detect the heart rate of the user. The electronic device 300 is for example configured to obtain, such as using the heart rate sensor, heart rate data. For example, the heart rate sensor 102 is configured to obtain heart rate data of the user (such as driver and / or passenger of the vehicle 1 ). The heart rate data for example comprises information indicative of the heart rate of a user. For example, the heart rate data comprises information indicative of the heart beats per minute (BPM) of a user.

[0039] The electronic device 300 is for example configured to obtain, using the microphone, audio data, such as audio data associated with a user. For example, the microphone 103 may be configured to obtain audio data indicative of the speech of a user. In some examples, the user may utter some words that may be indicate discomfort, anxiety, and / or interrogation regarding the operations of the vehicle, and the microphone 103 may capture audio data indicative of such discomfort, anxiety, and / or interrogation regarding the operations of the vehicle, for further processing.

[0040] In one or more examples, the electronic device 300 comprises the one or more sensors 100. In other words, in some examples, the one or more sensors can be internal sensors of the electronic device 300. In some examples, the one or more sensors can be sensors external to the electronic devices, such as one or more sensors of the vehicle and / or one or more sensors external to the vehicle 1 (such as roadside cameras).

[0041] In one or more example electronic devices, the one or more sensors comprise one or more sensors associated with the vehicle (for example, one or more sensors 100). In one or more example electronic devices, the one or more sensors comprise one or more sensors associated with a user device. The user device is for example a device of a user. The user device is for example a portable device (such as a portable electronic device). For example, user device is an electronic device (such as a smart phone, a wearable, a laptop, etc.) of a user. The user device for example comprises an optical sensor 101 , a heart rate sensor 102, and / or a microphone 103.

[0042] The electronic device 300 is configured to determine (such as via processor circuitry 302) one or more physical inputs, such as to a vehicle. The one or more physical inputs can for example be seen as one or more inputs of a user to the vehicle 1. In other words, the one or more physical inputs can be seen as physical inputs provided by a user for control of the vehicle 1. The one or more physical inputs may comprise physical body movement of the user (such as a gesture and / or a pose), and / or physical movement of the vehicle. In some examples, determining one or more physical inputs can be seen as obtaining data based on one or more physical inputs. For example, determining one or more physical inputs comprises obtaining physical input data, such as physical input sensor data, indicative of one or more physical inputs. The one or more physical inputs may for example comprise one or more inputs to the vehicle, for example one or more inputs that accelerate the vehicle 1 , steer the vehicle 1 in any direction, slows down the vehicle 1 , change a mode of the vehicle 1 (such as an electrical and / or mechanical mode), etc.

[0043] The term “physical input” can be used interchangeably with the term “vehicle input”. Vehicle input can be seen as input provided by the user to the vehicle. For example, the physical input is an input to a vehicle, wherein the vehicle is operatively coupled to the electronic device.

[0044] In some examples, the electronic device 300 is configured to determine, based on the sensor data, the one or more physical inputs. For example, when the vehicle 1 is autonomous or semi- autonomous, the user may press the brake pedal to slow the car (such as when a user feels there is imminent danger). The electronic device 300 may be configured to determine (such as obtain) the physical input of pressing the brake pedal. In some examples, the electronic device 300 determines the physical input based on the sensor data. For example, a user about to press the brake pedal may be captured by the sensor data and the electronic device can then identify the physical input even when the brake system has not detected it. Stated differently, the electronic device can determine, based on sensor data, a physical input that results from reflexive and / or apprehensive movement of the user that may or may not lead to engagement of the brakes and / or steering wheel but is an indication of discomfort and / or lack of trust.

[0045] In some examples, the determination of the one or more physical inputs comprises determining the number of occupants inside the vehicle 1 . For example, the electronic device 300 may determine the number of occupants (such as passengers) in the vehicle 1 using a scale and / or a balance (such as located in a seat of vehicle 1). In some examples, sensor data comprises data obtained using a scale and / or balance of the vehicle 1. The vehicle 1 may for example be configured to determine the location of one or more users (such as occupants) of the vehicle 1 (such as based on sensor data).

[0046] In one or more example electronic devices, the one or more physical inputs comprise one or more of: a steering wheel input, a pedal input, an accelerator input, a brake input, and / or a manual override. In some examples, the steering wheel input, pedal input, accelerator input, and / or brake input is performed by a user (such as driver 10 of Fig. 2). The steering wheel input is for example associated with a rotation of the steering wheel. The pedal input is for example associated with the pedal being pressed. For example, the pedal input may be a value associated with a force applied to the pedal. The accelerator input is for example associated with the accelerator pedal being pressed. For example, the accelerator pedal input may be a value associated with a force applied to the accelerator pedal. The brake input is for example associated with the brake pedal being pressed. For example, the brake pedal input may be a value associated with a force applied to the brake pedal. For example, a manual override can be a disengagement of a given control system, such as ACC.

[0047] In some examples, the physical inputs are obtained from one or more components of the vehicle. Components of the vehicle 1 for example include a steering wheel, pedals (such as accelerator pedal, brake pedal and / or clutch pedal), one or more buttons (such as a clutch button, handbrake button, etc.), one or more levers (such as a handbrake lever), and / or one or more shifters (such as one or more paddle shifters and / or gearstick, the gearstick for example being associated with a sequential and / or H-pattern gearbox)). In some examples, the components include a toggle (such as a switch, a button, etc.) for manual override of the autonomous driving mode of the vehicle 1. In other words, the vehicle 1 may comprise a toggle for enabling and / or disabling manual control (such as by a user) of the vehicle 1. The electronic device 300 is configured to determine (such as via processor circuitry 302) user data indicative of a user behaviour.

[0048] The user data can for example be seen as data characterizing a behaviour of the user (such as a driver or passenger of the vehicle 1). For example, the user data may be seen as user behaviour data. The user behaviour can for example be seen as a behaviour associated with a user, such as an expression of the user regarding comfort. For example, a user data of a user may be indicative of the user having one or more traits (such as characteristics, such as casual, stressed, in need of reassurance and / or information, pre-emptive, slow-moving, etc. For example, a user in need of reassurance and / or information may want more information to be provided by the electronic device 300. A casual passenger may want less information to be provided by the electronic device 300. The user behaviour is for example associated with the attention and / or awareness that a given user is allocating to the task of driving the vehicle 1. For example, the user data may comprise information indicative of a driving style of a user and / or a type of passenger. For example, the user data may comprise one or more user attributes characterizing the user behaviour, such as a user profile, a user category, and / or a user type and associated parameters.

[0049] In one or more example electronic devices, the user data comprises a facial expression, a comfort level, and / or a user profile. The user data for example comprises a facial expression parameter, a comfort level parameter, and / or a user type. The facial expression parameter may include a pupil size parameter for the user. The comfort level is for example indicative of how comfortable the user is. The user profile can be seen as a profile associated with the user. For example, the user profile may be stored on the electronic device 300 (such as memory circuitry 301 ). In some examples, the user profile may be stored on a different electronic device (such as an external server, the user device, etc.). The user profile for example comprises one or more control parameters associated with the user. The one or more control parameters may comprise parameters associated with one or more of: braking, acceleration, steering, distance to other vehicles (such as vehicle in front), lane keeping, amount of visual and / or audio feedback (such as user interface objects) provided (such as provided to a user).

[0050] In some examples, the user profile may be generated by the trained machine learning model and optionally based on user input. For example, a user may be providing various user information to an application and / or to a vehicle, which may be used to enrich the training and / or the execution of the machine learning model. In some examples, the information used for enriching the machine learning model is based on confidence metrics of the machine learning model. For example, the electronic device determines whether a confidence metric obtained for user data meets a criterion (such as by comparing with a threshold), and requests user input when the confidence metric does not meet the criterion. In some examples, the user may input user information into the electronic device 300 and / or any component of the vehicle (such as via a dashboard interface, audio request, user device, etc). In some examples, a user may be associated with one or more user profiles. For example, the electronic device may provide a user interface allowing a user to select from one or more user profiles (such as when a user is tired, they may select a corresponding profile).

[0051] In one or more example electronic devices, the electronic device 300 is configured to determine the user data by applying (such as via memory circuitry 301 and / or processor circuitry 303) a machine learning model to the sensor data and the one or more physical inputs. In one or more example electronic devices, the electronic device 300 characterizes the user behaviour by applying the machine learning model to the sensor data and the one or more physical inputs. For example, the electronic device 300 determines the user behaviour by extracting patterns of user behaviour based on the sensor data and / or one or more physical inputs of the user. The output of the machine learning model may be seen as a classified user data, such as user data providing a classification of a user behaviour and / or a user comfort.

[0052] In one or more example electronic devices, the machine learning model is configured to determine the user data based on the sensor data and the physical input(s). In some examples, the machine learning model is a classification model, such as a classifier. In some examples, the machine learning model is configured to (e.g., via training of the machine learning model) determine (such as detect) an expression of a user, such as a facial expression of the user, by exploiting the sensor data (such as one or more of: optical sensor data, head position data, gaze sensor data, movement data, and heart rate data) and the one or more physical inputs (such as one or more of: a steering wheel input, a pedal input, an accelerator input, a brake input, and / or a manual override). For example, when the driver is scared, the electronic device 300 may determine (by applying the machine learning model to the sensor data and the physical input(s) that the user is scared. For example, the sensor data may include optical data showing facial expression of discomfort and fear, and the physical input(s) indicating an erratic steering of the steering wheel. For example, the machine learning model generates, based on the learned user reactions (provided by the physical inputs, the sensor data for heart rate, movement, head position, gaze, and / or the previous determined facial expressions, etc.), a prediction of the user behaviour, which may be mapped onto a user profile.

[0053] In one or more example electronic devices, the machine learning model comprises one or more of: a convolutional neural network, a recurrent neural network, and a multimodal learning model with transformers. The machine learning model for example takes the sensor data and / or the one or more physical inputs as input. The machine learning model for example provides the user data indicative of a user behaviour as an output. Machine learning may be seen as identifying and / or recognizing patterns in existing data (such as sensor data and / or one or more physical inputs) to facilitate making classification and / or predictions for subsequent data, such as user data, and / or communication information. The machine learning model may be created based on example and / or previous (such as historical) inputs to make valid and reliable classifications for novel inputs and / or outputs. Additionally, or alternatively, the disclosed machine learning model and related machine learning programs may be trained by inputting sample data sets, certain data and / or previous (such as historical) data into the programs, such as sensor data and / or one or more physical inputs. The machine learning model may be configured to perform a numerical analysis thereof. The machine learning model may utilize deep learning algorithms that may be primarily focused on pattern recognition, and may be trained after processing multiple examples. In some examples, the electronic device applies a federated or incremental learning to update, tune and / or improve the machine learning model(s).

[0054] The convolutional neural network (CNN) can be seen as a type of neural network including a plurality of convolutional layers with a set of learnable filters (or kernels), and optionally a pooling layer, and optionally an output layer, which may be fully connected. For example, the convolutional layer applies filters to the input to extract features, the pooling layer down samples the input to reduce computation, and the output layer may make the final output or prediction. The CNN may employ forward propagation and / or backward propagation. For example, forward propagation processes the input data (such as sensor data and / or physical input(s)) from the input layer to the output layer. For example, in backward propagation, an error between the predicted / classified output and the actual detection is fed back into the input layer to reduce the error for subsequent operations.

[0055] The recurrent neural network can be seen as a type of neural network that allows previous outputs to be used as inputs while having hidden states. The recurrent neural network is for example a bidirectional neural network. For example, the recurrent neural network can use their internal state (such as memory) to process variable length sequences of inputs. The recurrent neural network can be a Long Short-Term Memory, LSTM, which uses a gating mechanism to selectively remember or forget previous inputs.

[0056] The multimodal learning model with transformers can be seen as a transformer that is independent of the modality or type of input data (such as audio, video, text, image). A transformer is configured to encode input data (such as sensor data and physical input(s)) and output the user data. The multimodal learning model with transformers can take as input various types of sensor data from various types of sensors: optical sensors, actuators, and / or microphones. The multimodal learning model with transformers used in this disclosure can provide user data that is based on emotion detection, such as facial expression.

[0057] In other words, the machine learning model may be trained on sensor data and physical inputs, such as from a plurality of users. In other words, the machine learning model may be trained on historical data (such as previously processed data) comprising historical user data and historical sensor data and / or historical physical inputs. In some examples, the training data can include user data of other users (such as other than driver 10 and passenger 12 of Fig. 2). The machine learning model may be trained using supervised or unsupervised machine learning. In some examples, the machine learning model can be seen as classifier configured to classify the sensor data and physical input(s) into user classes and / or user profiles. In some examples, the machine learning model is pre-trained to be tuned to a user, such as in a reinforcement learning manner. In some examples, the electronic device or another device may train the machine learning model (e.g., using historical user data, historical sensor data, and / or historical physical inputs) so that the machine learning model is capable of determining the user data based on the sensor data and the physical input(s), e.g. with sufficient accuracy.

[0058] In some examples, the machine learning model may be applied by one or more of: the electronic device 300 (such as by processor 302) and / or a server device. For example, the machine learning model may be applied in collaboration with another electronic device (such as an external server) and the electronic device 300. In some examples, the training of the machine learning model may be performed by a server device and the trained machine learning model may then be provided to the electronic device 300.

[0059] The electronic device 300 is configured to communicate (such as via the interface 303) control information based on the user data. In some examples, the electronic device 300 is configured to communicate the control information based on user data to an external device and / or to the user (such as the driver and / or the passenger of the vehicle 1 ), optionally via an interface of the vehicle. Control information can for example be seen as the information causing control the vehicle 1. In other words, the vehicle 1 may be configured to operate according to the control information. The control information comprises for example control parameters and / or control data for controlling the vehicle. In one or more example electronic devices, the electronic device 300 is configured to communicate the control information to a part of the vehicle for controlling an operation of the vehicle. For example, the electronic device 300 may be configured to generate control information based on the user data and transmit the control information to one or more component of the vehicle 1 , causing the vehicle to perform according to the control information. The electronic device 300 may be configured to classify sensor data and physical input(s) (which capture user reactions) and tune control parameters (such as automation parameters) accordingly (such as adapt braking characteristics according to user reactions). In one or more example electronic devices, the control information comprises one or more of: control parameter for operating the vehicle, and assistance information to the user. The control parameter for operating the control of the vehicle can be seen as a parameter associated with the operation of a vehicle. Operating the vehicle 1 may comprise controlling the vehicle 1 , such as driving, steering, manoeuvring the vehicle 1. In other words, the vehicle 1 may be operated according to one or more control parameters such as speed, direction, acceleration, etc. The vehicle 1 may for example be operating according to one or more control parameters. For example, a control parameter may include one or more of: brake smoothness parameter indicative of the smoothness of the brake application, a throttle smoothness parameter indicative of the smoothness of the throttle application, and a steering wheel smoothness parameter indicative of the smoothness of the rotation of the steering wheel. The control parameters for example comprise one or more values.

[0060] The assistance information for example comprises instructions for control of the vehicle 1 to be performed by the user. For example, the assistance information may comprise one or more action requests indicating one or more actions to be performed by the user (such as the driver and / or the passenger). For example, the assistance information may comprise a message indicating to a user that there is a vehicle in the blind spot of vehicle 1.

[0061] In one or more example electronic devices, the electronic device 300 is configured to identify the user based on the sensor data. For example, the electronic device 300 may be configured to identify the user based on optical sensor data, head position data, gaze sensor data, movement data, audio data, and heart rate data. In some examples, the electronic device 300 may be configured to identify the user based on the one or more physical inputs. In one or more examples, the electronic device 300 may be configured to carry out recognition techniques, such as facial recognition and / or geofencing of a user device. For example, when a user is in close proximity to the vehicle 1 and / or is located inside vehicle 1 , the electronic device 300 may be configured to perform recognition techniques on the user. The electronic device 300 may select, according to the output of the recognition techniques, a user profile. In other words, the electronic device 300 may be configured to select, using one or more recognition techniques, the user profile corresponding with a user. In one or more example electronic devices, the electronic device 300 is configured to communicate by displaying a user interface object indicative of the control information. In some examples, the electronic device 300 is configured to communicate to the user by displaying a user interface object indicative of the control information. For example, the electronic device 300 is configured to display a user interface object to a user (such as the driver and / or the passenger of the vehicle 1 ) on a display device (such as a display device of the vehicle 1 , and / or a display device of the electronic device 300, and / or a display device of a user device).

[0062] In one or more example electronic devices, the displaying of the user interface object comprises projecting the user interface object on a surface of the vehicle and / or a retina of the user. The surface of the vehicle 1 can for example be any surface of the vehicle 1 , such as the window, windscreen, dashboard, roof, doors, seats, etc. In some examples, the displaying the user interface object comprises projecting the user interface object on a display device of the vehicle 1 . In some examples, a display device can be seen as a device configured to display a projection, such as a projector. For example, the dashboard and / or the windscreen of vehicle 1 may be configured to act as a display device. The electronic device 300 is for example configured to display (such as project), using photonic crystal technology, the user interface object on the windscreen. The user interface object is for example based on the sensor data, user data, and / or the control information. In some examples, where the user interface object is displayed (such as projected) is based on the sensor data, such as the gaze sensor data, movement data and / or head position data. For example, the user interface object may be projected onto a surface (such as the surface being determined based on sensor data) at which the user is looking. In other words, the surface onto which the user interface object is projected may be determined based on gaze sensor data and / or head position data. For example, the electronic device 300 is configured to projecting, using a retinal projection system, the user interface object on a retina of the user.

[0063] In one or more example electronic devices, the displaying of the user interface object comprises causing the display of the user interface object on a display device. The display device can be seen as a device configured to display, such as to display a user interface object. The display device for example comprises an electronic display or screen configured to display a user interface object. The display device is for example a display device of the vehicle 1. For example, the display device may be a dashboard display, a headrest display, a display on the steering wheel, a wingmirror display, a rear-view mirror display, etc. In some examples, the display device is a user device (such as smartphone, wearable, laptop, glasses, contact lenses, etc.) of the user. In one or more example electronic devices, the user interface object comprises one or more augmented reality objects configured to support navigation and / or to provide information of surroundings of the vehicle. For example, the electronic device 300 may be configured to augment real world objects, such as using Computer Generated Imaging (CGI). In some examples, the display of the user interface object(s) (such as augmented reality object(s)) may be based on the sensor data and / or the one or more physical inputs. For example, the user interface object may be displayed (such as projected) on a surface corresponding with the gaze sensor data of the user. In other words, the location of the user interface object may be updated according to where the user is looking at a given moment. The gaze sensor data and / or the head movement data may advantageously enable the user interface object to be displayed in a correct perspective with respect to a user’s viewpoint (such as the eyes of the user, such as the driver and / or passenger of vehicle 1 ). In some examples, the electronic device 300 may be configured to display or to cause the display of a user interface object based on the sensor data, user data and / or one or more physical inputs. For example, the electronic device 300 may be configured to display, based on the comfort level of a user, a user object interface representative of the control information. In some examples, the electronic device 300 may be configured to provide information of the surroundings by providing information indicative of the area surrounding the vehicle 1 (such as other vehicles close to vehicle 1). In other words, the electronic device 300 may be configured to provide information of the surroundings of the vehicle 1 by providing information indicative of the dynamic surroundings of the vehicle 1. In some examples, the user interface object comprises information indicative of the awareness (such as a situational awareness) of the vehicle 1. For example, the electronic device 300 may be configured to provide information indicative of vehicles, pedestrians, roadside objects, etc.

[0064] The one or more augmented reality objects for example comprise bounding boxes associated with a vehicle, a line indicative of the determined trajectory of a vehicle, etc. In some examples, the electronic device 300 may be configured to indicate (such as to a user), the intent or intention (such as such as the intended trajectory) of the vehicle 1 . For example, a user interface object (such as a line) representative of the determined trajectory of the vehicle 1 may be projected (such as by projector 202 of Fig. 2) onto any surface of the vehicle 1 (such as the windscreen) and / or the surface on which the car is driving (such as the road, such as gravel road, tarmac road, asphalt road, etc.) for the electronic device to communicate control information regarding the intent of the vehicle 1 to the user.

[0065] In some examples, the one or more augmented reality objects comprise arrows (such as arrows, such as coloured arrows) indicating braking or acceleration of a vehicle. For example, the one or more augmented reality objects comprise red arrows indicating a vehicle ahead decelerating towards the vehicle 1. For example, the one or more augmented reality objects comprise green arrows indicating a vehicle ahead accelerating away from the vehicle 1 .

[0066] In some examples, the displaying of the user interface object comprises projecting one or more augmented reality objects onto the retina of a user.

[0067] In some examples, displaying a user interface object may re-assure the user that the vehicle 1 has a correct detection of the surroundings. The augmented reality objects for example can support the man-machine interaction between a user of the vehicle 1 and the vehicle 1 in operating the vehicle 1 with a correct interpretation of the environment and / or surroundings. In other words, the augmented reality objects may improve awareness (such as situational awareness, awareness of how the vehicle 1 is navigating, etc.), thereby enabling improved control of the vehicle 1 . For example, the augmented reality object can help the user confirming or disapproving, to the vehicle, certain aspects of the environment detected by the vehicle.

[0068] In other words, the electronic device 300 is configured to provide an adapted, context dependent and / or user dependent visual feedback and / or audio feedback to the user. For example, the electronic device 300 is configured to display information indicative of the environment and / or surroundings which the vehicle 1 has obtained. For example, the electronic device 300 may be configured to display (such as to a user) the road curvature.

[0069] In one or more example electronic devices, the electronic device 300 is configured to communicate by emitting one or more audio signals indicative of the control information. For example, one or more audio signal the electronic device 300 may be connected to one or more loudspeakers, earphones, headphones and / or any other type of audio device. In some examples, the electronic device 300 may determine, such as based on the sensor data and / or user data, that more attention, awareness and / or alertness of the user is required. For example, in a demanding situation, such as high density of traffic, bumpy road, etc., the electronic device 300 may communicate to the user of the vehicle, such as via a speaker of the vehicle 1 and / or of a user device. In some examples, the electronic device 300 is configured to determine a situation specific “sufficient alertness level” of a user (such as the driver) of the vehicle 1. In some examples, the sufficient alertness level is determined by applying the machine learning model to the sensor data.

[0070] The electronic device 300 may communicate to a user of the vehicle 1 , based on whether a user (such as the driver) of vehicle 1 is sufficiently alert regarding the task of driving vehicle 1. For example, when the electronic device 300 determines that a user is sufficiently alert, the electronic device 300 may refrain from communicating with the user. For example, when the electronic device 300 determines that a user is not sufficiently alert, such as based on sensor data and / or user data, the electronic device 300 may communicate with the user. For example, the electronic device 300 may communicate to the user a request for the user to increase their level of alertness, attention, and / or awareness. The electronic device 300 may for example be configured to determine when a user is sufficiently alert for a given scenario and to communicate accordingly. For example, a user may need to be highly alert on a narrow road with a high traffic concentration and the electronic device 300 adjusts operations accordingly.

[0071] The electronic device 300 may control an automation level of the vehicle 1 , based on whether a user (such as the driver) of vehicle 1 is sufficiently alert regarding the task of driving vehicle 1. For example, when the sensor data indicates that the user is not sufficiently alert, the electronic device may adjust the automation level to keep the vehicle safe. This may be increasing and / or decreasing the automation level.

[0072] The electronic device 300 may be configured to, such as based on the user data, refrain from unnecessary display of information. In some examples, the electronic device 300 may be configured to, such as based on the situational awareness, refrain from unnecessary display of information.

[0073] In some examples, the electronic device 300 may be configured to determine whether to display a user interface object to a user. For example, when the vehicle 1 is stationary and / or when the user is not moving (such as sleeping), the electronic device 300 may be configured to refrain from displaying a user interface object.

[0074] In one or more example electronic devices, the electronic device 300 is configured to receive feedback from the user via one or more of: sensor data, a user interface input, and an audio signal. The feedback from the user for example comprises user input that may be a confirmation, an affirmation, a correction, a disapproval (such as positive, neutral and / or negative feedback). The feedback from the user may for example be indicative of a user comfort and / or user satisfaction regarding the operation (such as the driving and / or communication of control information). The user may for example provide feedback during and / or after the ride. The electronic device 300 is for example configured to receive feedback from a user via voice (such as a voice of the user, such as via microphone 103). For example, the feedback may comprise information indicative of a new pothole in a road. For example, the feedback indicative of the new pothole in the road may comprise gaze sensor data (such as the user looking at the pothole). The user interface input can be seen as an input (such as provided by a user of the vehicle 1 ). For example, the user may press one or more buttons, touchscreens, etc., on the dashboard of the vehicle to provide feedback to the vehicle (such as the presence and / or location of a pothole in the road). The audio signal may for example be obtained by the microphone 103 of the one or more sensors. The audio signal may for example comprise a voice signal (such as speech) provided by a user. The audio signal may comprise feedback from the user of the vehicle 1 . For example, the user may state the location of a pothole.

[0075] In some examples, the electronic device 300 can be seen as executing a closed loop system, the closed loop for example being enabled by feedback, such as the received feedback from the user. For example, the machine learning model determines, such as based on the sensor data and / or the one or more physical inputs, the user data. The control data (based on user data) may then be communicated to the user. The user may provide feedback based on the communicated control information. As the control information is based on the user data, this feedback may also apply to the user data. For example, the electronic device 300 is thereby configured to, based on the received feedback (such as from a user), update the user data indicative of a user behaviour.

[0076] In one or more example electronic devices, the electronic device 300 is configured to adjust operations of the vehicle based on the feedback and / or the user data, and / or one or more navigation parameters, and / or an energy consumption parameter. The navigation parameters are for example associated with the navigation of the vehicle 1. The navigation parameter and / or the energy consumption parameter can be for example associated with the speed limit of the vehicle 1 and / or a driving profile of the vehicle 1 . The driving profile of the vehicle is for example an economic driving profile where unnecessary breaking is to be avoided, and the acceleration profile is adjusted for saving energy. For example, the electronic device 300 may be configured to increase and / or decrease the energy consumption parameter. In some examples, the electronic device 300 is configured to adjust a travel time parameter based on the feedback and / or the user data. For example, the electronic device 300 may be configured to obtain a selection of a driving profile by the user which may increase and / or decrease the energy consumption parameter.

[0077] For example, the electronic device 300 may be configured to adjust the speed limit of the vehicle. The navigation parameter, energy consumption parameter and / or time consumption parameter may be associated with the speed limit of the vehicle 1 and / or a driving profile of the vehicle.

[0078] In one or more example electronic devices, the vehicle is capable of autonomous operation or semi-autonomous operation. In some examples, the vehicle 1 is capable of semi-autonomous operation. In some examples, the vehicle 1 is capable of autonomous operation, such as any of levels 1-5 of the SAE standard. In semi-autonomous vehicles the electronic device 300 may be configured to, based on the user data, sensor data and / or the one or more physical inputs, determine whether a driver of vehicle 1 is sufficiently alert (such as is the driver falling asleep at the wheel). For example, the electronic device 300 may be configured to determine whether the eyes of the driver are drooping and / or whether the eyes of the user are looking through the windscreen. In some examples, the electronic device 300 may be configured to, based on the user data, sensor data and / or the one or more physical inputs, communicate control information to the user indicating a lack of alertness and / or control of the user (such as such that the user would increase alertness and / or control).

[0079] Processor circuitry 302 is optionally configured to perform any of the operations disclosed in Fig. 3 (such as any one or more of S202, S204, S206, S208). The operations of the electronic device 300 may be embodied in the form of executable logic routines (for example, lines of code, software programs, etc.) that are stored on a non-transitory computer readable medium (for example, memory circuitry 301) and are executed by processor circuitry 302).

[0080] The electronic device 300 may be configured to perform any of the methods disclosed in Fig. 3. The electronic device 300 is for example configured to communicate with a user device, such as the user device (such as user device 30 of Fig. 2) disclosed herein. The electronic device 300 can be seen as a communication device, such as for communicating control information to a user. The electronic device 300 can for example be seen as an AV computer configured to interact with a user. The electronic device 300 can be seen as a vehicle device for communicating control information to a user, and for interacting with the user, such as a vehicle infotainment device, and / or a dashboard device, and / or a control panel.

[0081] Fig. 2 is a diagram illustrating an example vehicle 1 according to this disclosure. The vehicle 1 shown in Fig. 2 is for example the same vehicle 1 as shown in Fig. 1 .

[0082] Vehicle 1 comprises a driver 10 and passenger 12. Both the driver 10 and the passenger 12 can be seen as users of the vehicle. The user device 30 is associated with user / passenger 12. The sensor data obtained indicates that the driver 10 has a driver gaze 11. The driver gaze 11 is indicative of where the driver 10 is looking. The sensor data obtained indicates that the passenger 12 has a passenger gaze 13. The passenger gaze 13 is indicative of where the passenger 12 is looking. In some examples, the vehicle 1 further comprises a projector 202, configured to project a user interface object (such as an augmented reality object) on a surface of the vehicle 1 and / or a retina of the driver 10 and / or the passenger 12. For example, the surface for projection for the driver may be the windscreen while for the passenger 12 the surface may be the side window.

[0083] For example, the projector projects the user interface object based on the gaze direction of the user(s).

[0084] In some examples, the electronic device can prioritise the projecting of the user interface object based on the location of the user in the vehicle. For example, this may allow prioritizing a driver over a passenger and / or certain passengers over other passengers based on their use data indicating an anxious facial expression.

[0085] In some examples, the electronic device can apply one or more weights based on the user data. For example, when the sensor data indicates that three passengers looking at a direction different to the driver, and the user data indicates that the passengers are scared while the driver is not, the electronic device applies more weight or higher prioritization towards displaying the user interface object for reassurance in the gaze direction of the three passengers. In some examples, when the sensor data indicates that there are children and adults in the vehicle, the electronic device may apply lower weights to displaying the user interface object towards the gaze direction of the children than towards the gaze direction of an adult.

[0086] Fig. 3 shows a flow diagram of an example method 100, performed by an electronic device according to the disclosure, for communicating control information. The electronic device is the electronic device disclosed herein, such as electronic device 300 of Fig. 1 , and Fig. 2.

[0087] The method 100 comprises obtaining S102 sensor data from one or more sensors. The method 100 comprises determining S104 one or more physical inputs. The method 100 comprises determining S106 user data indicative of a user behaviour, optionally by applying a machine learning model to the sensor data and the one or more physical inputs. The method 100 comprises communicating S108 control information based on the user data.

[0088] Examples of methods and products (electronic device) according to the disclosure are set out in the following items: Item 1 . An electronic device comprising memory circuitry, processor circuitry, and an interface, wherein the electronic device is configured to: obtain sensor data from one or more sensors; determine one or more physical inputs; determine user data indicative of a user behaviour by applying a machine learning model to the sensor data and the one or more physical inputs; and communicate control information based on the user data.

[0089] Item 2. The electronic device according to item 1 , wherein the sensor data comprises one or more of: optical sensor data, head position data, gaze sensor data, movement data, audio data, and heart rate data.

[0090] Item 3. The electronic device according to any of the previous items, wherein the one or more sensors comprise one or more sensors associated with the vehicle and / or one or more sensors associated with a user device.

[0091] Item 4. The electronic device according to any of the previous items, wherein the one or more sensors comprise one or more of: an optical sensor, a heart rate sensor, and a microphone.

[0092] Item 5. The electronic device according to any of the previous items, wherein the one or more physical inputs comprise one or more of: a steering wheel input, a pedal input, an accelerator input, a brake input, and / or a manual override.

[0093] Item 6. The electronic device according to any of the previous items, wherein the user data comprises a facial expression, a comfort level, and / or a user profile.

[0094] Item 7. The electronic device according to any of the previous items, wherein the machine learning model is configured to determine the user data based on the sensor data and the one or more physical inputs . Item 8. The electronic device according to any of the previous items, wherein the machine learning model comprises one or more of: a convolutional neural network, a recurrent neural network, and a multimodal learning model with transformers.

[0095] Item 9. The electronic device according to any of the previous items, wherein the electronic device is configured to identify the user based on the sensor data.

[0096] Item 10. The electronic device according to any of the previous items, wherein the electronic device is configured to communicate by displaying a user interface object indicative of the control information.

[0097] Item 11. The electronic device according to item 10, wherein the displaying of the user interface object comprises projecting the user interface object on a surface of a vehicle and / or a retina of the user.

[0098] Item 12. The electronic device according to any of items 10-11 , wherein the displaying of the user interface object comprises causing the display of the user interface object on a display device.

[0099] Item 13. The electronic device according to any of items 10-12, wherein the user interface object comprises one or more augmented reality objects configured to support navigation and / or to provide information of surroundings of a vehicle.

[0100] Item 14. The electronic device according to any of the previous items, wherein the electronic device is configured to communicate by emitting one or more audio signals indicative of the control information.

[0101] Item 15. The electronic device according to any of the previous items wherein the electronic device is configured to communicate the control information to a part of a vehicle for controlling an operation of the vehicle. Item 16. The electronic device according to any of the previous items, wherein the control information comprises one or more of: control parameter for operating a vehicle, and assistance information to the user.

[0102] Item 17. The electronic device according to any of the previous items, wherein the electronic device is configured to receive feedback from the user via one or more of: sensor data, a user interface input, and an audio signal.

[0103] Item 18. The electronic device according to item 15, wherein the electronic device is configured to adjust operations of the vehicle based on the feedback and / or the user data, and / or one or more navigation parameters, and / or an energy consumption parameter.

[0104] Item 19. The electronic device according to any of the previous items, wherein a vehicle is capable of autonomous operation or semi-autonomous operation.

[0105] Item 20. A method, performed by an electronic device, the method comprising: obtaining sensor data from one or more sensors; determining one or more physical inputs; determining user data indicative of a user behaviour by applying a machine learning model to the sensor data and the one or more physical inputs; and communicating control information based on the user data.

[0106] Item 21. A vehicle comprising memory circuitry, processor circuitry, and an interface, wherein the vehicle comprises an electronic device according to any of items 1-19.

[0107] The use of the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. does not imply any particular order, but are included to identify individual elements. Moreover, the use of the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. does not denote any order or importance, but rather the terms “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. are used to distinguish one element from another. Note that the words “first”, “second”, “third” and “fourth”, “primary”, “secondary”, “tertiary” etc. are used here and elsewhere for labelling purposes only and are not intended to denote any specific spatial or temporal ordering. Furthermore, the labelling of a first element does not imply the presence of a second element and vice versa.

[0108] It may be appreciated that Figures comprise some circuitries or operations which are illustrated with a solid line and some circuitries, components, features, or operations which are illustrated with a dashed line. Circuitries or operations which are comprised in a solid line are circuitries, components, features or operations which are comprised in the broadest example. Circuitries, components, features, or operations which are comprised in a dashed line are examples which may be comprised in, or a part of, or are further circuitries, components, features, or operations which may be taken in addition to circuitries, components, features, or operations of the solid line examples. It should be appreciated that these operations need not be performed in order presented. Furthermore, it should be appreciated that not all of the operations need to be performed. The example operations may be performed in any order and in any combination. It should be appreciated that these operations need not be performed in order presented.

[0109] Circuitries, components, features, or operations which are comprised in a dashed line may be considered optional.

[0110] Other operations that are not described herein can be incorporated in the example operations. For example, one or more additional operations can be performed before, after, simultaneously, or between any of the described operations.

[0111] Certain features discussed above as separate implementations can also be implemented in combination as a single implementation. Conversely, features described as a single implementation can also be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations, one or more features from a claimed combination can, in some cases, be excised from the combination, and the combination may be claimed as any sub-combination or variation of any sub-combination.

[0112] It is to be noted that the word "comprising" does not necessarily exclude the presence of other elements or steps than those listed.

[0113] It is to be noted that the words "a" or "an" preceding an element do not exclude the presence of a plurality of such elements.

[0114] It should further be noted that any reference signs do not limit the scope of the claims, that the examples may be implemented at least in part by means of both hardware and software, and that several "means", "units" or "devices" may be represented by the same item of hardware. Language of degree used herein, such as the terms “approximately,” “about,” “generally,” and “substantially” as used herein represent a value, amount, or characteristic close to the stated value, amount, or characteristic that still performs a desired function or achieves a desired result. For example, the terms “approximately”, “about”, “generally,” and “substantially” may refer to an amount that is within less than or equal to 10% of, within less than or equal to 5% of, within less than or equal to 1% of, within less than or equal to 0.1 % of, and within less than or equal to 0.01% of the stated amount. If the stated amount is 0 (such as none, having no), the above recited ranges can be specific ranges, and not within a particular % of the value.

[0115] The various example methods, devices, nodes and systems described herein are described in the general context of method steps or processes, which may be implemented in one aspect by a computer program product, embodied in a computer-readable medium, including computerexecutable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Generally, program circuitries may include routines, programs, objects, components, data structures, etc. that perform specified tasks or implement specific abstract data types. Computer-executable instructions, associated data structures, and program circuitries represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.

[0116] Although features have been shown and described, it will be understood that they are not intended to limit the claimed disclosure, and it will be made obvious to those skilled in the art that various changes and modifications may be made without departing from the scope of the claimed disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than restrictive sense. The claimed disclosure is intended to cover all alternatives, modifications, and equivalents.

Claims

CLAIMS1 . An electronic device comprising memory circuitry, processor circuitry, and an interface, wherein the electronic device is configured to: obtain sensor data from one or more sensors; determine one or more physical inputs; determine user data indicative of a user behaviour by applying a machine learning model to the sensor data and the one or more physical inputs; and communicate control information based on the user data.

2. The electronic device according to claim 1 , wherein the sensor data comprises one or more of: optical sensor data, head position data, gaze sensor data, movement data, audio data, and heart rate data.

3. The electronic device according to any of the previous claims, wherein the one or more sensors comprise one or more sensors associated with a vehicle and / or one or more sensors associated with a user device.

4. The electronic device according to any of the previous claims, wherein the one or more sensors comprise one or more of: an optical sensor, a heart rate sensor, and a microphone.

5. The electronic device according to any of the previous claims, wherein the one or more physical inputs comprise one or more of: a steering wheel input, a pedal input, an accelerator input, a brake input, and / or a manual override.

6. The electronic device according to any of the previous claims, wherein the user data comprises a facial expression, a comfort level, and / or a user profile.

7. The electronic device according to any of the previous claims, wherein the machine learning model is configured to determine the user data based on the sensor data and the one or more physical inputs.

8. The electronic device according to any of the previous claims, wherein the machine learning model comprises one or more of: a convolutional neural network, a recurrent neural network, and a multimodal learning model with transformers.

9. The electronic device according to any of the previous claims, wherein the electronic device is configured to identify the user based on the sensor data.

10. The electronic device according to any of the previous claims, wherein the electronic device is configured to communicate by displaying a user interface object indicative of the control information.

11. The electronic device according to claim 10, wherein the displaying of the user interface object comprises projecting the user interface object on a surface of a vehicle and / or a retina of the user.

12. The electronic device according to any of claims 10-11 , wherein the displaying of the user interface object comprises causing the display of the user interface object on a display device.

13. The electronic device according to any of claims 10-12, wherein the user interface object comprises one or more augmented reality objects configured to support navigation and / or to provide information of surroundings of the vehicle.

14. The electronic device according to any of the previous claims, wherein the electronic device is configured to communicate by emitting one or more audio signals indicative of the control information.

15. The electronic device according to any of the previous claims, wherein the electronic device is configured to communicate the control information to a part of a vehicle for controlling an operation of the vehicle.

16. The electronic device according to any of the previous claims, wherein the control information comprises one or more of: control parameter for operating a vehicle, and assistance information to the user.

17. The electronic device according to any of the previous claims, wherein the electronic device is configured to receive feedback from the user via one or more of: sensor data, a user interface input, and an audio signal.

18. The electronic device according to any of claims 15-17, wherein the electronic device is configured to adjust operations of the vehicle based on the feedback and / or the user data, and / or one or more navigation parameters, and / or an energy consumption parameter.

19. The electronic device according to any of the previous claims, wherein a vehicle is capable of autonomous operation or semi-autonomous operation.

20. A method, performed by an electronic device, the method comprising: obtaining sensor data from one or more sensors; determining one or more physical inputs; determining user data indicative of a user behaviour by applying a machine learning model to the sensor data and the one or more physical inputs; and communicating control information based on the user data.

21. A vehicle comprising memory circuitry, processor circuitry, and an interface, wherein the vehicle comprises an electronic device according to any of claims 1-19.

Images