System and method for monitoring operational cognitive condition of a driver

Abstract

A method and system are presented for assessing an operational cognitive condition of a human operator while operating a vehicle having a semi-autonomous operation system. The method comprises: obtaining first motion data being human operator related motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in a cognitive condition of the human operator; and analyzing said first motion data by applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous operational motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

Description

[0001] SYSTEM AND METHOD FOR MONITORING OPERATIONAL COGNITIVE

[0002] CONDITION OF A DRIVER

[0003] TECHNOLOGICAL FIELD

[0004] The present disclosure is generally in the field of safety driving of vehicles and relates to a system and method for monitoring operational cognitive condition of a vehicle’s driver, in particular useful for drivers of autonomous or semi- autonomous vehicles.

[0005] BACKGROUND

[0006] Vehicles providing autonomous or semi- autonomous driving functionalities are known and their increased use on the roads is expected in the nearest future. In such vehicles, an autonomous driving system (autonomous driving functionalities) is installed, Such autonomous driving system typically utilizes various in-cabin sensors monitoring operation of various vehicle control components and providing data about steering the vehicle, accelerating, braking, etc. as well as sensors monitoring vehicle’s surroundings.

[0007] According to Society of Automotive Engineering (SAE) international taxonomy, vehicle automation can be classified into six different levels. The levels of increased autonomy are ranging from level zero (LO), where the human driver is completely responsible for the execution of all the tasks related to driving and monitoring of the external environment at all times, to level five (L5), where human driver is just a passenger, and the on-board operator system can perform all the tasks related to driving function, under all roadway and environmental circumstances.

[0008] Some vehicles having autonomous driving functionalities require that upon request of the autonomous driving system, the human driver rapidly takes over the control of the vehicle. This may be the case for vehicles offering autonomous driving functionalities according to Level 3 as defined by the Society of Automotive Engineers (SAE), for example, when the vehicle approaches a construction area, and the autonomous driving system detects that it will not be able to further guide the vehicle, e.g., due to the absence of road markings. Consequently, the human driver is requested to take over control of the vehicle.

[0009] GENERAL DESCRIPTION

[0010] Considering driving of vehicles with autonomous driving functionalities, the safety requirement of all traffic participants, transitioning of the control from the autonomous driving system to the human driver, as well as control of the autonomous driving system by the human driver, needs to ensure that the human driver is fully competent to implement such tasks. In other words, an impaired operational cognitive state of the driver, caused by an impaired / abnormal cognitive condition of the driver, should be avoided.

[0011] There is therefore a need in the art for a novel approach to assess the operational cognitive condition of a driver during driving a vehicle having a semi-autonomous driving system such that a real time detection of an impaired cognitive condition of the driver can be timely identified under various driving conditions. Furthermore, such an approach should enable generation of a corresponding alert to at least one of the driver and an authorized entity about the impaired state of the operational cognitive condition of the driver.

[0012] Generally, the above approach is also suitable and can advantageously be used not only to regulate the vehicle's travel but also operation of vehicles that alternatively or additionally have other operational functions, such as raising and / or lowering the forklift, in a shovel, raising and lowering the bucket, etc., as well as driving / operational functions implemented by joystick or computer mouse, as well as level / speed of eyes' movement of a human operator (e.g., a camera captures the eye movement and the vehicle moves according to the direction and intensity of the eye movement).

[0013] The present disclosure provides creation and use of a model-based (machine learning-based) system configured to analyze human operating (e.g., driving) behaviors with those of an ideal operator (driver) represented by an autonomous operational (driving) system in order to identify whether the human operating behaviour corresponds to a normal operational cognitive condition of the human operator or to an impaired operational cognitive condition. In the latter case, warning / alert can be generated to prevent the human operator from further operating a vehicle.

[0014] The model is a difference model which is effectively trained for detecting cognitive function decline in human operators, corresponding to an impaired operational cognitive condition of the human operator. The technique of the present disclosure thus aims to provide an improved method for real-time detection of impaired operational cognitive conditions under various operational conditions.

[0015] In the description below, the terms “autonomous” and “semi -autonomous” are used interchangeably. It should be understood that, generally, both terms relate to “autonomous driving functionalities” or autonomous driving system of a vehicle. However, since the present disclosure relates to human driver involvement, at least per demand, in controlling of the vehicle, the term “semi -autonomous” is used at times.

[0016] Thus, according to one broad aspect of the present disclosure, there is provided a method for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operation system, the method comprising: obtaining first motion data being human operator related motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in a cognitive condition of the human operator; analyzing said first motion data by applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous operational motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

[0017] In some embodiments, the method further comprises: obtaining second motion data in relation to said at least one vehicle control component from the semi-autonomous operational system, said second motion data being indicative of real time condition of said semi- autonomous operational system during said operating of the vehicle; and analyzing said first and second motion data by applying model-based processing to said first and second motion data utilizing said predetermined trained model based on reference autonomous operational motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating of a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

[0018] The method may further comprise: in response to the warning data, generating a corresponding alert to at least one of the human operator and an authorized entity about the impaired state of the operational cognitive condition of the human operator; and / or in response to the warning data, activating an operational mode of the semi- autonomous vehicle to compensate for impairment in the operational cognitive condition of the human operator.

[0019] In some embodiments, the obtaining of the first motion data comprises communication with a sensing system installed in the vehicle to receive, from said sensing system, sensing signals indicative of motion dynamics of at least one vehicle control component during the operating of the vehicle. In the embodiments utilizing the use of second motion data, the obtaining of the second motion data comprises communication with a controller of the semi-autonomous operational system to receive sensing signals indicative of motion dynamics of at least one vehicle control component during the operating of the vehicle as being read by said controller of the semi-autonomous operational system.

[0020] The motion dynamics of at least one vehicle control component may comprise at least one of the following: steering wheel angular motion, motion associated with a vehicle's control joystick, motion associated with a vehicle's control computer mouse, motion associated with eyes' movement of the human driver, motion associated with pressure applied by the human operator on a gas pedal, motion associated with pressure applied by the human operator on a breaking pedal, motion associated with raising and / or lowering of forklift in a shovel.

[0021] In some embodiments, the vehicle operational session is performed continuously by the human operator, in which case the steps of said obtaining the first motion data and analyzing the first motion data are performed substantially continuously during an operational session of the vehicle.

[0022] In some embodiments, the steps of obtaining the first motion data and analyzing the first motion data are performed periodically during an operational session of the vehicle.

[0023] In some embodiments, the steps of obtaining the first motion data and analyzing the first motion data are performed in response to a control signal from a controller of the semi-autonomous operational system.

[0024] In some embodiments, the steps of obtaining the first motion data and analyzing the first motion data are performed in response to a triggering signal being indicative of a critical level of a cognitive condition of the human operator affected by one or more factors, being independently determined by a model-based processing of the sensing signals.

[0025] The semi- autonomous operational system may operate in a background control mode while a vehicle's operational session is performed continuously by the human operator. In this case the steps of obtaining the first and second motion data and analyzing the first and second motion data are performed substantially continuously during the vehicle's operational session.

[0026] The steps of obtaining the first and second motion data and analyzing of the first and second motion data may be performed periodically during a vehicle's operational session; and / or may be in response to a control signal from a controller of the semi- autonomous operational system; and / or may be in response to a triggering signal being indicative of a critical level of a cognitive condition of the human operator affected by one or more factors, being independently monitored by a model-based processing of sensing signals.

[0027] The independent monitoring of the cognitive condition of the human operator may utilize analysis of independently measured motion data to identify changes in the independently measured motion data caused by one or more characteristic factors defining an effect on the level of the cognitive condition. The changes in the independently measured motion data may be caused by at least first and second independent characteristic factors defining together a combined effect on the cognitive condition of the human operator. Accordingly, the independent monitoring may comprise determining whether said independently measured motion data comprises one or more features corresponding to a predetermined thresholding condition of said combined effect, irrespective of whether or not a separate effect of one of said first and second characteristics factors has reached a corresponding thresholding level.

[0028] Alternatively or additionally, the independent monitoring of the cognitive condition of the human operator to identify the critical level thereof may comprise analysis of the first motion data to identify changes in the first motion data caused by one or more characteristic factors defining an effect on the level of the cognitive condition. The changes in the first motion data may be caused by at least first and second independent characteristic factors defining together a combined effect on the cognitive condition of the human operator. Accordingly, the independent monitoring comprises determining whether said first motion data comprises one or more features corresponding to a predetermined thresholding condition of said combined effect, irrespective of whether or not a separate effect of one of said first and second characteristics factors has reached a corresponding thresholding level.

[0029] The characteristic factor may be associated with fatigue and / or drunkenness of the human operator.

[0030] In some other embodiments, being alternative or additional to the above embodiments, the independent monitoring of the cognitive condition of the human operator may utilize analysis of independently measured image data indicative of the level of the cognitive condition of the human operator; and / or analysis of independently measured one or more physiological parameters of the human operator indicative of the level of the cognitive condition of the human operator.

[0031] In the embodiments utilizing collection and analysis of the second motion data, the method may further comprise: prior to applying said model-based processing of the first and second motion data, momentarily deactivating the at least one vehicle control component associated with said at least one body part of the human operator from affecting operation of the vehicle, such that control of the operation of the vehicle is passed to the semi-autonomous operational system while keeping the first motion data indicative of motion dynamics originated on said at least one body part of the human operator and related to said at least one vehicle control component.

[0032] According to another broad aspect of the present disclosure, it provides a method for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operational system, the method comprising: obtaining first motion data being human operator related motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in a cognitive condition of the human operator; obtaining second motion data in relation to said at least one vehicle control component from the semi- autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating of the vehicle; analyzing said first and second motion data by applying model-based processing to said first and second motion data utilizing a predetermined trained model based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

[0033] The present invention, in its yet further broad aspect, provides a system for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operational system, the system comprising a control system configured as a computerized system comprising input and output utilities, a memory utility and a data processor and analyzer utility, and being configured and operable to communicate with a sensing system to receive first motion data indicative of motion dynamics, originated on at least one body part of the human operator and related to at least one vehicle control component, wherein: the data processor and analyzer is configured and operable for analyzing said first motion data to identify changes in an operational cognitive condition of the human operator, said analyzing comprising applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of an impaired state of the operational cognitive condition of the human operator.

[0034] The system may further comprise the sensing system comprising one or more motion sensors configured and operable to provide said first motion data.

[0035] The control system may be further configured and operable to communicate with a controller of the semi-autonomous operational system to receive sensing signals indicative of motion dynamics of said at least one vehicle control component during the operating of the vehicle as being read by said controller of the semi-autonomous operational system; and said data processor and analyzer is further configured and operable to carry out the following: obtain second motion data in relation to said at least one vehicle control component from a controller of the semi-autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating the vehicle; and analyze said first and second motion data by applying model-based processing to said first and second motion data utilizing said predetermined model trained based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

[0036] The system may further comprise a notification utility configured and operable to be responsive to said warning data to generate a corresponding alert to at least one of the human operator and an authorized entity about the impaired state of the operational cognitive condition of the human operator. The system may further comprise an activator configured and operable to be responsive to said warning data or to said alert to activate an operational mode of the semi-autonomous vehicle to compensate for impairment in the operational cognitive condition of the human operator.

[0037] The control system may be of one of the following configurations: (i) substantially continuously during an operating session, obtain and analyze the first motion data; (ii) obtain and analyze the first motion data periodically during an operating session; and (iii) obtain and analyze the first motion data in response to a control signal from a controller of the semi- autonomous operational system.

[0038] The system may further comprise: a characteristic factor model-based processor configured and operable to analyze the sensing signals to independently monitor a cognitive condition of the human operator; and a triggering utility configured and operable upon identifying a critical level of the cognitive condition of the human operator affected by one or more factors, generating a triggering signal; the control system being configured and operable to be responsive to the triggering signal to perform said obtaining and analyzing of the first motion data. The control system may be of one of the following configurations: (i) substantially continuously during an operating session, obtain and analyze the first and second motion data; (ii) obtain and analyze the first and second motion data periodically during an operating session; and (iii) obtain and analyze the first and second motion data in response to a control signal from a controller of the semi- autonomous operational system.

[0039] The system may further comprise a controller configured and operable to momentarily deactivate the at least one vehicle control component associated with said at least one body part of the human operator from affecting operation of the vehicle, such that control of the operation of the vehicle is passed to the semi-autonomous operational system while keeping the first motion data indicative of motion dynamics originated on said at least one body part of the human operator and related to said at least one vehicle control component; and allow the data processor and analyzer to perform said modelbased processing of the first and second motion data.

[0040] The system may further comprise: a characteristic factor model-based processor configured and operable to analyze the sensing signals to independently monitor a cognitive condition of the human operator to identify a critical level of the cognitive condition of the human operator affected by one or more factors; and a triggering utility configured and operable to generate triggering signal, upon identifying the critical level of the cognitive condition of the human operator; the control system being configured and operable to be responsive to the triggering signal to perform said obtaining and analyzing of the first and second motion data. The independent monitoring of the cognitive condition of the human operator may utilize analysis of independently measured motion data to identify changes in said independently measured motion data caused by one or more characteristic factors defining an effect on the level of the cognitive condition. The changes in the independently measured motion data may be caused by at least first and second independent characteristic factors defining together a combined effect on the cognitive condition of the human operator; said independent monitoring comprising determining whether said independently measured motion data comprises one or more features corresponding to a predetermined thresholding condition of said combined effect, irrespective of whether or not a separate effect of one of said first and second characteristics factors has reached a corresponding thresholding level.

[0041] The independent monitoring of the cognitive condition of the human operator may utilize analysis of independently measured image data indicative of the level of the cognitive condition of the human operator, and / or analysis of independently measured one or more physiological parameters of the driver indicative of the level of the cognitive condition of the human operator.

[0042] According to yet another broad aspect of the present disclosure, it provides a system for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operational system, the system comprising: a sensing system comprising one or more motion sensors configured and operable to provide first motion data indicative of motion dynamics, originated on at least one body part of the human operator r and related to at least one vehicle control component, said first motion data being indicative of changes in an operational cognitive condition of the human operator, a control system configured and operable to communicate with the sensing system to receive said first motion data, the control system comprising a data processor and analyzer configured and operable to: analyze said first motion data by applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous driving motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of an impaired state of the operational cognitive condition of the human operator.

[0043] According to yet another broad aspect of the present disclosure, it provides a system for assessing an operational cognitive condition of a human operator r during operating a vehicle having a semi-autonomous operational system, the system comprising: a sensing system comprising one or more motion sensors configured and operable to provide first motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in an operational cognitive condition of the human operator, a control system comprising a data processor and analyzer configured and operable to: obtain said first motion data and obtain second motion data in relation to said at least one vehicle control component from said semi- autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating the vehicle; and analyze said first and second motion data by applying model-based processing to said first and second motion data utilizing a predetermined trained model based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

[0044] The present disclosure further provides a method for creating a model for use in monitoring an operational cognitive condition of a driver during driving a vehicle having a semi- autonomous driving system, the method comprising: providing a first train set of motion patterns indicative of motions originated in at least one body part of a driver and measured on at least one vehicle control component during a plurality of driving sessions by drivers having different levels of impaired cognitive conditions affected by different values of one or more characteristic factor, said first train set of motion patterns corresponding to driver-related motion data in relation to said at least one vehicle control component; providing a second train set of motion patterns measured on said at least one vehicle control component by a controller of an autonomous driving system during a plurality of autonomous driving sessions under different external conditions, said second train set presenting ideal motion data in relation to said at least one vehicle control component and various external conditions and corresponding to driving of a semi- autonomous vehicle by a driver at a normal cognitive condition thereof at each external condition; using said first and second train sets to perform machine learning processing and create one or more difference models, wherein each difference model indicates a thresholding condition for a difference between said driver-related motion data and said ideal motion data in relation to said at least one vehicle control component, said thresholding condition defining a critical impaired state of the operational cognitive condition of the driver.

[0045] More specifically, the present disclosure is used for controlling a human driver while driving a vehicle having semi-autonomous or autonomous driving system and is therefore exemplified below with respect to this specific application. However, the respective terms should be interpreted broadly to cover also the above-mentioned operational options.

[0046] In the description below, the term “human driver” is referred to as “driver”, and autonomous or semi-autonomous driving system is at times referred to as “ideal driver Also, in the description below, the term “impaired operational cognitive condition ” or “impaired operational cognitive state ” of a driver refers to an abnormal state / condition of individual's brain as compared to a normal cognitive condition. Typically, an abnormality is defined with respect to the individual’s current activity, which is driving activity for the purposes of the present disclosure. Considering driving activity / behaviour, the abnormality can be caused by various factors, such as drug use, acute physical condition, exhaustion, intoxication, fatigue, drunkenness, stress, attention withdrawal, motion sickness.

[0047] BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to better understand the subject matter that is disclosed herein and to exemplify how it may be carried out in practice, embodiments will now be described, by way of non-limiting examples only, with reference to the accompanying drawings, in which:

[0049] Fig. 1 exemplifies, by way of a block diagram, the configuration and operation of the system of the present disclosure for assessing an operational cognitive condition of a driver during driving a vehicle having a semi-autonomous driving system;

[0050] Figs. 2A and 2B show two examples of a method for assessing an operational cognitive condition of a driver during driving a vehicle having a semi-autonomous driving system, according to the present disclosure;

[0051] Fig. 3 exemplifies various operational modes of the system of the present disclosure during a driving session; and

[0052] Fig. 4 exemplifies the technique of the present disclosure for creation of machine learning model to be used to implement in assessment of an operational cognitive condition of a driver during driving a vehicle having a semi-autonomous driving system.

[0053] Fig. 5 exemplifies three traces of steering wheel movement under different conditions: an autonomous driver, a sober human driver, and a drunk human driver.

[0054] DETAILED DESCRIPTION OF EMBODIMENTS

[0055] Reference is made to Fig. 1 showing by way of a block diagram the configuration and operation of a system 100 according to the present disclosure for assessing an operational cognitive condition of a human operator during operating of a vehicle having a semi- autonomous operational system 122.

[0056] It should be noted that, generally, the principles of the present disclosure can be used for controlling the operational cognitive condition of a driver (constituting a human operator) operating a driving session of a vehicle, having semi-autonomous or autonomous operational system, while moving along a road, as well as can be used for controlling the operational cognitive condition of a human operator while operating such vehicle for raising and / or lowering a forklift, in a shovel, raising and lowering the bucket, etc.

[0057] More specifically, the present disclosure is used for controlling the driver while driving a vehicle having semi- autonomous or autonomous driving system and is therefore exemplified below with respect to this specific application. However, the respective terms should be interpreted broadly to cover also the above-mentioned operational options.

[0058] The system 100 is installed in a vehicle and includes a control system 102 which is a computerized system including, inter alia, an input utility 106, an output utility 114, a memory 108, and a processor & analyzer 110.

[0059] In some embodiments, the system 100 also includes a notification utility 116, and / or a triggering utility 120, and / or an autonomous drive mode activator 128. In some other embodiments, the system 100 further includes a controller 130 configured and operable to control the operation of one or more vehicle control components VCCi (i = l,...,m), belonging to vehicle’s control system 140.

[0060] The control system 102 is configured and operable to communicate with a sensing system 104 installed in the vehicle to receive various types of motion data from the sensing system 104.

[0061] The sensing system 104 includes a number N (N>1) of sensors including one or more motion sensors - Sensor 1, Sensor 2,. . . Sensor N being exemplified in the figure, each configured and operable to provide first motion data MD1 presenting driver-related data (constituting human operator related data) sensed / read on at least one vehicle control component VCCI (e.g., steering wheel, braking and acceleration pedals , etc.) whose operation is associated with / operated by interaction with at least one body part of the driver. Such first motion data is thus indicative of motion dynamics originated on the at least one body part of the driver and related to the at least one vehicle control component, and presents driver’s behavior during a driving session (constituting a vehicle's operation session) indicative of changes in an operational cognitive condition of the driver.

[0062] Typically, each motion sensor of the sensing system 104 has its unique identification data (ID) which can be identified by the control system 102 (e.g., the sensing signals include also the sensor's ID). In case the same sensor is intended to sense motions originated by different sources, as the case may be, the sensor is assigned with different IDs, respectively, thus allowing the control system to properly identify the sensing signals being received.

[0063] For example, the vehicle control component may be the steering wheel of a vehicle. In this example, the sensing system 104 may include the sensor (e.g., Sensor 1) providing first driver motion data (MDl)i indicative of the steering wheel angular motion, while the driver takes control of the driving. In some embodiments, this Sensor 1 is utilized only for monitoring / sensing the steering wheel angular motion during manual driving (i.e. driving session performed by the driver), and accordingly the control system 102 identifies (MDl)i as originating by the motion of driver's hand(s), i.e. driver-related motion data.

[0064] Thus, during a driving session, for the purposes of real time assessment of an operational cognitive condition of a driver, the control system 102 communicates with the sensing system 104 to receive and analyze the first motion data MD1 being driver- related data. It should be understood that the sensing data can be sensed / read in relation to two or more different vehicle control components, and thus the first motion data MD1 can generally be presented by n sensing data pieces obtained from n sensors, i.e., (MD1 )N.

[0065] The data processor & analyzer 110 is configured and operable to obtain and process the first motion data MD1 provided by the sensing system 104 in relation to the vehicle control component(s) The data processor & analyzer 110 includes a model-based processor 112 configured and operable as a so-called “ideal model based processor”. To this end, the processor 112 analyzes the first motion data MD1 by applying thereto a predetermined trained model based on reference autonomous driving motion data representing ideal motion data in relation to the respective at least one vehicle control component. Such ideal motion data corresponds to driving a vehicle by a driver at a normal cognitive condition thereof. Using such ideal model based analysis of the first motion data allows for automatic identifying a thresholding condition for the first motion data indicative of a critical impaired state of the operational cognitive condition of the driver.

[0066] The ideal model is previously created and stored e.g., in the memory 108 of the control system 102 or in an external storage device 107 to which the control system 102 has access via a communication network, in which case the control system 102 is properly equipped by a communication utility of any known suitable type.

[0067] Thus, the data processor & analyzer 110 analyzes the first motion data by applying thereto the above-described model-based processing. Alternatively, or additionally, the data processor & analyzer 110 also analyzes motion dynamics of the respective at least one vehicle control component read by the controller of the semi-autonomous driving system (constituting a semi-autonomous or autonomous operational system) during the real time driving session (constituting a vehicle's operational session).

[0068] More specifically, during a driving session, for the purposes of real time assessment of an operational cognitive condition of a driver, the control system 102 may be additionally configured and operable to communicate with the semi- autonomous driving system 122 installed in the vehicle to receive therefrom second motion data MD2 indicative of sensing signals presenting real time condition of motion dynamics of the at least one vehicle control component during the driving.

[0069] Such semi- autonomous driving system 122 can be of any known suitable configuration including suitable electronics and software / hardware utilities implementing autonomous driving functionalities. The configuration and operation of the semi- autonomous driving system 122 do not form part of the present disclosure and therefore need not be specifically described, except to note the following: The semi-autonomous driving system 122 typically includes a sensing system 124 including various sensors for monitoring and providing sensing data / signals indicative of operation of various vehicle control components, such as steering wheel, braking and acceleration pedals, etc.; and includes a local controller 126 which analyzes the sensing signals and properly controls / adjusts the respective autonomous driving functionalities. The semi-autonomous driving system 122 may be configured to communicate with the sensing system 104, and / or the semi-autonomous driving system 122 utilizes its sensing system 124 and the control system 102 is thus configured and operable to communicate with the local controller 126 of the semi- autonomous driving system 122 to receive the sensing signals provided by the sensing system 124 of the semi- autonomous driving system 122.

[0070] The autonomous driving motion data provided by the semi-autonomous driving system 122 represents ideal motion data in relation to the at least one vehicle control component and corresponds to driving of a vehicle by a driver at a normal cognitive condition thereof. Thus, in some embodiments, during a driving session, the control system 102 communicates with the local controller 126 of the semi-autonomous driving system 122 to receive data indicative of sensing signals presenting motion dynamics of the at least one vehicle control component during the driving as being read by the local controller 126. The data processor & analyzer 110 analyzes the first motion data MD1 and the second motion data MD2 by applying the model-based processing utilizing the previously created difference model (i.e., a difference between driver-related motion data and ideal motion data), to thereby identify a thresholding condition for such difference being indicative of a critical impaired state of the operational cognitive condition of the driver.

[0071] Thus, the control system 102 is configured and operable to identify a thresholding condition for the first motion data MD1, corresponding to the critical impaired state of the operational cognitive condition of the driver , where such thresholding condition is identified based on the model-based difference between the driver-related data (the first motion data MD1) and the ideal driving data being pre-stored data or data provided in real time MD2. In both cases, warning data WD is properly generated by the processor and analyzer 110 upon identifying the thresholding condition for the first measure data MD1.

[0072] Such warning data can be output via output utility 114 in any suitable format, e.g., audio, visual to attract attention of the driver or to be transmitted to an authorized entity. The notification utility 116 may be provided and configured and operable to be responsive to the warning data WD sent by the output utility 114, to generate a corresponding alert to at least one of the driver and an authorized entity about the impaired state of the operational cognitive condition of the driver. As indicated above, in some embodiments, the control system 102 also includes the autonomous drive mode activator 128. The actuator 128 may be configured and operable to be responsive to the warning data WD or to the alert described above, to generate a corresponding activate signal AS to the to the semi-autonomous driving system 122 and by this activate an operational mode of the semi-autonomous functionalities of vehicle (and possibly deactivate vehicle's responsiveness to the driver's driving activity) to compensate for impairment in the operational cognitive condition of the driver.

[0073] As already mentioned above, the present disclosure allows to evaluate the driver’s competence continuously while driving or from time to time (when the driver is not required to hold the steering wheel while driving) without distracting the driver from the road.

[0074] As exemplified in Fig. 3 and will be described more specifically further below, in either one of the above described embodiments, the control system 102 may operate in one of the following configurations to monitor the driver's operational cognitive condition:

[0075] (i) during a substantially continuous driving session performed by the driver (while the semi- autonomous driving system operates in the background), either obtain and analyze the first motion data MD1 for a case (a) where the human driver's driving in real time is analyzed using the machine learning model of the ideal driving (without considering the driving of the semi-autonomous vehicle in real time), or obtain and analyze the first and second motion data, MD1 and MD2 for a case (b) where the human driver's driving is analyzed using the machine learning difference model in relation to the real time driving of the vehicle by the semi-autonomous driving system installed in said vehicle;

[0076] (ii) periodically, during a driving session, obtain and analyze the first motion data MD1 for the case (a), or obtain and analyze the first and second motion data, MD1 and MD2 for the case (b); and

[0077] (iii) in response to a control signal from the local controller 126 of the semi- autonomous driving system 122, obtain and analyze the first motion data MD1 for the case (a) or obtain and analyze the first and second motion data, MD1 and MD2 for the case (b). It should also be noted that in some embodiments the control system 102 may operate in a configuration where the controller 130 deactivates momentarily one or more of the vehicle’s control components VCCs (controlled by the vehicle’s control system 140) from affecting / being involved in the vehicle's operation, to thereby transfer the full control of the vehicle's operation to the semi-autonomous driving system 122. This is implemented for the purpose of checking the driver's competence, and can be initiated periodically or upon identifying some impairment in the operational cognitive condition of the driver, either using the above-described model-based processing or independent evaluation of the cognitive condition of the driver as will be described further below. To this end, the controller 130 may for example momentarily disconnect the movement of the steering wheel from the wheels only for the purpose of checking the driver's competence. Thus, the semi-autonomous driving system 122 will continue to drive the vehicle while the driver continues to move the steering wheel (though, without influencing the wheels). Such configuration may be safer in case the human driver appears to be unable to drive at all.

[0078] For example, the driver is so drunk that he cannot drive at all. However, for the purposes of a test, if in order to detect the level of alcohol in the driver's blood, the driver is given the control of driving the vehicle even for a short period of time, this endangers the driver. In order to solve this problem, the driver moves the steering wheel as if he / she were driving, but the control of the driving remains in the "hands" of the autonomous system.

[0079] In this configuration, the first and second motion data, MD1 and MD2, are obtained and analyzed, wherein during the momentary deactivation of the vehicle’s control components VCCs from affecting / being involved in the vehicle's operation, the semi-autonomous driving system 122 performs the actual driving and provides real time second data MD2 while the sensing system 104 concurrently provides the first motion data MD1 related to driver’s motion data (although not producing actual control of / effect on the vehicle's operation) and being indicative of driver’s behavior during a driving session and indicative of changes in an operational cognitive condition of the driver.

[0080] In the above non-limiting example of a steering wheel, during such momentary disconnection between the movement of the steering wheel by the driver and the wheels, the model-based analysis is performed in relation to the first motion data MD1 (movement of the steering wheel operated by the driver) and the second motion data MD2 (movement of the wheels) originated from the semi-autonomous driving system.

[0081] As described above, the sensing system 104 may include one or more motion sensors providing the first motion data MD1, and the autonomous driving system 122 may also include the sensing system 124 and / or may utilize sensing signals / data provided by one or more sensors of the sensing system 104 to provide the second motion data MD2.

[0082] For example, the same sensor Sensor 1 may be utilized by both the control system 102 and the semi- autonomous vehicle driving system 122 in association with the steering wheel angular motion during the driving session. As noted above, the sensor provides its ID and thus the control system 102 identifies that the received steering wheel angle data provided by Sensor 1 is indicative of the first motion data (MDl)i corresponding to the steering wheel angular motion. However, even when the driver drives (or, in some embodiments, is required to drive) the vehicle for a short time, the semi-autonomous driving system 122 continues its operation to real time implement "ideal" driving and provides to the control system 102 second motion data (MD2)i in relation to "ideal" operation of the wheels corresponding to "ideal" operation of the steering wheel. This enables the data processor & analyzer 110 to apply the previously created difference model to analyze the real time differences between the first motion data (MDl)i and the second motion data (MD2)i.

[0083] In some embodiments, the sensing system 124 of the semi- autonomous vehicle may include dedicated sensor(s) to measure steering wheel angle only during autonomous control of the vehicle. In these embodiments, the control system 102 identifies the second motion data MD2 as originating from the semi-autonomous driving system, being indicative of real time condition of said semi-autonomous driving system 122 during the driving.

[0084] It is noted that the above discussion regarding the motion dynamics of a vehicle control component being the steering wheel is relevant to other vehicle control components, e.g., motion associated with driver's pressure on a gas pedal and / or motion associated with driver's pressure on a breaking pedal, etc.

[0085] Reference is made to Figs. 2A and 2B showing, by way of block diagrams 300 and 400 two examples, respectively, of a method for assessing an operational cognitive condition of a driver (constituting a human operator) during driving (operating) a vehicle having a semi-autonomous driving system (operational system), according to the present disclosure.

[0086] As shown in Fig. 2A, in step 302 first motion data MD1 is obtained, being driver- related motion data indicative of motion originated on at least one body part of the driver, related to at least one vehicle control component (e.g., (MD1)N per n vehicle control components). The first motion data MD1 is indicative of changes in the cognitive condition of the driver. In step 304, the first motion data MD1 is analyzed by applying model-based processing to this data MD1 utilizing a pre-stored trained model(s) TM trained on reference autonomous driving motion data representing ideal motion data in relation to each of the at least one vehicle control component (e.g. TMN per n vehicle control components) and corresponding to driving of a vehicle by a driver at a normal cognitive condition. In step 306, a relation between the first motion data MD1 and the ideal motion data is analyzed to identify whether said relation corresponds to a thresholding condition for the first motion data MD1. If the thresholding condition is identified for the first motion data MD1, a critical impaired state of the operational cognitive condition of the driver is identified (step 308) and warning data is generated (step 310). If a thresholding condition is not identified, the monitoring of the first motion data MD1 proceeds (step 302) in accordance with the predetermined regime / mode as discussed above and shown in Fig. 3.

[0087] As shown in Fig. 2B, in step 402, first motion data MD1 is obtained, being driver- related motion data indicative of motion originated on at least one body part of the driver, related to at least one vehicle control component (e.g., (MD1)N per n vehicle control components). The first motion data MD1 is indicative of changes in the cognitive condition of the driver. Concurrently or with predetermined time slots, second motion data MD2 is obtained (step 404) in relation to the at least one vehicle control component, read by the semi- autonomous driving system. The second motion data MD2 is indicative of real time condition of the semi-autonomous driving system during the driving. The obtaining of the second motion data MD2 may include communication with the local controller 126 of the semi-autonomous driving system 122 to receive sensing signals indicative of motion dynamics of at least one vehicle control component during the driving as being read by said local controller 126 of the semi-autonomous driving system 122. In step 406, the first and second motion data MD1 and MD2 are analyzed by applying model-based processing to these data MD1 and MD2 utilizing pre-stored trained model(s) TM trained on reference autonomous driving motion data representing ideal motion data in relation to the at least one vehicle control component (e.g. TMN per n vehicle control components) and corresponding to driving of a semi-autonomous vehicle by a driver at a normal cognitive condition. In step 408, a difference (or generally, a predetermined relation) between the first motion data MD1 and the second real-time motion data MD2 is examined to determine whether a thresholding condition for such relation exists. In case a thresholding condition is identified, a critical impaired state of the operational cognitive condition of the driver is identified (step 410) and warning data is generated (step 412). If a thresholding condition was not identified, monitoring of the first and second motion data, MD1 and MD2, is repeated from step 402 in accordance with the predetermined regime / mode as discussed above and shown in Fig. 3.

[0088] Fig. 3 exemplifies (in a self-explanatory manner) various operational modes of the system of the present disclosure during a driving session. As shown, once a driving session is started (step 502), the above described method (300 or 400) may be performed according to one of the following configurations:

[0089] (i) Semi- autonomous driving system operates in a background control mode while a vehicle driving session is performed continuously by the driver (step 504), and the above-described steps of obtaining the first and second motion data MD1 and MD2 (or the first motion data MD1 only) and analyzing the first and second motion data MD1 and MD2 (or the first motion data MD1 only) using the respective model-based processing is performed substantially continuously during the driving session.

[0090] (ii) The steps of obtaining the first and second motion data MD1 and MD2 (or the first motion data MD1 only) and analyzing of the first and second motion data MD1 and MD2 (or the first motion data MD1 only) are performed periodically during the driving session (step 506);

[0091] (iii) The steps of obtaining the first and second motion data MD1 and MD2 (or the first motion data MD1 only) and analyzing of the first and second motion data MD1 and MD2 (or the first motion data MD1 only) are performed in response to a control signal from the local controller 126 of the semi-autonomous driving system 122 (step 508); and (iv) The steps of obtaining the first and second motion data MD1 and MD2 (or the first motion data MD1 only) and analyzing of the first and second motion data MD1 and MD2 (or the first motion data MD1 only) are performed in response to a triggering signal being indicative of a critical level of a cognitive condition of the driver affected by one or more characteristic factors (e.g., drunkenness and fatigue), being independently monitored by a model-based processing of the sensing signals (step 510).

[0092] With regards to the option (iv), reference is made back to Fig. 1 showing that the system 100 may additionally include a triggering utility 120, and the control system 102 additionally includes a characteristic factor model based processor 115. The independent determination of the cognitive condition of the driver utilizes analysis, by the processor 115, of independently measured data using the sensing signals provided by the sensing system 104 which may include motion-relating sensing signals or other sensing signals (e.g., image data about e.g., driver's eyes and / or head behavior and / or position) to identify changes in the cognitive condition of the driver caused by one or more characteristic factors defining an effect on the level of the cognitive condition.

[0093] The characteristic factor model based processor 115 may be configured and operable as described in the patent publication WO2023242842, assigned to the assignee of the present application and incorporated herein by reference with respect to this specific example. According to such technique, the model is created describing a relation between a value of the specific characteristic factor and its effect on a change in the level of the cognitive condition of a driver from the normal cognitive condition, where such relation is reflected in a motion pattern sensed on at least one body part of the driver and / or vehicle's control components interacting with the body part of the driver. More specifically, certain features can be identified in the measured / sensed motion patterns specifically related to the value / level of such characteristic factor as alcohol intoxication which corresponds to a critical level of drunkenness that signals of an impaired cognitive condition of the driver. Different characteristic factors may differently affect the change of the cognitive condition of a driver (e.g., intoxication, fatigue) during the driving activity from the normal cognitive condition. The characteristic factor model based processor 115 thus analyzes the first motion data, and upon identifying a corresponding thresholding condition of the level of the cognitive condition of the driver, activates the triggering utility 120 which generates a triggering signal TS to the control 102. It should also be noted that different characteristic factors may interact (i.e. are concurrently present) forming a combined effect on the cognitive condition of a driver (e.g., affecting alertness while driving a car), and the characteristic factor model based processor 115 may be configured and operable to apply to the first motion data MD1 a predetermined model-based processing utilizing a model describing changes in the motion data corresponding to the driver's cognitive condition while being affected by the combined effect of the two or more different characteristic factors and that of the normal cognitive condition during driving. Upon identifying a corresponding thresholding condition, the triggering utility 120 is activated to generate a triggering signal TS to the control system 102.

[0094] With regards to the combined effect, it should be noted that it might exceed the "combined" warning level (threshold) in relation to the driving activity, even if each characteristic factor independently does not exceed its respective warning level (threshold). Moreover, the case may be such that increase of an effect of one factor reduces that of the other factor on the cognitive condition to be below the warning threshold (i.e., antagonistic effect), and only the combined effect appears to be the effective measure of the operational cognitive state of the driver.

[0095] In particular, there might be a situation where an effect of each of such different factors and each corresponding cognitive condition, when considered independently from the other factor(s) and condition(s), may not exceed a warning threshold level defined for the respective cognitive condition. However, the combined effect of two or more different factors and corresponding cognitive conditions on a driver's operational cognitive state surpasses the warning threshold, necessitating a warning or an alert.

[0096] Thus, in some embodiments of the present disclosure, upon identifying a critical level of the cognitive condition of the driver affected by one or more factors (e.g., drunkenness or fatigue), the triggering utility 120 is configured and operable to generate the triggering signal TS. The control system 102 is configured and operable to be responsive to the triggering signal TS to obtain and analyze the first motion data MD1, or the first and second motion data MD1 and MD2 (as described above), thereby increasing the certainty of detection of a critically impaired cognitive condition of the driver. Reference is made to Fig. 4 exemplifying a flow diagram 200 of a method of the present disclosure for creation of machine learning model to be used to implement the aboveO-described model-based processing for assessment of the operational cognitive condition of a driver (constituting a human operator) during driving (operating) a vehicle having a semi-autonomous driving system (operational system).

[0097] In step 202, a first train set of motion patterns is provided, indicative of motions originated in at least one body part of a driver and measured on at least one vehicle control component (e.g., the steering wheel of a vehicle) during a plurality of driving sessions performed by drivers having different levels of impaired cognitive conditions (affected by different values of one or more characteristic factors). The first train set of motion patterns corresponds to driver-related motion data in relation to the at least one vehicle control component.

[0098] The one or more characteristic factors may for example include one of fatigue and drunkenness. Different values of one or more characteristic factors may be, for example, values of Blood Alcohol Content (BAC) in the range of 0 - 0.1%, and Karolinska Sleepiness Scale (KSS) levels 1-10 of fatigue.

[0099] Motion data related to at least one vehicle control component may include at least one of the following: steering wheel angular motion, motion associated with driver's pressure on a gas pedal, motion associated with driver's pressure on a breaking pedal.

[0100] In step 204, a second train set of motion patterns is provided, measured on the at least one vehicle control component by a controller of an autonomous driving system (e.g., by local controller 126 of the autonomous driving system 122) during a plurality of autonomous driving sessions under different external conditions.

[0101] The inventors have found that, since many factors may affect the driving dynamics and style of a human driver, the second train set obtained from the autonomous driving system may serve as a reference autonomous driving motion data representing an ideal motion data of a driver at a normal cognitive condition in relation to the at least one vehicle control component at each external condition.

[0102] Using said first and second train sets, in step 206, machine learning processing is performed, and one or more difference models are created, wherein each difference model indicates a thresholding condition for a difference between the driver-related motion data and the ideal motion data in relation to the at least one vehicle control component. The thresholding condition defines a critical impaired state of the operational cognitive condition of the driver.

[0103] The algorithm / model can be trained on samples of data from different levels of the cognitive state. The input is provided with custom made features after extensive manipulation on the signal, in order to separate the components which are relevant to an abnormal cognitive state. In order to classify cognitive states, an ensemble of machine learning models can be used and deep learning algorithms as boosted decision trees, bagging decision trees, Ridge and Lasso regression, SVM and residual neural network.

[0104] More specifically, the inventors have analyzed the motion data obtained under various conditions to find embeddings (vector representations) that may be used by machine learning models for the purpose of identifying meaningful data about each motion data. Technically, embeddings are vectors (i.e., arrays of numbers) created by machine learning models making it possible to search for similar objects. In the present disclosure, embeddings representing movements are identified using deep learning and transformers for time series. Then, the distances of the embeddings related to the human action from the embeddings related to the autonomous driver are quantified.

[0105] Outlier scores may be obtained using known outlier detection techniques like Isolation Forest (detecting anomalies using binary trees) and / or COPula-based Outlier Detection (COPOD).

[0106] Further, the present disclosure may utilize known techniques used under the Multi- Agent Reinforcement Learning (MARL) paradigm. MARL is specifically adapted to be used in autonomous mobility and traffic scenarios where multiple agents need to interact to find a common solution. In particular, the techniques disclosed in Tianyi Hu et al., “Measuring Policy Distance for Multi-Agent Reinforcement Learning”, arXiv:2401.11257, 2024, can be used to improve the performance of MARL.

[0107] Reference is made to Fig. 5 exemplifying three traces of steering wheel movement under different conditions: an autonomous driver, a sober human driver, and a drunk human driver. It can be appreciated that the traces representing a sober human driver and an autonomous driver are similar to a large extent, whereas the trace representing a drunk human driver is noticeably different (e.g., is characterized by sharp slopes and high (absolute) magnitudes) and thus can be differentiated from the traces of the autonomous driver by finding the respective embeddings as described above.

[0108] Thus, the present disclosure provides an effective technique for assessment of an operational cognitive condition of a human operator during operation of a vehicle having a (semi) autonomous operation system to meet safety requirement of the vehicle's operation, and enable, when needed transitioning of the control from the (semi) autonomous operation system to the human operator, as well as control of the (semi) autonomous operation system by the human operator, while ensuring that the human operator is fully competent to implement such tasks.

Claims

CLAIMS:

1. A method for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operation system, the method comprising: obtaining first motion data being human operator related motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in a cognitive condition of the human operator; analyzing said first motion data by applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous operational motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

2. The method according to claim 1, further comprising: obtaining second motion data in relation to said at least one vehicle control component from the semi- autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating of the vehicle; and analyzing said first and second motion data by applying model-based processing to said first and second motion data utilizing said predetermined trained model based on reference autonomous operational motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating of a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

3. The method according to claim 1 or 2, further comprising in response to said warning data, generating a corresponding alert to at least one of the human operator and an authorized entity about the impaired state of the operational cognitive condition of the human operator.

4. The method according to any one of the preceding claims, further comprising in response to said warning data, activating an operational mode of the semi-autonomous vehicle to compensate for impairment in the operational cognitive condition of the human operator.

5. The method according to any one of the preceding claims, wherein said obtaining of the first motion data comprises communication with a sensing system installed in the vehicle to receive, from said sensing system, sensing signals indicative of motion dynamics of at least one vehicle control component during the operating of the vehicle.

6. The method according to any one of claims 2 to 5, wherein said obtaining of the second motion data comprises communication with a controller of the semi-autonomous operational system to receive sensing signals indicative of motion dynamics of at least one vehicle control component during the operating of the vehicle as being read by said controller of the semi-autonomous operational system.

7. The method according to claim 5 or 6, wherein said motion dynamics of at least one vehicle control component comprises at least one of the following: steering wheel angular motion, motion associated with a vehicle's control joystick, motion associated with a vehicle's control computer mouse, motion associated with eyes' movement of the human driver, motion associated with pressure applied by the human operator on a gas pedal, motion associated with pressure applied by the human operator on a breaking pedal, motion associated with raising and / or lowering of forklift in a shovel.

8. The method according to claim 1, wherein a vehicle operational session is performed continuously by the human operator, steps of said obtaining of the first motion data and said analyzing of the first motion data being performed substantially continuously during an operational session of the vehicle.

9. The method according to claim 1, wherein steps of said obtaining of the first motion data and said analyzing of the first motion data are performed periodically during an operational session of the vehicle.

10. The method according to claim 1, wherein steps of said obtaining of the first motion data and said analyzing of the first motion data are performed in response to a control signal from a controller of the semi-autonomous operational system.

11. The method according to claim 1, wherein steps of said obtaining of the first motion data and said analyzing of the first motion data are performed in response to a triggering signal being indicative of a critical level of a cognitive condition of the human operator affected by one or more factors, being independently determined by a modelbased processing of the sensing signals.

12. The method according to any one of claims 2 to 7, wherein said semi-autonomous operational system operates in a background control mode while a vehicle's operational session is performed continuously by the human operator, steps of said obtaining of the first and second motion data and said analyzing of the first and second motion data being performed substantially continuously during the vehicle's operational session.

13. The method according to any one of claims 2 to 7, wherein steps of said obtaining of the first and second motion data and said analyzing of the first and second motion data are performed periodically during a vehicle's operational session.

14. The method according to any one of claims 2 to 7, wherein steps of said obtaining of the first and second motion data and said analyzing of the first and second motion data are performed in response to a control signal from a controller of the semi-autonomous operational system.

15. The method according to any one of claims 2 to 7, wherein steps of said obtaining of the first and second motion data and said analyzing of the first and second motion data are performed in response to a triggering signal being indicative of a critical level of a cognitive condition of the human operator affected by one or more factors, being independently monitored by a model-based processing of sensing signals.

16. The method according to claim 15, wherein said independent monitoring of the cognitive condition of the human operator utilizes analysis of independently measured motion data to identify changes in the independently measured motion data caused by one or more characteristic factors defining an effect on the level of the cognitive condition.

17. The method according to claim 16, wherein the changes in the independently measured motion data are caused by at least first and second independent characteristic factors defining together a combined effect on the cognitive condition of the human operator; said independent monitoring comprising determining whether said independently measured motion data comprises one or more features corresponding to apredetermined thresholding condition of said combined effect, irrespective of whether or not a separate effect of one of said first and second characteristics factors has reached a corresponding thresholding level.

18. The method according to claim 15, wherein the independent monitoring of the cognitive condition of the human operator to identify the critical level of the cognitive condition of the human operator comprises analysis of the first motion data to identify changes in the first motion data caused by one or more characteristic factors defining an effect on the level of the cognitive condition.

19. The method according to claim 18, wherein the changes in the first motion data are caused by at least first and second independent characteristic factors defining together a combined effect on the cognitive condition of the human operator; said independent monitoring comprising determining whether said first motion data comprises one or more features corresponding to a predetermined thresholding condition of said combined effect, irrespective of whether or not a separate effect of one of said first and second characteristics factors has reached a corresponding thresholding level.

20. The method according to any one of claims 16 to 19, wherein the one or more characteristic factors comprise one of fatigue and drunkenness.

21. The method according to claim 15, wherein said independent monitoring of the cognitive condition of the human operator utilizes analysis of independently measured image data indicative of the level of the cognitive condition of the human operator.

22. The method according to claim 15, wherein said independent monitoring of the cognitive condition of the human operator utilizes analysis of independently measured one or more physiological parameters of the human operator indicative of the level of the cognitive condition of the human operator.

23. The method according to any one of claims 2 to 22, further comprising: prior to applying said model -based processing of the first and second motion data, momentarily deactivating the at least one vehicle control component associated with said at least one body part of the human operator from affecting operation of the vehicle, such that control of the operation of the vehicle is passed to the semi-autonomous operational system while keeping the first motion data indicative of motion dynamics originated onsaid at least one body part of the human operator and related to said at least one vehicle control component.

24. A method for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operational system, the method comprising: obtaining first motion data being human operator related motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in a cognitive condition of the human operator; obtaining second motion data in relation to said at least one vehicle control component from the semi- autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating of the vehicle; analyzing said first and second motion data by applying model-based processing to said first and second motion data utilizing a predetermined trained model based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

25. A system for carrying out the method of any one of the preceding claims, for assessing an operational cognitive condition of a human operator while operating a vehicle having a semi-autonomous operation system.

26. A system for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operational system, the system comprising a control system configured as a computerized system comprising input and output utilities, a memory utility and a data processor and analyzer utility, and being configured and operable to communicate with a sensing system to receive first motion data indicative of motion dynamics, originated on at least one body part of the human operator and related to at least one vehicle control component, wherein:the data processor and analyzer is configured and operable for analyzing said first motion data to identify changes in an operational cognitive condition of the human operator, said analyzing comprising applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of an impaired state of the operational cognitive condition of the human operator.

27. The system according to claim 26, further comprising the sensing system comprising one or more motion sensors configured and operable to provide said first motion data.

28. The system according to claim 26 or 27, wherein said control system is further configured and operable to communicate with a controller of the semi-autonomous operational system to receive sensing signals indicative of motion dynamics of said at least one vehicle control component during the operating of the vehicle as being read by said controller of the semi-autonomous operational system; and said data processor and analyzer is further configured and operable to carry out the following: obtain second motion data in relation to said at least one vehicle control component from a controller of the semi-autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating the vehicle; and analyze said first and second motion data by applying model-based processing to said first and second motion data utilizing said predetermined model trained based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

29. The system according to any one of claims 26 to 28, further comprising a notification utility configured and operable to be responsive to said warning data togenerate a corresponding alert to at least one of the human operator and an authorized entity about the impaired state of the operational cognitive condition of the human operator.

30. The system according to any one of claims 26 to 29, further comprising an activator configured and operable to be responsive to said warning data or to said alert to activate an operational mode of the semi-autonomous vehicle to compensate for impairment in the operational cognitive condition of the human operator.

31. The system according to any one of claims 26 to 30, wherein said motion dynamics of at least one vehicle control component comprises at least one of the following: steering wheel angular motion, motion associated with a vehicle's control joystick, motion associated with a vehicle's control computer mouse, motion associated with eyes' movement of the human driver, motion associated with pressure applied by the human operator on a gas pedal, motion associated with pressure applied by the human operator on a breaking pedal, motion associated with raising and / or lowering of forklift in a shovel.

32. The system according to claim 26 or 27, wherein said control system has one of the following configurations: (i) substantially continuously during an operating session, obtain and analyze the first motion data; (ii) obtain and analyze the first motion data periodically during an operating session; and (iii) obtain and analyze the first motion data in response to a control signal from a controller of the semi-autonomous operational system.

33. The system according to claim 26 or 27, further comprising: a characteristic factor model-based processor configured and operable to analyze the sensing signals to independently monitor a cognitive condition of the human operator; and a triggering utility configured and operable upon identifying a critical level of the cognitive condition of the human operator affected by one or more factors, generating a triggering signal; the control system being configured and operable to be responsive to the triggering signal to perform said obtaining and analyzing of the first motion data.

34. The system according to any one of claims 28 to 33, wherein said control system has one of the following configurations: (i) substantially continuously during an operatingsession, obtain and analyze the first and second motion data; (ii) obtain and analyze the first and second motion data periodically during an operating session; and (iii) obtain and analyze the first and second motion data in response to a control signal from a controller of the semi- autonomous operational system.

35. The system according to any one of claims 28 to 33, further comprising a controller configured and operable to momentarily deactivate the at least one vehicle control component associated with said at least one body part of the human operator from affecting operation of the vehicle, such that control of the operation of the vehicle is passed to the semi-autonomous operational system while keeping the first motion data indicative of motion dynamics originated on said at least one body part of the human operator and related to said at least one vehicle control component; and allow the data processor and analyzer to perform said model-based processing of the first and second motion data.

36. The system according to any one of claims 28 to 35, further comprising: a characteristic factor model-based processor configured and operable to analyze the sensing signals to independently monitor a cognitive condition of the human operator to identify a critical level of the cognitive condition of the human operator affected by one or more factors; and a triggering utility configured and operable to generate triggering signal, upon identifying the critical level of the cognitive condition of the human operator; the control system being configured and operable to be responsive to the triggering signal to perform said obtaining and analyzing of the first and second motion data.

37. The system according to claim 34 or 36, wherein said independent monitoring of the cognitive condition of the human operator utilizes analysis of independently measured motion data to identify changes in said independently measured motion data caused by one or more characteristic factors defining an effect on the level of the cognitive condition.

38. The system according to claim 37, wherein the changes in the independently measured motion data are caused by at least first and second independent characteristic factors defining together a combined effect on the cognitive condition of the human operator; said independent monitoring comprising determining whether saidindependently measured motion data comprises one or more features corresponding to a predetermined thresholding condition of said combined effect, irrespective of whether or not a separate effect of one of said first and second characteristics factors has reached a corresponding thresholding level.

39. The system according to claim 37 or 38, wherein the characteristic factor comprises one of fatigue and drunkenness.

40. The system according to claim 34 or 36, wherein said independent monitoring of the cognitive condition of the human operator utilizes analysis of independently measured image data indicative of the level of the cognitive condition of the human operator.

41. The system according to claim 34 or 36, wherein said independent monitoring of the cognitive condition of the human operator utilizes analysis of independently measured one or more physiological parameters of the driver indicative of the level of the cognitive condition of the human operator.

42. A system for assessing an operational cognitive condition of a human operator during operating a vehicle having a semi-autonomous operational system, the system comprising: a sensing system comprising one or more motion sensors configured and operable to provide first motion data indicative of motion dynamics, originated on at least one body part of the human operator r and related to at least one vehicle control component, said first motion data being indicative of changes in an operational cognitive condition of the human operator, a control system configured and operable to communicate with the sensing system to receive said first motion data, the control system comprising a data processor and analyzer configured and operable to: analyze said first motion data by applying model-based processing to said first motion data utilizing a predetermined trained model based on reference autonomous driving motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for said first motion data generating warning data indicative of an impaired state of the operational cognitive condition of the human operator.

43. A system for assessing an operational cognitive condition of a human operator r during operating a vehicle having a semi-autonomous operational system, the system comprising: a sensing system comprising one or more motion sensors configured and operable to provide first motion data indicative of motion originated on at least one body part of the human operator, related to at least one vehicle control component, said first motion data being indicative of changes in an operational cognitive condition of the human operator, a control system comprising a data processor and analyzer configured and operable to: obtain said first motion data and obtain second motion data in relation to said at least one vehicle control component from said semi- autonomous operational system, said second motion data being indicative of real time condition of said semi-autonomous operational system during said operating the vehicle; and analyze said first and second motion data by applying model-based processing to said first and second motion data utilizing a predetermined trained model based on reference autonomous operation motion data representing ideal motion data in relation to said at least one vehicle control component and corresponding to operating a semi- autonomous vehicle by a human operator at a normal cognitive condition thereof, and upon identifying a thresholding condition for a difference between said first motion data and said second motion data generating warning data indicative of a critical impaired state of the operational cognitive condition of the human operator.

44. A method for creating a model for use in monitoring an operational cognitive condition of a driver during driving a vehicle having a semi-autonomous driving system, the method comprising: providing a first train set of motion patterns indicative of motions originated in at least one body part of a driver and measured on at least one vehicle control component during a plurality of driving sessions by drivers having different levels of impaired cognitive conditions affected by different values of one or more characteristic factor, said first train set of motion patterns corresponding to driver-related motion data in relation to said at least one vehicle control component;providing a second train set of motion patterns measured on said at least one vehicle control component by a controller of an autonomous driving system during a plurality of autonomous driving sessions under different external conditions, said second train set presenting ideal motion data in relation to said at least one vehicle control component and various external conditions and corresponding to driving of a semi- autonomous vehicle by a driver at a normal cognitive condition thereof at each external condition; using said first and second train sets to perform machine learning processing and create one or more difference models, wherein each difference model indicates a thresholding condition for a difference between said driver-related motion data and said ideal motion data in relation to said at least one vehicle control component, said thresholding condition defining a critical impaired state of the operational cognitive condition of the driver.

45. The method according to claim 44, wherein said one or more characteristic factors comprise at least one of fatigue and drunkenness.

Images