Distraction detection system

Abstract

System comprising: an electrodermal potential detection system integrated into the vehicle cabin (115) to detect a person and designed to output an electrodermal potential signal; at least one additional vehicle sensor for acquiring vehicle-related data; and a controller (102) for receiving the electrodermal potential signal from the electrodermal potential detection system and the vehicle sensor to determine a person's state of distraction using both the electrodermal potential signal and the vehicle-related data, and to reduce the probability of a false state of distraction using only one of the vehicle-related data or the electrodermal potential signal, wherein the controller (102) is designed to output a control signal when the state of distraction is determined and confirmed to be correct.

Description

Technical field

[0001] This patent application relates to systems with integrated sensors to provide captured information about a person's distracted state. background

[0002] Being able to monitor a person's focus and attention is advantageous. For example, distracted driving, a type of driving error, is a significant cause of avoidable traffic accidents. Vehicle systems that assist in warning a driver of distracted driving or take action in such cases can reduce the number of such accidents or represent an attempt to mitigate damage caused by driver distraction. Summary

[0003] Systems and methods for detecting distraction or lack of concentration are described. The system may include an electrodermal potential (EDP) detection system, which may be at least partially integrated into the vehicle cabin and may have a singular or combined configuration, including the vehicle seat, headliner, pillars, instrument panels, and steering wheel to detect a person, and which may be designed to output an electrodermal potential signal; and at least one additional sensor to acquire additional data that can be used to determine distraction.and a controller for receiving the electrodermal potential signal from the electrodermal potential detection system and the vehicle sensor to determine a person's state of distraction using both the electrodermal potential signal and the vehicle-related data, and to reduce the probability of a false state of distraction using only one of the vehicle-related data or the electrodermal potential signal, wherein the controller is designed to output a control signal when the state of distraction is identified or exceeds a distraction threshold. In one embodiment, the system can also determine whether or not a false distraction exists.

[0004] In one embodiment, the control system is designed to determine a false deflection state based on both the vehicle-related data and the electrodermal potential signal.

[0005] In one embodiment, the control signal serves to adjust the operation of an adaptive braking system in the vehicle.

[0006] In one embodiment, the system includes a seat designed to support the person as an occupant and to be installed in a vehicle; and wherein the electrodermal potential detection system includes a non-contact sensor mounted in the seat next to the occupant's head.

[0007] A vehicle seating system with sensors for detecting the distraction of a driver or vehicle occupant seated in a vehicle seat. The seat can be designed to support an occupant and be installed in a vehicle. An electrodermal potential detection system is at least partially integrated into the seat to detect physiological characteristics of an occupant, such as a driver, and is designed to output an electrodermal potential signal. The detected physiological characteristics of the occupant may include cortical brain activity. A controller is located in the vehicle to receive the electrodermal potential signal from the electrodermal potential detection system and to determine the driver's distraction level.The control system also determines a false distraction state using the distraction state and other sensor signals in a vehicle; the control system outputs a control signal if the distraction state exceeds a distraction threshold and if no false distraction is determined.

[0008] In one embodiment, the control signal serves to adjust the operation of a collision avoidance system or an adaptive braking system in the vehicle.

[0009] In one embodiment, the electrodermal potential system includes a multitude of contactless sensors mounted in the vehicle cabin.

[0010] In one embodiment, the seat includes a headrest. The array of non-contact sensors includes one or more headrest sensors mounted in the headrest to measure the electrodermal potential on the driver's head.

[0011] In one embodiment, the seat includes a driver warning device to indicate to the driver that the control signal is being issued by the control unit.

[0012] In one embodiment, the control system measures the driver's distraction based on individual frequency components or parts thereof in the electrodermal potential signal.

[0013] In one embodiment, the controller uses the electrodermal potential signal as an input to determine the driver's distraction and, when the distraction is determined, outputs the control signal to increase a variable "time to impact" in the object avoidance calculation.

[0014] In one embodiment, the sensor signals include a video output from a cabin camera to identify the driver. The control system can use the video output and the electrodermal potential signal to determine the driver's level of distraction.

[0015] In one embodiment, the sensor signals include a navigation position signal from a navigation position sensor to determine the vehicle's position. The control unit can use the navigation position signal and the electrodermal potential signal to determine whether the driver is experiencing an incorrect state of distraction.

[0016] In one embodiment, the sensor signals include an external camera signal from an outward-facing imaging device to generate a video of the vehicle's exterior. The control system can use the external camera signal and the electrodermal potential signal to determine the driver's state of false distraction.

[0017] In one embodiment, the sensor signals include an internal video signal, an external camera signal, a navigation position signal, and a vehicle speed signal. The controller can use the internal video signal, the external camera signal, the navigation position signal, the vehicle speed signal, and the electrodermal potential signal to determine a possible false state of driver distraction or to correct the distraction state through a series of countermeasures.

[0018] A vehicle system is described which includes a vehicle safety sensor system designed to detect external objects around the vehicle and output an external sensor signal. The vehicle system may also include a seat designed to support an occupant and be installed in a vehicle, and an electrodermal potential system, at least partially integrated into the seat and designed to output an electrodermal potential signal. The electrodermal potential system comprises a variety of non-contact sensors mounted in the vehicle. A controller serves to receive the electrodermal potential signal from the electrodermal potential system and the external sensor signal, and to output a control signal using the electrodermal potential signal and the external sensor signal to adjust the operation of the vehicle safety sensor system within the vehicle.

[0019] In one embodiment, the vehicle safety sensor system includes a detection and distance measurement system with a range setting for detecting external objects, including the position and distance of an external object, e.g., a natural obstacle, another vehicle, an animal, or a person, and the external sensor signal includes the position and distance of the external object.

[0020] In one embodiment, the controller outputs a range extension signal when, using the electrodermal potential signal, the controller determines that the driver is distracted or lacks concentration on driving a vehicle, and the vehicle safety system extends the range setting when the controller outputs the range extension signal.

[0021] In one embodiment, the vehicle safety sensor system includes a light sensor, a LIDAR, a camera, or combinations thereof.

[0022] In one embodiment, the vehicle safety sensor system includes a radio frequency sensor, radar, or both.

[0023] In one embodiment, the vehicle system includes a collision avoidance system with a trigger time based on the control signal from the controller. The collision avoidance system is designed to initiate an avoidance action based on this trigger time.

[0024] In one embodiment, the collision avoidance system has a first trigger time when no deflection is detected, and a second trigger time when a deflection is detected. The second trigger time is shorter than the first trigger time.

[0025] A vehicle system is described that uses at least two sensors to detect two different criteria. When processed by a controller, this information indicates distraction or inattention of the occupant or driver. In one example, a first sensor detects a first criterion related to distracted driving, which is controlled by the driver. In another example, a second sensor detects a second criterion related to distracted driving, which represents an environmental condition not controlled by the driver. A controller receives the first and second criteria and determines a relative relationship between them. When this relative relationship exceeds a distraction threshold, it indicates distracted driving.

[0026] In one example, the first criterion is the vehicle speed, and the second criterion is a traffic jam, other vehicles traveling at different speeds alongside the vehicle, or a combination of both. In this example, the control system compares the vehicle speed relative to the second criterion, and if the vehicle speed is slower relative to the second criterion, the control system will indicate driver distraction.

[0027] The examples mentioned above can be combined to form additional embodiments of the present disclosure document. List of characters Fig. Figure 1 is a schematic representation of a vehicle according to an exemplary embodiment. Fig. Figure 2 is a schematic representation of a vehicle seat with built-in sensors according to an exemplary embodiment. Fig. 3A is a functional block diagram of a vehicle system according to an exemplary embodiment. Fig. 3B is a functional block diagram of a vehicle system according to an exemplary embodiment. Fig. Figure 4 is an overview of an incorrect deflection determination according to an exemplary implementation. Detailed description

[0028] As requested, detailed embodiments of the present invention are disclosed here. However, it is understood that the disclosed embodiments are merely examples of the invention and can be implemented in various and alternative forms. The figures are not necessarily to scale; some features are shown enlarged or reduced to highlight details of certain components. Specific structural and functional details disclosed here are therefore not to be considered limiting, but merely as a representative basis for illustrating the many embodiments of the present invention to those skilled in the art.

[0029] This patent application generally relates to in-vehicle sensors that may be embedded, at least partially, in the vehicle cabin or in a part of the foam, upholstery, headrest, frame, or a combination thereof of a vehicle seat. The sensors may also be located in the headliner, instrument panel, vehicle pillars, steering wheel, or combinations thereof. At least one of the sensors determines the electrodermal potential, which originates primarily from cortical brain activity. Such EDP measurement can be performed with or without contact (e.g., field measurement) and can also detect muscle activity and skin characteristics. This reveals higher-order functions of the central nervous system (CNS), such as distraction or drowsiness. The systems described here utilize real-time processing of fluctuations in the electrical potential, caused, for example, by...Comparing different frequency bands of the captured signal with each other. These can serve as quantitative classifiers for primary brain activity. The systems presented here can use the captured signals together with other sensor information to determine false positive distraction reports based on the captured EDP signal. By capturing the appropriate physiological metric and using a software algorithm, this system can determine whether the occupant is distracted or inattentive to the current road demands and simultaneously correct false positive indications of distraction.

[0030] A distraction detection system can be integrated into the seat and include one or more sensors embedded in any part of the seat, such as the foam, upholstery, headrest, or combinations thereof. The contactless EDP detection system can be augmented by suitable physiological metrics [heart rate, heart rate variability (HRV), cardiorespiratory coupling / synchrogram (CRS), respiratory rate, EDP pattern shift, and the like for both standard and complex nonlinear dynamics] of the occupant, e.g., the driver. A controller can receive the relevant signals from the detected physiological metrics and determine whether the occupant is distracted, and thus whether attention and reaction time are impaired. The controller can be adapted to individual occupants using automated, user-specific calibration.

[0031] This system can also include cameras strategically positioned to face the driver. Inward-facing cameras can be used in conjunction with the seat sensors to achieve sensor aggregation and increase the specificity and accuracy of distraction level determination. The camera generates multiple images of the occupant, which can be analyzed to determine additional occupant metrics. These metrics can include head position, blink rate, pupil dilation, eye position, staring, gaze patterns, eyelid closure, head movement, facial expression, overall posture, and the like. The camera system captures an image, and an image processing circuit analyzes the image to determine the image metrics.

[0032] Using various metrics from different sources provides an objective quantification of occupant distraction. This distraction quantification can be combined with other vehicle data to avoid false indications of distraction, such as vehicle performance, driving environment, and the like. If the distraction quantification level exceeds a distraction threshold, the vehicle can automatically initiate countermeasures, such as messages, alarms, collision avoidance, and the like. With the driver's distraction status quantified, the vehicle can modify the reaction times of collision avoidance systems, such as the adaptive braking system, to optimize the system's response based on the driver's state, which is determined, at least in part, by the distraction level.

[0033] A vehicle system is described that uses at least two sensors to detect two different criteria. When processed by a controller, this information indicates distraction or inattention of the occupant or driver. In one example, a first sensor detects a first criterion related to distracted driving, which is controlled by the driver. In another example, a second sensor detects a second criterion related to distracted driving, which represents an environmental condition not controlled by the driver. A controller receives the first and second criteria and determines a relative relationship between them. When this relative relationship exceeds a distraction threshold, it indicates distracted driving.

[0034] Fig. 1 shows a vehicle 100 with a cabin 115 and an engine compartment116 , who was standing in front of the cabin 115 The engine compartment 116 contains a motor 101 , which provides the driving force for the vehicle. A control system 102 It includes an electrical signal processor configured to perform tasks that can be stored in memory. These tasks can process captured signals according to rules that are fed into the control system. 102 The collected data can be loaded. It can be stored in a memory that is part of the control system. 102 heard.

[0035] Visual systems 103 are designed to receive commands from the controller 102to receive and generate visual displays in the vehicle, such as on display screens in the cabin, the dashboard, or a mobile electronic device belonging to the vehicle. The displays generated by the visual systems can be images taken by an internal camera. 104 , an external camera 105 These systems generate collision warnings, distraction warnings, and the like. The visual systems 103 Can the image data from the cameras be accessed? 104 , 105 process before the image data is used for control 102 be provided. The visual system 103 In one embodiment, the images can be processed to identify objects and the driver's position. This data can be used for control. 102 be provided.

[0036] An audio system 106It can be part of a head unit in the vehicle. The head unit can be an electronic processor for processing audio signals or signals captured within the vehicle. The audio system 106 Can noises in the cabin 115 Capture and output sounds into the cabin, e.g., using multiple loudspeakers. Audio output from the audio system 106 It could be a warning, as described here, or a command from the controller. 102 The acoustic warnings can be spoken words or sounds to indicate driver distraction, a change in settings, an impending hazard, activation of the collision warning system, or combinations thereof.

[0037] A vehicle speed sensor 107 is intended to determine the vehicle speed and provide a speed signal for the control system. 102to provide. The vehicle speed sensor may include the throttle position sensor.

[0038] A navigation positioning system 108 The navigation positioning system determines the vehicle's position by receiving satellite signals or ground-based position signals. 108 It may include a global navigation satellite system (GNSS) such as GPS, Beidou, COMPASS, Galileo, GLONASS, the Indian Regional Navigation Satellite System (IRNSS), or QZSS. The navigation system may include a receiver that receives various correction signals in North America from the US Federal Aviation Administration's (FAA) WAAS system. The navigation position system 108 Provides the accurate position of the vehicle for control. 102 ready.

[0039] A distraction alarm 109 is in the cabin 115 positioned. The distraction alarm 109It can include a mechanical alarm, such as vibration devices, which may be located in the steering wheel or seat. The distraction alarm 109 It could be a signal that causes an electronic mobile device belonging to the vehicle and an occupant of the vehicle to vibrate.

[0040] A vehicle seat 110 is in the cabin 115 arranged and designed to carry a person, e.g., a driver or a passenger. The seat 110 can accommodate a variety of sensors 150 , 155 , 156 They include devices designed to determine various biometric characteristics of the person. The sensors 150 They can be contactless and detect EDP near the head of a seated person. The sensors 155 and 156 They can also determine other biometric information. The sensors 155 , 156It can be contactless and, for example, capture parameters from the occupant without coming into physical contact with them. In some cases, at least some of the sensors can 156 to be in contact with the inmate.

[0041] A braking system 111 is designed to brake the vehicle's wheels. The braking system 111 can be activated by the driver and can also be activated automatically by the control system 102 They can be activated, for example, when distracted driving is detected, an imminent collision is detected, or an imminent hazard is detected, as described here.

[0042] A laser detection system 112 A device such as a LiDAR is provided. The laser detection system 112 emits pulsed light and detects the returning light after it is emitted from an object outside the vehicle. 100 was reflected. The laser detection system112 It can generate a digital three-dimensional representation of the external environment around the vehicle in the direction of the light pulses. The laser detection system 112 It can perform a laser scan to generate a representation around the vehicle. The external environment can include other vehicles, traffic signs, animals, people, and other objects. The representation, or individually identified objects, can be used for control. 102 to be provided for use in the vehicle as described here.

[0043] A radar detection system 113 The vehicle is equipped with a radar detection system. 113 It emits radio frequency energy pulses and detects the returning pulses to identify objects around the vehicle or to map the external environment. The mapped or individually identified objects can be used for control. 102to be provided for use in the vehicle as described here.

[0044] Other typical vehicle systems can be found in the vehicle 100 They may be included, but are not shown here for the sake of clarity in the drawings. The control system 102 can provide input for these other systems.

[0045] Fig. 2 shows the driver's seat 110 , which is designed to be fixed in the cabin of a motor vehicle. The seat 110 set up to provide a person on a basis 201 in an upright position against a backrest 202 to maintain the base 201 It is fixed to the floor of the vehicle cabin, for example, using rails. A headrest 203 It can be positioned at the top of the backrest and serve as a headrest. The base 201 , the backrest 202 and the headrest 203Each seat includes a rigid frame, layers of padding on the frame, and an outer cover. A variety of sensors can be integrated into the seat. 150 , 155 , 156 be held. A multitude of initial sensors 150 can be in the headrest 203 be arranged and equipped to receive EDP signals from the seat occupant 110 to capture a multitude of secondary sensors 155 can be in the backrest 202 be arranged. The multitude of secondary sensors 155 It can also detect EDP signals from the seated occupant. The multitude of secondary sensors 155 May include at least one sensor that does not detect EDP signals. One or more third sensors 156 are located in the seat base. The third sensors 156 They can also capture EDP signals. The multitude of secondary sensors 155It can include at least one sensor that does not detect EDP signals and can, for example, detect the presence of a person in the seat using sensors in the backrest, or detect the weight of the seat occupant using sensors in the seat base. The sensors 150 They develop raw EDP signals that are filtered to generate analysis signals including frequency components relevant to the EDP of the person in the seat, while irrelevant frequency components are attenuated.

[0046] In another aspect, a method is provided for monitoring the mental state of a person whose body, including the head, is positioned against a headrest next to sensors in the headrest. The method also involves positioning a sensor at least near parts of the body's skin below the head to generate raw signals, and processing these raw signals to produce at least one bandpass-filtered, state-indicating signal, representing the raw signal amplitude within a predefined frequency range, as an indicator of the person's mental state (e.g., distracted state).

[0047] At least one sensor 150It is positioned behind the head, near or adjacent to the occipital-visual cortical region. This can facilitate the accurate measurement of brainwaves, for example, using EDP. Since driving is a visually dominated cognitive task, the ability to identify processing in this anatomical area of ​​the brain (e.g., the visual cortex), as well as other processing and cognitive networks involved in mental processing, offers the ability to specifically monitor visual attention levels. For example, visual habituation is the brain's ability to reduce its response to repeated stimuli once the information has been processed and is no longer perceived as a relevant processing requirement.In addition to generally low visual attention, the inmate should not experience significant habituation patterns, as the visual scenery, though sometimes mundane, is constantly changing and conditions in such areas demand attention. The lack of activity related to visual processing or habituation to visual stimuli can serve as a classification subset for potential distraction, in addition to other brainwave responses and secondary monitoring systems.

[0048] Fig. Figure 3A shows a schematic representation of a process. 300 , which can be implemented to reduce distraction using sensors, e.g., in a vehicle 100 to determine. At 301 The controller monitors data from a sensor array. 303The sensors that belong to the vehicle are recorded. These can be any of the sensors described here. Examples of sensors in the sensor arrangement are shown below. 303 These include the EDP sensor, internal and external imaging devices, laser-based sensors, seat sensors, and the like. The sensor array can accommodate up to... N sensors are included, whereby N any positive integer.

[0049] The sensor arrangement 303 It can monitor a driver or a vehicle seat occupant. Monitoring can be achieved through EDP scanning using contactless sensors. 150The EDP signals are used to determine the driver's level of distraction. These signals can be divided into various sub-signals, for example, at different frequencies, using filters to allow for specific subdivisions into sub-bands. These sub-bands can overlap in frequency ranges. A general frequency range for each sub-band can be defined within a reasonable variance. A first sub-signal can be up to four hertz. A second sub-signal can be from four to seven hertz. A third sub-signal can be from seven to fourteen hertz. A fourth sub-signal can be from fourteen to approximately thirty hertz. A fifth sub-signal can be from approximately thirty hertz to approximately one hundred hertz. Further sub-signals can overlap these ranges for the first through sixth sub-signals, for example, from eight hertz to thirteen hertz.The relationships between these sub-signals can be used to determine whether the driver is distracted from the task of driving. The patterns of the sub-signals or the relationships between several sub-signals can be used to determine whether a distraction is occurring.

[0050] The sensor arrangement 303The system may include a vehicle cabin imaging device, such as a camera, used to detect the driver in the vehicle seat. The camera data is used to determine a driver distraction pattern. The camera can detect driver movement or lack thereof, the driver's facial expressions, or both. The camera data may consist of video signals sent to a data processor in the control unit to determine if the driver conforms to a distraction pattern. The data processor can determine whether the driver's actions or image match a known distraction pattern. Examples of such distraction patterns include a head position or eye position that is not forward-facing in the manner a driver would look out toward a windshield.

[0051] The sensor arrangement 303It can include external imaging devices, cameras, LiDAR (Light Detection and Ranging), RADAR (Radio Detection and Ranging), and SONAR (Sound Navigation and Ranging). These systems can detect objects around the vehicle, such as other vehicles, stop signs, stop lights, and the like. The imaging devices can determine the color and shape of external objects. LiDAR and RADAR can determine the size and position of external objects relative to the vehicle, which may be in motion.

[0052] At 304The data acquired by each sensor is compared to threshold values ​​for the respective sensors. Image data from an imaging device is compared to image data; that is, changes in pixels can be used to indicate that a threshold in the image data has been exceeded. EDP data can be compared across multiple frequencies to determine if an EDP signal, such as a brainwave, exceeds an EDP threshold and indicates distraction. The EDP data can also be compared to EDP patterns over a period of time, with these patterns indicating a person who is focused or distracted. A lidar, radar, or sonar sensor can determine the relative position of the vehicle compared to objects or other vehicles.A navigation sensor can be used to determine the vehicle's position and provide speed data for the current vehicle and other vehicles in its vicinity, driven by the person whose distraction is being assessed. Seat sensors can determine the person's position, metabolic and physiological parameters, biometric parameters, EDP (Emergency Dietary Pattern), and other data related to the person. The individual data points collected can be compared to threshold values ​​stored in a memory within the vehicle. This memory is part of the vehicle's control system.

[0053] At 305 The process determines whether at least two or more of the recorded data points exceed a distraction threshold. 300This depends on the deflection being determined based on data from at least two sensors. The primary sensor can be the EDP sensor, which can be weighted more heavily in the process to help control the vehicle. If only one sensor indicates deflection, the process proceeds step by step. 306 continues and returns to monitoring at step 301 back. If two or more sensors indicate a distraction exceeding triggered thresholds, the process continues at step 310 determines whether the distraction assessment is incorrect. In one example, the EDP data from the EDP sensor itself may incorrectly indicate distraction. The other data can be combined with the EDP data or the sensor results to determine whether the EDP is incorrectly indicating distraction or whether the person is focused on another task. An example of a process for determining incorrectness is described in more detail with reference to Fig. 4 described.

[0054] At step 311 The combined data from the sensor array determine whether the driver is distracted, using the results from two or more sensors or using all the results from the individual sensors.

[0055] If at step 315 If distraction is detected, the vehicle can be stopped at step 317 Initiate countermeasures. If no distraction is found, i.e., the person is focused, the process continues with step 313 and returns to the fact that the control system uses the sensors 301 monitored. Regarding countermeasures at step 317Distraction warnings can include, for example, an audible warning through the vehicle's audio system, a visual warning, or mechanical warnings. Mechanical warnings can include vibration alerts, such as those in the steering wheel, seat, pedals, the driver's mobile phone, or combinations thereof. These warnings can cause vehicle components in contact with the occupant to vibrate or notify the occupant. The driver's mobile phone can be connected to the vehicle electrically via a wired connection or wirelessly via Bluetooth, Wi-Fi, or similar technologies. Countermeasures can also include secondary countermeasures, such as activating or increasing the range of the vehicle's anti-collision systems or adaptive cruise control. Secondary countermeasures are vehicle controls and processes.The primary countermeasures are those that encourage the distracted occupant to refocus and not be distracted.

[0056] In one embodiment, the LIDAR / RADAR range can be increased as a countermeasure. The external camera area can be enlarged. Increasing the detection range allows the systems to detect objects farther away from the vehicle and provides more time for automatic processing to compensate for the driver's distraction. Increasing the time buffers, which represent the distance from the vehicle to an object outside the vehicle, such as other vehicles on the road, road hazards, and the like, increases the distance to an object at which the vehicle can automatically activate a countermeasure or detect the object. Increasing the time buffers reduces the time until impact between the vehicle and the object. The vehicle will then activate the collision avoidance systems earlier if the driver is distracted. After step 317 can the system proceed to step 301return.

[0057] Fig. 3B shows the process 340 , which can be implemented in the vehicle to detect a distracted state of the seat occupant. At step 341 The system determines whether the vehicle is running, for example by checking that the ignition is in an "ON" state. If the vehicle is then switched off, the process ends. 340 at step 365 When the vehicle is ON, it activates its distraction detection system by loading the commands into the control circuit. In one embodiment, the activation of the distraction detection system in the vehicle can be optional, determined, for example, by a setting procedure or deactivated by a switch. In another example, when the vehicle is switched ON, it also initiates the vehicle monitoring system. At step 345The driver of the vehicle has the option to manually deactivate the distraction detection system. If the driver at step 345 If the system decides to deactivate the distraction detection system, the system will be activated at step 363 Switched off. The process 340 It can also be switched OFF if the vehicle is stationary or in park for a period of time. The decision to keep the distraction detection system switched ON can be based on receiving at least two or more sensor signals that can be used to determine distraction. In one example, at least one of the two sensor signals could be the EDP signal.

[0058] When the distraction process 340 at step 345 If the sensor array remains switched on, the process continues with its activation at step 303 continued. The sensor arrangement can be the same sensor arrangement as described in relation to Fig. 3A is described and can contain any sensor as described there. The sensor arrangement is given in step 303 captured data to the control system 102 out, to view the collected data at step 351 to process. The processing at step 351 Can the filtering or normalization of the raw data from the sensors in the sensor array from step 303 include the data collected from step 351 are compared with threshold values ​​that are individualized for each type of sensor data. If at 355 If two or more thresholds indicating that the person is focused are not reached, the process for data collection by the sensor array reverses at step 303Back. The present process is based on the assumption that the occupant is not distracted. It is understood that a similar process, which assumes that the driver is distracted and the system must prove that the occupant is not distracted, is also within the scope of this disclosure.

[0059] It goes without saying that data collection at step 303 This can occur simultaneously with the feedback indicating that the available data does not show any distractions. If two or more different thresholds are reached, the process proceeds with the determination of a false alarm at step 1. 357If an incorrect distraction is determined, the vehicle cannot initiate countermeasures. In one example, navigation position data (e.g., GPS) is used to confirm the distraction determination based on the distraction determination by the EDP signal. In another example, vehicle speed data is used to confirm the distraction determination. In yet another example, images from the inward-facing camera, the outward-facing camera, or both are used to confirm the distraction determination.

[0060] At step 359 A final determination of the distraction is made, taking into account the sensor data on which the determination of the distraction or the incorrect distraction determination depends, in order to reduce the probability of incorrect distraction determinations. If at step 359 Once it is determined that the person is not distracted, the process returns to step 303back. If a distraction is detected, the vehicle may stop at step 360 Initiate countermeasures. After the countermeasures have been initiated, the process returns to the determination stage to run the distraction algorithm at step 1. 345 to remain switched ON. The countermeasures can remain switched ON for a specified period of time, or until the system determines that the occupant is no longer distracted, or until it is determined that the distraction is incorrect. In one embodiment, the countermeasures remain switched ON until the present methods and systems determine that the driver is no longer distracted.

[0061] The present distraction determination processes 300 , 350They use data from two or more different sensors to determine distraction. The EDP sensor can provide the primary data, but data from other vehicle sensors can be combined to determine distraction even more accurately. The other collected data can be data relating to the occupant inside the vehicle cabin, data from outside the vehicle cabin, or both. Combining the distraction determination based on the individual sensors can be used to reduce the likelihood of false distraction readings.

[0062] In one embodiment, the step 343 to activate the distraction monitoring in the vehicle with the sensors 303 at YES, and with step 363 , when monitoring is switched OFF. The step 345 To keep monitoring switched ON, a choice can be made between initiating countermeasures at step 360and the sensors 303 lay.

[0063] Fig. 4 shows a table 400 from various example scenarios 401 , 405 for using numerous inputs from various sources to determine a person's distraction state. Any sensor can be used for the primary data. Any other sensor can be used for secondary data to correct, validate, or invalidate the distraction determination based on the primary data. For example, in Fig. As shown in Figure 4, the captured EDP data can be the primary input for determining a person's distraction as detected by the sensors, as described here. The primary data sensors can include vehicle speed, relative vehicle speed, position data, navigation data, data from the outward-facing camera, data from the inward-facing camera, and the like. The additional use of secondary data can help correct false indications of driver distraction based solely on the primary data. Inputs for secondary data can include vehicle speed, navigation position information (GPS in North America), an outward-facing camera, an inward-facing camera (possibly directed at the driver), and the like.The data from these devices is used in the control circuit to determine whether there is a false indication of the driver's distraction.

[0064] In the first scenario 401Using the sensors as described above, it is determined that the initial signals acquired by the EDP sensor are normal, i.e., within a defined tolerance range. The output from the control circuit indicates that the driver is focused. A focused state of an occupant means that the person in the vehicle is not distracted from the task of driving. Thus, the primary sensor system first determines that the person is not distracted. Subsequently, the secondary data can be used to verify the driver's state of distraction. The vehicle speed relative to the surrounding traffic is assessed as abnormal as a secondary input. The vehicle's current speed is known from the sensors on the dashboard. The speeds of surrounding vehicles can be determined using sonar, radar, or lidar sensors, or a combination thereof, on the vehicle.The speeds of surrounding vehicles can also be transmitted between vehicles in a vehicle communication network. In this case, the relative vehicle speed is abnormal, i.e., outside a defined tolerance or expected range. An additional secondary sensor signal includes navigation position data regarding a vehicle and any potential traffic congestion at the vehicle's location. In this embodiment, the navigation or vehicle position sensor (e.g., GPS in North America) does not detect a traffic congestion. A traffic congestion can be a measurement of a vehicle's position over time relative to the normal overall traffic flow for a specific time of day. This data can include the current vehicle's position and be combined with aggregated data from a server regarding traffic at that location and time of day.Traffic congestion can also be detected by an outward-facing sensor, such as an image sensor, camera sensor, radar sensor, laser sensor, or sonar sensor. In this case, the outward-facing sensor (camera or image sensor) does not detect a traffic congestion. The inward-facing imaging device or camera detects that the occupant is alert and focused. The corrected distraction analysis, based on a summary of the primary and secondary data, leads the control circuit to determine that the occupant is focused and not distracted. Therefore, no countermeasures are triggered to address the occupant's distraction.

[0065] In the second scenario 402The EDP is normal, i.e., within a defined tolerance or range, and the analysis based on the primary sensor data indicates that the occupant is focused. The secondary sensors can be used to verify or confirm the primary sensor analysis. The relative vehicle speed is abnormal. The navigation result indicates no traffic jam. The outward-facing image sensor shows that there is no traffic jam. The inward-facing camera detects abnormal occupant behavior, i.e., outside a defined tolerance or range. The corrected analysis of the primary sensor data, in combination with the secondary sensor data, leads to the determination that the occupant is distracted. The system further determines that the primary sensor data was analyzed and led to an incorrect result.This can serve to inform the algorithm that the result was incorrect. The algorithm in the control circuit for analyzing the primary sensor data can then modify its parameters so that they better match the corrected results from the revised analysis. The system can use the revised analysis of "not concentrated" to trigger countermeasures, as described here.

[0066] In the scenarios 403 - 405The primary sensor determines that the acquired EDP data is abnormal, indicating that the occupant is unfocused or distracted. The acquired EDP data can be compared to concentrated and unfocused waveforms in the control circuit. If the result falls outside the concentrated waveform's variability thresholds, the control circuit can determine that the occupant is unfocused and therefore distracted. Data acquired from the secondary sensors can then be used to correct this determination.

[0067] In the third scenario 403The secondary sensors provide additional data for use with the primary sensor's abnormal determination. The relative vehicle speed is abnormal. The navigation system sensor determines that there is no traffic congestion. The outward-facing imaging device determines that there is no traffic congestion. The inward-facing imaging device provides data indicating an abnormal condition, suggesting that the occupant is distracted. The primary data, or analysis, is combined with the secondary data (one input or more than one output), and it is determined that the seat occupant is not focused on the task of driving, i.e., distracted. The analysis from the first sensor input is confirmed or does not produce a false reading of distraction or an alarm.The countermeasures in the vehicle can be triggered based on the primary analysis and the aggregation of sensor data with the secondary data acquired. These countermeasures can be any type of countermeasure, as described here.

[0068] In the fourth scenario 404The secondary sensors provide additional data for use with the primary sensor's abnormal determination. The relative vehicle speed is normal. The navigation system sensor determines that there is a traffic jam. The outward-facing imaging device determines that there is a traffic jam. The inward-facing imaging device provides data indicating a normal state, suggesting that the occupant is concentrating. The primary data, or analysis, is combined with the secondary data, and it is determined that the occupant is concentrating. It is determined that the analysis from the first sensor input is a false reading, indicating that the driver is distracted. The secondary sensor data corrects the erroneous or false alarm from the primary sensor analysis. The vehicle's countermeasures are not triggered.

[0069] In the fifth scenario 405The secondary sensors provide additional data for use with the primary sensor's abnormal determination. The relative vehicle speed is normal. The navigation system sensor determines that there is a traffic jam. The outward-facing imaging device determines that there is a traffic jam. The inward-facing imaging device provides data indicating an abnormal condition, suggesting that the occupant is distracted. The primary data, or analysis, is combined with the secondary data, and it is determined that the occupant is not concentrating, i.e., distracted. The analysis from the first sensor input is confirmed, or does not produce a false reading of distraction or an alarm. Countermeasures in the vehicle can be triggered based on the primary analysis and the merging of the sensor data with the secondary data.The countermeasures can be any countermeasures as described here.

[0070] The scenarios 401 - 405 This represents an example where the circuit applies an algorithm to the acquired data to output a resulting signal to trigger an alarm, vehicle control, or vehicle countermeasures in an attempt to mitigate the effects of distracted driving. The circuit can utilize artificial intelligence and neural networks to determine distraction and / or false distraction.

[0071] The criteria from the secondary sensors, e.g., as in Fig.Figure 4 shows that, in some embodiments, the secondary sensors can be used to indicate distraction. The secondary sensors can determine secondary criteria related to the occupant or driver inside the vehicle cabin. The secondary criteria can also include detected data related to the environment outside the vehicle. The detected data can be used to derive the secondary criteria. Relative relationships between the criteria detected by the secondary sensors can be determined, and if these relationships indicate distraction, the vehicle can warn the occupant or modify the vehicle's operation. In one embodiment, the vehicle speed is detected and used as a first criterion in determining distraction. This criterion is controlled by the driver. Other first criteria can include detected signals related to the driver, such as…Brain waves, heart rate, heart rate variability, eye movement, posture, and movement, and the like. These are directly controlled by the driver or generated by the driver's body. The second criterion can be a traffic jam, the speeds of other vehicles alongside the vehicle, or a combination of both. The second criterion is not controlled by the driver. The second criterion can relate to a vehicle or data outside the vehicle that is neither controlled nor generated by the driver. The relationship between the first and second criteria can indicate distraction. For example, if the first criterion changes in a known way and the second criterion also changes in a known way, this indicates distraction. A distracted driver may slow down their vehicle (i.e., first criterion) if the surrounding traffic does not slow down (i.e., second criterion).An additional first criterion can also be used to confirm distraction, such as a slower and deeper breathing pattern, a change in posture, a slowed heart rate, etc. In another example, a distracted driver might slow the vehicle (i.e., first criterion), and there is no slowdown due to a traffic jam, other obstacle, a traffic light, or the like (i.e., second criterion). This indicates possible driver distraction. The control system can also consider the vehicle's operational status. If the vehicle experiences some kind of malfunction, the driver does not necessarily have to be distracted. The control system compares the first criterion, such as vehicle speed, relative to the second criterion. If the vehicle speed decreases relative to the second criterion, the control system indicates driver distraction.

[0072] The second criterion can also be relative to vehicle behavior. For vehicles communicating with each other, the first criterion can be the change in vehicle speeds around a predefined speed. The second criterion can be the rate at which the speeds of other vehicles change relative to the speed change of a vehicle driven by the driver that is being detected. If the vehicle in question changes its speed more frequently than the other vehicles, this indicates a distraction.

[0073] The throttle position can be detected using a throttle angle sensor. The throttle position can be used as part of the first criterion, as a supplement to vehicle speed, or as a replacement for vehicle speed. The driver controls the throttle position while cruise control or adaptive cruise control is switched off, which in turn controls the vehicle speed.

[0074] The system can employ adaptive tolerance of the detection system thresholds for both the occupant and the external environment's detection technology to improve the assessment of situational distraction and the readiness for countermeasures. The thresholds of the external system can be influenced by readings from the internal system, and vice versa.

[0075] For example, cognitive attention measurement via recorded electrical brain activity (e.g., EDP) may lower its distraction indicator threshold if metrics such as vehicle speed, weather, and / or lidar / radar systems indicate that the environment contains certain conditions (e.g., proximity to external objects, vehicle speed, wet or icy road conditions) that require higher attention in that situation compared to normal operating conditions when there is no distraction, but where elevated attention levels would not be cause for concern. As with field or contact-based measurement of EDP with respect to the underlying brain activity, internal eye and skeletal monitoring, autonomic nervous system monitoring, external systems, and all other monitoring may have multiple thresholds both in terms of the magnitude of their metric (e.g.,The monitoring system can display cognitive attention (eye opening range, rotation, heart rate, respiratory rate, distance, speed) as well as its temporal resolution (e.g., seconds per degree of permissible head rotation). For example, eye and skeletal monitoring can indicate different degrees of rotation away from the optimal display of the attentional window for varying durations. In situations requiring a higher level of attention, both the maximum degree of deviation from this plane and the time allowed for each degree of deviation can be reduced before an alert is issued and / or a countermeasure is prepared and potentially activated.

[0076] Long-term data regarding the detected distraction can be processed secondarily to the real-time algorithms to provide a wealth of statistical information for both the occupant and machine learning systems. This long-term data can be stored in the vehicle or externally on a remote server. The vehicle may have electronic communication with an external server, for example, via Wi-Fi, mobile communication networks such as cellular networks, and the like. The long-term distraction calculations can be used to modify the commands used to determine distraction or to mitigate false positive results. This patent application quantifies the driver's distraction / concentration status and simultaneously corrects false indications of distraction.The vehicle can use the driver's distraction / concentration state to manipulate the reaction times of various vehicle safety systems, such as the adaptive braking system, in order to optimize the system's own response. This can reduce the risk of frontal collisions.

[0077] The present system can be used in an autonomous vehicle, e.g., automobile(s) of autonomy level 1 until 2 , are used when the vehicle uses the distraction level, a determination of distraction, or the detection of a distracted driver by multiple sensors to determine the most appropriate time to switch from manual driving to autonomous driving and vice versa, or to initiate certain levels of countermeasures.

[0078] The system is suitable for all modes of transport, including those that extend beyond automobiles and private vehicles.

[0079] The present disclosure document shows a control system. 102 It is within the scope of this disclosure that the control 102 It represents multiple processors, memory units, and electronic control units that can operate independently with different systems to perform the functions and tasks described herein. The vehicle can be a more distributed control system than a single controller and still fall within the scope of this patent application. The controller 102 It includes a circuit for processing captured signals that represent real conditions and data.

[0080] This patent application describes the captured EDP data as primary data, and other data relating to the person or vehicle as secondary data. However, some embodiments described in this patent application use the captured EDP as secondary data to correct erroneous distraction determinations based on other non-EDP data. For example, the internal camera and vehicle operation, such as drifting or crossing lane markings, can be used to determine distraction. The EDP signal can then be used to validate a distraction determination.

[0081] Other secondary data can include time, such as the time of day and the time of vehicle use. The vehicle can input the time of day it is driven as a secondary input to prevent false distraction detection or to adjust the camera thresholds for detecting distraction. If the time of day is night, the distraction thresholds can be lowered. The vehicle can also track the typical driving time within a given time frame. If the vehicle is driven outside of these typical times, there may be a higher probability of distracted driving.

[0082] An example of electrodermal potential is a type of electroencephalography (EEG), an electrophysiological monitoring technique used to record the brain's electrical activity. It is typically non-invasive, with electrodes placed along the scalp, although invasive electrodes are occasionally used in certain applications. EEG measures voltage fluctuations resulting from ion currents in the brain's neurons. In clinical contexts, EEG refers to the recording of spontaneous electrical brain activity over a specific period of time, as captured by multiple electrodes positioned on the scalp. Diagnostic applications generally focus on the spectral content of the EEG, that is, the type of neural oscillations that can be observed in the EEG signals.

[0083] Although some exemplary embodiments have been described above, these embodiments by no means encompass all possible forms of the invention. The terms used in the description are not limiting but rather descriptive, and it is understood that various modifications can be made without deviating from the spirit and scope of the invention. Furthermore, the features of the different exemplary embodiments can be combined to form further embodiments of the invention.

Claims

[1] System, encompassing: an electrodermal potential detection system integrated into the vehicle cabin to detect a person and designed to output an electrodermal potential signal; at least one additional vehicle sensor for recording vehicle-related data; and A controller for receiving the electrodermal potential signal from the electrodermal potential detection system and the vehicle sensor to determine a person's state of distraction using both the electrodermal potential signal and the vehicle-related data, and to reduce the probability of a false state of distraction using only one of the vehicle-related data or the electrodermal potential signal, wherein the controller serves to output a control signal when the state of distraction is determined and confirmed to be not false. [2] System according to claim 1, wherein the control is designed to determine a false deflection state based on both the vehicle-related data and the electrodermal potential signal. [3] System according to claim 1, wherein the control signal serves to adapt the operation of an adaptive braking system in the vehicle. [4] System according to claim 1, further comprising a seat designed to support the person as an occupant and to be installed in a vehicle; and wherein the electrodermal potential detection system includes a non-contact sensor mounted in the seat next to the occupant's head. [5] Vehicle seat system according to claim 4, wherein the electrodermal potential system includes a plurality of non-contact sensors mounted in the seat; and wherein the seat includes a headrest, and wherein the plurality of non-contact sensors includes one or more headrest sensors mounted in the headrest to measure the electrodermal potential at the driver's head. [6] Vehicle seating system according to claim 4, wherein the seat includes a driver warning device to indicate to the driver that the control signal is being issued by the control unit. [7] Vehicle seating system according to claim 1, wherein the control measures the driver's distraction based on individual frequency components in the electrodermal potential signal. [8] Vehicle seating system according to claim 1, wherein the controller uses the electrodermal potential signal as an input to determine the deflection of the person, and when the deflection is determined, outputs the control signal to increase a variable “time to impact” in the object avoidance calculation. [9] Vehicle seating system according to claim 8, wherein the sensor signals include a video image output from a cabin camera to identify the person, and wherein the control uses the video output and the electrodermal potential signal to determine the false distraction state of the person. [10] System according to claim 8, wherein the sensor signals include a navigation position signal from a navigation position sensor to determine the position of a vehicle, and wherein the controller uses the navigation position signal and the electrodermal potential signal to determine the false distraction state of the person. [11] System according to claim 8, wherein the sensor signals include an external camera signal from an outward-facing imaging device to generate a video of the outside of the vehicle, and wherein the control uses the external camera signal and the electrodermal potential signal to determine the false distraction state of the person. [12] System according to claim 1, wherein the sensor signals include an internal video signal, an external camera signal, a navigation position signal and a vehicle speed signal, and wherein the controller uses the internal video signal, an external camera signal, the navigation position signal, the vehicle speed signal and the electrodermal potential signal to determine the driver's false state of distraction. [13] Vehicle system, comprising: a vehicle safety sensor system designed to detect external objects around the vehicle and output an external sensor signal; a seat designed to carry an occupant and to be fitted in a vehicle; an electrodermal potential system that is at least partially integrated into the vehicle cabin and designed to output an electrodermal potential signal; and a control unit that serves to receive the electrodermal potential signal from the electrodermal potential system and the external sensor signal, and to output a control signal using the electrodermal potential signal and the external sensor signal to adjust the operation of the vehicle safety sensor system in the vehicle. [14] Vehicle system according to claim 13, wherein the electrodermal potential system includes a plurality of contactless sensors mounted in the seat. [15] Vehicle system according to claim 13, wherein the vehicle safety sensor system includes a detection and distance measurement system with a range setting for detecting external objects including a position and distance of an external object, and the external sensor signal includes the position and distance of the external object. [16] Vehicle system according to claim 15, wherein the controller outputs a range extension signal when the controller determines, using the electrodermal potential signal, that the driver is distracted, and wherein the vehicle safety system extends the range setting when the controller outputs the range extension signal. [17] Vehicle system according to claim 13, further comprising a collision avoidance system with a trigger time based on the control signal from the controller, and wherein the collision avoidance system triggers an avoidance action based on the trigger time. [18] Vehicle system according to claim 17, wherein the collision avoidance system has a first trigger time when the deflection is not detected and a second trigger time when the deflection is detected, wherein the second trigger time is shorter than the first trigger time. [19] Vehicle system, comprising: a first sensor to detect a first criterion relating to distracted driving and controlled by the driver; a second sensor for detecting a second criterion relating to distracted driving and representing an environmental condition not controlled by the driver; and a control for receiving the first criterion and the second criterion and for determining a relative relationship between the first criterion and the second criterion, wherein the relative relationship exceeds a distraction threshold to indicate distracted driving. [20] Vehicle system according to claim 19, wherein the first criterion is the vehicle speed, wherein the second criterion is a traffic jam, other vehicle speeds alongside the vehicle, or a combination of both, and the control compares the vehicle speed relative to the second criterion, and wherein, if the vehicle speed slows down relative to the second criterion, the control indicates a distraction of the driver.

Images