Method for determining a posture of a driver

Abstract

A method for determining the posture of a driver of a vehicle, the vehicle comprising a camera able to generate a sequence of images of the position of the driver of the vehicle and an electronic control unit comprising a storage area in which a plurality of masks for image processing are recorded, each mask being associated with a predetermined posture of the driver on his seat, the method comprising: the camera generating (E1) a sequence of images of the position of the driver and sending the sequence of images to the electronic control unit, the electronic control unit calculating (E2) for each mask of the plurality of masks the convolution of the mask with at least one image of the sequence of images received from the camera in order to obtain a correlation index, and determining (E3) the posture of the driver on the basis of the mask with the highest correlation index.

Description

Technical Field

[0001] This invention relates to the field of transportation, and more particularly to a method for warning vehicle drivers and a vehicle for implementing such a method. The object of this invention is to improve existing warning solutions for vehicle drivers, especially when the driver adopts a dangerous driving posture. This invention is particularly applicable to driving motor vehicles, trains, or aircraft. Background Technology

[0002] Today, it is common practice to monitor motor vehicle drivers so that they can be notified when their behavior may pose a danger to their own safety and the safety of other passengers in the vehicle. For example, it is known to monitor a motor vehicle driver's face to notify him when he shows signs of inattention or drowsiness. It is also known to monitor motor vehicle drivers so that the behavior of the vehicle's airbag system can be adapted to the driver's posture.

[0003] For example, document CN103057503A describes an intelligent airbag device that protects the driver or passenger based on their posture and physical condition. The intelligent airbag device includes a detection unit, a control unit, and an airbag unit. The control unit is connected to both the detection unit and the airbag unit. The detection unit detects the driver's or passenger's physique and posture, as well as the severity of the vehicle collision. The control unit uses this information to determine whether the airbag should be inflated and to determine the inflation mode. The device protects the passenger based on real-time information about their physique and posture, as well as real-time information about the severity of the accident.

[0004] However, this system is particularly complex and expensive because it requires the use of sensors in the driver's seat, collision sensors, and a 3D camera, and all the corresponding data needs to be processed, which requires high and therefore expensive processing power. Furthermore, the system proposes to deactivate the vehicle's airbag system when it deems the driver's posture dangerous, which could be dangerous for the driver and other passengers in the vehicle.

[0005] Therefore, a simple, reliable, and effective solution is needed to at least partially compensate for these shortcomings. Summary of the Invention

[0006] Therefore, the first subject of the present invention is a method for determining the posture of a driver of a vehicle (especially a motor vehicle), the vehicle including a camera capable of generating an image sequence of the driver's position and an electronic control unit including a storage area containing a plurality of masks for image processing, each mask being associated with a predetermined posture of the driver in his seat, including a correct posture of the driver in his seat and at least one incorrect posture of the driver in his seat, the method comprising the following steps:

[0007] The camera generates a sequence of images showing the driver's position and sends the sequence of images to the electronic control unit.

[0008] - The electronic control unit calculates the convolution of each of the plurality of masks with at least one image from the image sequence received from the camera to obtain a correlation coefficient.

[0009] - The driver's posture is determined based on the mask with the highest correlation coefficient.

[0010] The method according to the invention enables the simple, reliable, and efficient detection of a driver's posture using a camera, without the use of other sensors. The invention is particularly applicable to motor vehicles, where it is advantageous to detect the posture of the driver in their seat relative to the vehicle's steering wheel in order to notify them in dangerous posture situations. The invention is also applicable to driving trains or piloting aircraft.

[0011] Preferably, an incorrect driver posture corresponds to a posture in which the driver's chest or head is too far from or too close to the steering wheel, for example, when his chest or head is less than 15cm from the steering wheel or more than 130cm from the steering wheel.

[0012] According to one aspect of the invention, the method further includes the step of generating a warning to the driver when the detected posture corresponds to an incorrect posture of the driver, so as to inform him that his posture is incorrect so that he can correct his posture and thereby avoid changing the state of the vehicle's entire airbag system.

[0013] According to one feature of the invention, the camera is a 2D camera, so that image sequences generated by the camera can be processed simply and quickly.

[0014] Preferably, the camera is placed behind the driver's center rearview mirror so that it can simultaneously capture the driver's chest and head, thereby improving the accuracy of detecting his posture.

[0015] Preferably, the predetermined posture includes a first incorrect posture where the driver is too far from the steering wheel and a second incorrect posture where the driver is too close to the steering wheel.

[0016] In one embodiment, the vehicle further includes a second camera capable of generating a sequence of images of the external environment surrounding the vehicle, particularly the environment in front of the vehicle, and capable of sending the image sequence to an electronic control unit (ECU). The ECU is able to detect a collision risk between the vehicle and external factors in the received image sequence, and when a collision risk is confirmed and the detected posture corresponds to an incorrect posture, generate a warning to the driver. The use of the additional second camera enables the detection of potential collision risks and their notification to the driver.

[0017] According to one aspect of the invention, the method includes a preparatory step of determining a mask for image processing, the preparatory step including generating an image sequence for each predetermined posture of the driver in his seat, and determining a mask for each generated image sequence so that the mask can be applied to images generated by a camera while the vehicle is in motion, thereby determining the driver's posture.

[0018] According to one feature of the invention, determining a mask for each generated image sequence includes determining a pixel region in the generated image sequence corresponding to the driver's body.

[0019] Advantageously, determining a mask for each generated image sequence includes defining the weight of each pixel based on the pose.

[0020] In one embodiment, when the vehicle includes an airbag system (safety inflatable airbag system), the method includes: after detecting a second incorrect posture of the driver, an electronic control unit deactivates the airbag system, preferably only deactivating the airbag located in the steering wheel, to prevent the steering wheel airbag from injuring the driver upon activation; and then, once the driver's posture is detected as correct again, reactivating the airbag or system.

[0021] The present invention also relates to a computer program product, characterized in that it comprises a set of program code instructions, which, when executed by one or more processors, configure the one or more processors to implement the method described above.

[0022] The present invention also relates to an electronic control unit for a vehicle (particularly a motor vehicle), the vehicle including a camera capable of generating an image sequence of the driver's position, the electronic control unit including a storage area recording a plurality of masks for image processing, each mask being associated with a predetermined posture of the driver in his seat, including a correct posture of the driver in his seat and at least one incorrect posture of the driver in his seat, the electronic control unit being able to:

[0023] - Receive a sequence of images showing the driver's position from the camera.

[0024] - For each of the plurality of masks, calculate the convolution of the mask with at least one image from the image sequence received from the camera to obtain a correlation coefficient.

[0025] - The driver's posture is determined based on the mask with the highest correlation coefficient.

[0026] - When the detected posture corresponds to an incorrect posture of the driver, a warning is generated to the driver so that he can correct his posture.

[0027] In one embodiment, the electronic control unit is able to detect the risk of collision between the vehicle and external factors in a sequence of images received from a second camera, and generate a warning to the driver when a collision risk is confirmed and the detected posture corresponds to an incorrect posture.

[0028] In one embodiment, when the vehicle includes an airbag system (safety inflatable airbag system), the electronic control unit is able to deactivate the airbag system after detecting a second incorrect posture of the driver, preferably only deactivating the airbag located in the steering wheel to prevent injury to the driver upon triggering of the steering wheel airbag, and then reactivating the airbag or the system once the driver's posture is detected as correct again.

[0029] The present invention also relates to a vehicle, particularly a motor vehicle, railway vehicle or aircraft, comprising a camera capable of generating an image sequence of the position of the driver of the vehicle and an electronic control unit as described above.

[0030] In one embodiment, the vehicle includes a second camera capable of generating a sequence of images of the external environment of the vehicle, particularly a sequence of images of the environment in front of the vehicle, and capable of sending the image sequence to an electronic control unit.

[0031] In one embodiment, the vehicle includes an airbag system.

[0032] In one embodiment, the vehicle is an autonomous vehicle, and the electronic control unit is programmed to detect when the vehicle is driving in autonomous driving mode in order to avoid generating a warning when an incorrect posture of the driver is detected.

[0033] In one embodiment, the vehicle is an autonomous vehicle, and there are many correct postures for the driver when autonomous driving mode is activated. For example, correct postures may correspond to the driver's sleeping posture, resting posture, reading posture, etc. Therefore, the electronic control unit will not warn the driver of these positions that are considered correct when autonomous driving is activated, but when autonomous driving is deactivated, these positions become incorrect and therefore dangerous, and the electronic control unit warns the driver. Attached Figure Description

[0034] Other features and advantages of the invention will become more apparent from the following description. This description is purely illustrative and should be read with reference to the accompanying drawings, in which:

[0035] Figure 1 An embodiment of the vehicle according to the present invention is shown;

[0036] Figure 2 An embodiment of the method according to the present invention is shown;

[0037] Figure 3 An example of a mask corresponding to the correct pose is shown;

[0038] Figure 4 An example of a mask corresponding to the first incorrect posture of being too far from the steering wheel is shown;

[0039] Figure 5 An example of a mask corresponding to a second incorrect posture of being too close to the steering wheel is shown;

[0040] Figure 6 An example of a mask corresponding to a pose without a driver is shown. Detailed Implementation

[0041] Figure 1 An example of a vehicle 1 according to the invention is shown schematically. In this example, vehicle 1 is a motor vehicle, but in other examples, the vehicle may be a train or an aircraft.

[0042] In addition to the driver's seat and steering wheel (not shown for clarity), the vehicle 1 also includes a first camera 10 and an electronic control unit 20.

[0043] The first camera 10 is capable of generating image sequences showing the position of the driver 5 in the vehicle 1, particularly image sequences of the driver 5's chest and head when the driver 5 is seated in the driver's seat. Preferably, the first camera 10 is a 2D camera positioned behind the driver 5's central rearview mirror, which is mounted in front of the windshield of the vehicle 1 from inside the vehicle 1. The first camera 10 is capable of sending the generated image sequences to the electronic control unit 20.

[0044] In this example, vehicle 1 also includes an airbag system 30 and a second camera 40, which is capable of generating a sequence of images of the surrounding environment in front of vehicle 1, particularly a sequence of images of the road when vehicle 1 is traveling on a road.

[0045] The second camera 40 enables the imaging of the surrounding environment located in front of the vehicle 1, in order to detect collisions, as will be detailed below. The second camera 40 is capable of sending the generated image sequence to the electronic control unit 20. The second camera 40 is optional and not essential for implementing the invention, although it is advantageous.

[0046] The electronic control unit 20 is capable of receiving image sequences transmitted by the first camera 10 and the second camera 40.

[0047] The electronic control unit 20 includes a storage area 21 in which multiple masks M (reference) for image processing are recorded. Figure 2 Each mask M is associated with a predetermined posture of the driver 5 in their seat, including the correct posture of the driver 5 in their seat and at least one incorrect posture of the driver 5 in their seat. In this example, the storage area 21 includes a first mask M corresponding to the correct posture of the driver 5 in their seat, a second mask M corresponding to the posture of the driver 5 being too far from the vehicle's steering wheel, a third mask M corresponding to the posture of the driver 5 being too close to the vehicle's steering wheel, and a fourth mask M corresponding to the driver 5 not being in their seat. For example, in a factory or laboratory, i.e., before the vehicle 1 is used on the road, masks M for image processing are predefined and recorded in the storage area 21 of the electronic control unit 20.

[0048] The electronic control unit 20 is capable of: receiving an image sequence of the driver 5's position from the first camera 10; calculating a correlation coefficient by convolving each of the plurality of masks M with at least one image in the image sequence received from the first camera 10 (i.e., applying a convolution filter, which is known in itself); determining the driver 5's posture based on the mask M with the highest correlation coefficient; and generating a warning to the driver 5 to correct his posture when the detected posture corresponds to an incorrect posture of the driver 5—i.e., when the driver 5 is too far from or too close to the steering wheel, for example, when his chest or head is less than 15 cm from the steering wheel or more than 130 cm from the steering wheel, respectively. Preferably, the warning continues to operate as long as the driver 5 does not return to the correct posture.

[0049] In this non-limiting example, the electronic control unit 20 is capable of: detecting a collision risk between the vehicle 1 and external factors in an image sequence received from the second camera 40; and generating a warning to the driver 5 when a collision risk is confirmed and the detected posture corresponds to an incorrect posture. External factors may be, for example, another vehicle, a pedestrian, or road infrastructure factors.

[0050] In this embodiment where vehicle 1 includes an airbag system 30 (safety inflatable air cushion system), electronic control unit 20 is capable of: deactivating the airbag system 30 when the driver 5 is in a second incorrect posture, i.e., too close to the steering wheel, preferably only deactivating the steering wheel airbag to prevent injury to the driver upon activation of the steering wheel airbag; and then reactivating the airbag or system once the driver 5's posture is detected by electronic control unit 20 as correct again.

[0051] The electronic control unit 20 includes a processor that is capable of implementing a set of instructions that enable the execution of these functions.

[0052] Now refer to Figures 2 to 6 The invention is described in one embodiment.

[0053] First, in the preliminary initialization step E0 performed, for example in a factory or laboratory, the processing mask M is determined and then recorded in the storage area of ​​the electronic control unit. To this end, an accurate vehicle driving configuration is reproduced, specifically by placing the same driver's seat, steering wheel, and camera as will be installed in the vehicle, while adhering to the distances between these components as defined in the vehicle.

[0054] Next, a model driver, preferably a human or a dummy, is positioned in various postures to define a corresponding mask M based on various image sequences. In this preferred example, the model driver is initially positioned in the seat in a "correct" posture, neither too far nor too close to the steering wheel, a posture considered safe for driving, especially in the event of a collision requiring the airbag system to be triggered. The camera then generates an image sequence, such as 25 images, which at least show the chest and head of the model driver sitting in this correct posture. The model driver is then positioned in a so-called first "incorrect" posture—too far from the steering wheel—a posture considered dangerous for driving, especially in the event of a collision requiring the airbag system to be triggered. The camera then generates an image sequence, such as 25 images, which at least show the chest and head of the model driver sitting in this first incorrect posture. The model driver is then positioned in a second incorrect posture—too close to the steering wheel—a posture considered dangerous for driving, especially in the event of a collision requiring the airbag system to be triggered. The camera then generates a sequence of images, such as 25 images, that at least show the chest and head of the model driver sitting in their seat in the second incorrect posture. Finally, in this example, the camera also generates a sequence of images, such as 25 images, showing the seat without the model driver. Each of the four image sequences will enable the definition of a mask M for image processing.

[0055] In this example, each of the four image sequences undergoes the following steps.

[0056] First, histogram normalization is performed on each of the four image sequences. Histogram normalization expands the range of values ​​for each image in the sequence by uniformly spreading the gray levels across the entire range of available values. In other words, this operation allows for a better distribution of pixel intensity across the entire range of possible values, thus adjusting the contrast of each image in the sequence. The four normalized image sequences are then recorded on a storage medium.

[0057] Next, a median filter is applied to the images in each of the four image sequences to filter out noise and outliers in pixel intensity. Median filtering allows for noise reduction while preserving image contours and involves replacing each input value with the median of its neighborhood.

[0058] The images in each of the four image sequences are then segmented. This segmentation involves selecting pixels whose intensity falls between a minimum and a maximum threshold, and modifying their intensity to equal the maximum intensity, for example, 255. This operation allows pixels that are neither too dark nor too bright to be distinguished from pixels that are too dark or too bright by making them appear as the brightest pixels (in other words, the lightest pixels).

[0059] Next, all convex hulls are detected on the images in each of the four image sequences, and the largest convex hull is retained to form the image region. This is based on the principle that the largest convex hull represents the outline of the model driver's body when the model driver's body is visible in the image, or the largest convex hull represents the outline of the seat when the model driver is not in the seat.

[0060] Finally, in the images of the four normalized image sequences recorded on the storage medium, the intensity of all pixels located outside the region formed in the previous step is reduced to zero (the darkest intensity) to obtain four masks M in which the model driver or seat is clearly visible and can be identified in the corresponding four poses.

[0061] Figure 3 An example photo of a mask corresponding to the correct pose is shown. Figure 4 An example photo of a mask corresponding to the first incorrect posture (driver 5 is too far from the steering wheel) is shown. Figure 5 An example photo of a mask corresponding to the second incorrect posture (driver 5 is too close to the steering wheel) is shown. Figure 6 An example photo of a mask corresponding to a pose without driver 5 is shown.

[0062] After performing the preparatory step E0, the four masks M are stored in the storage area 21 of the electronic control unit 20 so that they can be used when the vehicle 1 is driven in the future.

[0063] Therefore, during the usage phase, in step E1, when the vehicle is in motion, the first camera 10 generates a sequence of images showing the position of the driver 5 in his seat and sends them to the electronic control unit 20.

[0064] In step E2, the electronic control unit 20 receives an image sequence of the driver 5's position from the first camera 10, and calculates the convolution of each of the four masks M recorded in its storage area 21 with at least one image from the image sequence received from the first camera 10 to obtain a correlation coefficient CC ranging from 0 to 1.

[0065] Then, in step E3, the electronic control unit 20 determines the driver 5's posture P as the posture associated with the mask M with the highest correlation coefficient, which is the mask M with the highest probability of corresponding to the driver 5's actual posture.

[0066] In step E4, when the detected posture corresponds to one in which driver 5 is considered too far away (first incorrect posture), Figure 4 Or too close (second incorrect posture) Figure 5 When the mask M is in the correct posture, the electronic control unit 20 generates a warning A for the driver 5 to correct his posture. This warning A can be, for example, an audible warning, a visual warning, or a tactile warning (vibration, etc.). Preferably, as long as the driver 5 does not return to the correct posture (…), the warning A will be issued. Figure 3 Warning A continues to work.

[0067] In the method according to the invention, a second camera 40 may optionally be used. When the second camera 40 is used, it also generates an image sequence while the vehicle 1 is in motion and sends it to the electronic control unit 20. This image sequence represents the surrounding environment of the vehicle 1, preferably the area in front of the vehicle 1 (road and surrounding area). Upon receiving this image sequence, the electronic control unit 20 detects external factors (other vehicles, pedestrians, road infrastructure factors, etc.) in the image sequence received from the second camera 40 and assesses the risk of collision between the vehicle 1 and said external factors. Since such collision risk detection is known in itself, it will not be described in detail here. In this example, the risk of collision with external factors can be low or high. Furthermore, when the detected collision risk is high and the detected posture corresponds to a correct posture, the electronic control unit 20 does not trigger warning A to the driver 5 because the collision hazard is not directly related. Similarly, in this example, when the collision risk is assessed as low, warning A is not triggered regardless of the driver 5's posture (correct or incorrect). Conversely, when the collision risk is high and the detected posture corresponds to an incorrect posture ( Figure 4 or Figure 5 This triggers warning A.

[0068] There is no driver in the seat (empty seat). Figure 6 This can be used to avoid false detections, that is, to avoid detecting the correct posture when the driver 5 is in an incorrect posture, particularly being too close to the steering wheel, in which case the seat appears to be partially empty in the image. In fact, in this case, the mask M corresponding to the posture without the driver 5 (completely empty seat) makes it possible to reliably distinguish between empty seats and partially empty seats.

[0069] In the case of autonomous vehicle 1, electronic control unit 20 can be programmed to detect that vehicle 1 is driving in autonomous driving mode, and in this case, avoid detecting that driver 5 is in an incorrect posture. Figure 4 or Figure 5 A warning will be generated when the driver is in autonomous vehicle 1. The driver's correct posture can be more varied, such as sleeping, resting, or reading postures.

[0070] When vehicle 1 includes an airbag system (inflatable safety cushion system), electronic control unit 20 can detect when driver 5 is in a second incorrect posture, i.e., too close to the steering wheel. Figure 5 When the driver 5 is in the correct posture, the airbag system 30 is deactivated, preferably the steering wheel airbag, to prevent injury from the steering wheel airbag triggering. Then, the airbag is reactivated once the driver 5 is in the correct posture again.

[0071] Therefore, the present invention advantageously enables the vehicle driver 5 to be warned when the vehicle driver 5 is in an incorrect posture and poses a danger to his safety while driving the vehicle (especially in the case of a collision).

Claims

1. A method for determining the posture of a driver (5) of a vehicle (1), the vehicle (1) comprising a camera capable of generating an image sequence of the position of the driver (5) of the vehicle (1) and an electronic control unit (20) including a storage area (21), the method comprising the steps of: A preparatory step for determining multiple masks for image processing includes generating an image sequence for each predetermined posture of the driver (5) in his seat and determining a mask for each generated image sequence, wherein each of the multiple masks for image processing is associated with a predetermined posture of the driver (5) in his seat, including a correct posture of the driver (5) in his seat and at least one incorrect posture of the driver (5) in his seat; Each of the plurality of masks used for image processing is recorded in the storage area (21); The camera (10) generates an image sequence of the driver's (5) position and sends the image sequence to the electronic control unit (20). The electronic control unit (20) calculates the convolution of each of the plurality of masks for image processing recorded in the storage area (21) with at least one image from the image sequence received from the camera (10) to obtain the correlation coefficient (CC). The driver’s (5) posture is determined based on the mask with the highest correlation coefficient (CC).

2. The method according to claim 1, further comprising the step (E4): generating a warning to the driver (5) when the determined posture corresponds to an incorrect posture of the driver (5) so that he corrects the posture.

3. The method according to claim 1, wherein, The camera (10) is a 2D camera.

4. The method according to any one of claims 1 to 3, wherein, The pre-defined postures include the first incorrect posture where the driver (5) is too far from the steering wheel and the second incorrect posture where the driver (5) is too close to the steering wheel.

5. The method according to claim 2, wherein, The vehicle (1) also includes a second camera (40) which is capable of generating an image sequence of the external environment of the vehicle (1) and sending the image sequence to the electronic control unit (20); and the electronic control unit (20) is configured to detect the collision risk of the vehicle (1) with external factors in the received image sequence, and when a collision risk is confirmed and the detected posture corresponds to an incorrect posture, it is capable of generating a warning to the driver (5).

6. The method according to claim 1, wherein, Determining a mask for each generated image sequence includes determining pixels in the generated image sequence that correspond to the body of the driver (5).

7. A computer program product, characterized in that... It includes a set of program code instructions that, when executed by one or more processors, configure the one or more processors to perform the method according to any one of the preceding claims.

8. An electronic control unit (20) for a vehicle (1), the vehicle (1) including a camera (10) capable of generating an image sequence of the position of the driver (5) of the vehicle, the electronic control unit (20) including a storage area (21), the electronic control unit (20) being configured to: Multiple masks for image processing are determined by generating an image sequence for each predetermined posture of the driver (5) in his seat and determining a mask for each generated image sequence, each of the multiple masks for image processing being associated with a predetermined posture of the driver (5) in his seat, including a correct posture of the driver (5) in his seat and at least one incorrect posture of the driver (5) in his seat; Each of the plurality of masks used for image processing is recorded in the storage area (21); Receive a sequence of images of the driver's (5) position from the camera (10). For each of the plurality of masks for image processing recorded in the storage area (21), the convolution of the mask with at least one image in the image sequence received from the camera (10) is calculated to obtain a correlation coefficient (CC). - The driver's (5) posture is determined based on the mask with the highest correlation coefficient (CC).

9. A vehicle (1) comprising a camera (10) capable of generating an image sequence of the position of the driver (5) of the vehicle (1) and an electronic control unit (20) according to claim 8.

Images