Method for displaying extended images, system, computer program and associated vehicle

Abstract

Method for displaying extended images, system, computer program and associated vehicle. The present invention relates to a method for displaying extended images of the lower front end of a vehicle (2) having a front hood (6), each extended image (10) comprising a visible area (12) by a user (4) of the vehicle (2) and an area masked (14) by the front hood of the vehicle, said vehicle (2) being equipped with at least one external camera (8) having a field of vision oriented towards the lower front end of said vehicle, comprising steps of: - estimation of an eye position of the vehicle user, - processing of images captured by said at least one external camera to obtain extended images of the lower front end of the vehicle, implementing a superimposition, on each captured image, of a representation of a contour (16) of the front hood (6) of the vehicle, said representation being made according to said eye position of the vehicle user,and display of said extended images on an on-board display device. Figure for the abbreviation: Figure 1,

Description

Title of the invention: Method for displaying extended images, system, computer program and associated vehicle

[0001] The present invention relates to a method for displaying extended images of a lower front end of a vehicle having a front hood, each extended image comprising an area visible to a user of the vehicle and an area masked by the front hood of the vehicle.

[0002] The invention also relates to a system for displaying extended images of a low front end of an associated vehicle, an associated computer program and an associated vehicle.

[0003] The invention is situated in the field of vehicles, in particular motor vehicles, equipped with improved functionalities, in particular to increase safety and user satisfaction.

[0004] More and more motor vehicles are equipped with external cameras, which provide driving assistance, in particular to view hidden areas or blind spots, thus making it possible to avoid collisions or to avoid damaging the motor vehicle during a maneuver.

[0005] In particular, it is well known that motor vehicles are equipped with a front hood, which is located at the lower front of the vehicle and blocks the visibility of an area at the lower front of the vehicle, which is thus obscured. Such an obscured area prevents a vehicle user, for example the driver, from seeing low obstacles, whether they are fixed objects or moving obstacles, such as small animals. Depending on the circumstances, this lack of visibility can lead to collisions or accidents.

[0006] There is therefore a need to improve the visibility of areas located at the lower front of the motor vehicle, in particular areas masked by the presence of the front hood.

[0007] A solution to this problem is known, consisting of displaying images captured by an external camera placed outside the vehicle, whose field of view captures the lower front of the vehicle. However, the simultaneous display of the actual image of the lower front of the vehicle, including the area visible only to the user, and images captured by an external camera requires effort from the user to reconcile the two views, and is therefore not always satisfactory in terms of comfort and reducing cognitive load.

[0008] Complex image processing solutions are also known, implementing image storage of images captured by several external cameras to calculate images of the masked areas. Such solutions require adding hardware components, including processing units and memory, are therefore expensive.

[0009] The aim of the invention is therefore to propose a method and a system for displaying extended images allowing the user to view an area hidden by the front hood of the vehicle, which is less consuming of material resources while providing comfort and an improved user experience for the user.

[0010] To this end, the invention relates to a method for displaying extended images of the lower front end of a vehicle having a front hood, each extended image comprising an area visible to a vehicle user and an area masked by the vehicle's front hood, said vehicle being equipped with at least one external camera having a field of view directed towards the lower front end of said vehicle, and including said visible and masked areas, the method being implemented by a processing unit of an embedded programmable electronic device connected to said at least one external camera. This method comprises the steps of: - estimation of the vehicle user's eye position, - processing of images captured by said at least one outdoor camera to obtain extended images of the lower front of the vehicle, implementing a superimposition, on each image captured by said at least one external camera, of a representation of an outline of the front hood of the vehicle, said representation being made according to said position of the eyes of the vehicle user, and - display of said extended images on an on-board display device.

[0011] Advantageously, the proposed method requires only one or more external cameras located at the lower front of the vehicle and a conventional computing processor, thus eliminating the need for additional expensive hardware resources. Furthermore, advantageously, superimposing a representation of the vehicle's front hood contour based on the user's eye position helps the user visualize the boundary between the area hidden by the hood and the visible area, thereby enabling them to orient themselves spatially and understand the environment without any particular cognitive effort. This improves user comfort and experience.

[0012] According to other advantageous aspects of the invention, the method for displaying extended images of a low front end of a vehicle comprises one or more of the following features, taken individually or in all technically possible combinations.

[0013] The processing implements a calculation of said representation of the outline of the front hood of the vehicle using further information, previously recorded relating to the model of the vehicle.

[0014] The user's eye position is predetermined and previously recorded.

[0015] The method is implemented in a vehicle further comprising an interior camera directed towards the interior of the vehicle, the estimation of a user's eye position comprising a determination of the user's eye position using images captured by said interior camera.

[0016] The method further includes a step of correcting said extended images to obtain corrected extended images before the display step, the correction step being configured to transform said field of view of said at least one camera into a user's field of view according to said user's eye position.

[0017] The invention also relates to a system for displaying extended images of the lower front of a vehicle having a front hood, each extended image comprising an area visible to a user of the vehicle and an area masked by the front hood of the vehicle, said vehicle being equipped with at least one external camera having a field of view directed towards the lower front of said vehicle, and including said visible area and masked area, the display system comprising said at least one external camera, an embedded programmable electronic device connected to said at least one external camera, the embedded programmable electronic device comprising a processing unit and an embedded display device. This system is such that the processing unit is configured to implement: - a module for estimating the eye position of the vehicle user, - an image processing module for images captured by said at least one camera to obtain extended images of the lower front of the vehicle, implementing a superimposition, on each image captured by said at least one camera, of a representation of an outline of the front hood of the vehicle, said representation being made according to said position of the eyes of the vehicle user, - a display module on an embedded display device for said extended images.

[0018] According to one feature, the processing module implements a calculation of said representation of the outline of the front hood of the vehicle, further using previously recorded information relating to the vehicle model.

[0019] According to one feature, the system further comprises an interior camera directed towards the interior of the vehicle, the user eye position estimation module implementing a determination of the user's eye position using images captured by said interior camera.

[0020] According to one feature, the on-board display device is a display screen located on a dashboard of said vehicle.

[0021] The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a method for displaying extended images of a low front end of a vehicle as defined above.

[0022] The invention also relates to a vehicle comprising a system for displaying extended images of a low front of a vehicle as defined above.

[0023] The invention will become clearer upon reading the following description, given solely by way of non-limiting example, and made with reference to the drawings in which:

[0024] [Fig-1] [Fig.1] is a schematic illustration of a motor vehicle equipped of an extended image display system;

[0025] [Fig.2] a schematic representation of an image display system extended according to a mode of embodiment;

[0026] [Fig.3] [Fig.3] is a synoptic diagram of the main steps in a display process of extended images according to a method of realization.

[0027] A motor vehicle 2 and a user 4 of the motor vehicle, for example the driver of the motor vehicle, are schematically represented in [Fig.1].

[0028] In the rest of the description, reference will be made to vehicle 2.

[0029] A three-dimensional reference frame R, consisting of three axes (X, Y, Z) perpendicular to each other, is associated with vehicle 2. The eyes of user 4 are located at a height hi in the reference frame R, where hi is considered to be the position of user 4's eyes, allowing us to define a direction D delimiting a spatial area Pi visible to the user at the lower front of the vehicle. Indeed, the visible area Pi is delimited by the front hood 6 of vehicle 2. In other words, the direction D delimits the user's field of vision. The direction D is, for example, represented by a vector in the three-dimensional reference frame R.

[0030] Another spatial area P2, also located at the lower front of vehicle 2, is an area masked by the front hood 6 of vehicle 2.

[0031] The vehicle 2 is further equipped with an external camera 8, or several external cameras not shown in the figures, having a field of vision directed towards the lower front of the vehicle. The external camera 8 is adapted to capture images including an area visible 12 to the user 4 (corresponding to the spatial area PI in the captured image) and an area masked 14 by the front hood 6 (corresponding to the spatial area P2 in the captured image).

[0032] Furthermore, advantageously, the invention provides for the display of extended images 10, formed from the images captured by the external camera 8 and also comprising, superimposed, a representation of the contour 16 of the front hood 6 of the vehicle 2. This representation is a function of the user's eye position, and by as a consequence of direction D. In other words, the contour 16 of the hood shown superimposed on each extended image 10 is a representation of the contour of the front hood 6 as seen by the user 4 of the vehicle 2. As an example, the contour 16 is represented by a smooth curve with a color, preferably with a green color, and the width of the smooth curve is predefined or modifiable by the user.

[0033] Fig. 2 schematically illustrates a system 20 for displaying extended images of a low front end of a vehicle 2.

[0034] The system 20 is embedded in a vehicle, for example a motor vehicle.

[0035] The system 20 comprises a first camera 8, which is an external camera, having a field of vision directed towards the lower front of the vehicle 2.

[0036] According to variants, several external cameras 8 are positioned at the lower front of the vehicle 2, so as to allow an expanded field of vision to be captured by combining the fields of vision of each external camera.

[0037] The outdoor camera 8 is connected to an embedded programmable electronic device 22 forming part of the system 20.

[0038] The programmable electronic device 22 includes or is connected to an on-board display device 24.

[0039] For example, the display device 24 is a display screen located on a dashboard (not shown) of the vehicle 2.

[0040] Optionally, the vehicle 2 also includes an interior camera 26, positioned inside the vehicle's passenger compartment and directed towards the interior of the vehicle's passenger compartment.

[0041] The interior camera 26 is associated with or includes a unit for determining the eye position of a user 4 of the vehicle 2. Thus, the position of the user's eyes can be determined in real time and this eye position information is used to dynamically determine the position of the vehicle's contour.

[0042] For example, the interior camera 26 is part of a DMS (for “Driver Monitoring Systems”) device, connected to the on-board programmable electronic device 22.

[0043] The embedded programmable electronic device 22 is for example an embedded electronic computing unit ECU, and includes at least one computing processor 28 and an electronic memory unit 30.

[0044] The programmable electronic device 22 includes: - a module 32 for estimating the eye position of the vehicle user, - a 34-channel image processing module for images captured by the outdoor camera 8 to obtain extended images of the lower front of the vehicle, implementing a superimposition, on each image captured by said camera, of a representation of the outline of the vehicle's front hood, the representation being created based on the position of the vehicle user's eyes. - a module 36 for displaying extended images on the on-board display device 24.

[0045] For example, the position of the driver's eyes is expressed by a height in the spatial reference frame R associated with the vehicle.

[0046] In the case where an indoor camera 26 is present, the module 32 is configured to estimate a position of the user's eyes from images provided by the indoor camera 26.

[0047] Preferably, the electronic memory unit 30 is configured to store information 38 relating to the model of the vehicle 2, this information 38 including information relating to the shape of the vehicle 2, this information being particularly useful for generating the outline of the front hood of the vehicle which is represented in the extended images.

[0048] Preferably, the display of the extended images is carried out substantially in real time following the capture of the images by the external camera 8, advantageously allowing the user 4 to view in real time the extended images of the lower front of the vehicle, including the masked area.

[0049] The computing processor 28 is configured to implement modules 32, 34, 36 when the programmable electronic device 22 is powered on.

[0050] In one embodiment, modules 32, 34, 36 are implemented in the form of software instructions forming a computer program, which, when executed by a computer, implements a method for displaying extended images according to the invention.

[0051] The computer program comprising software instructions is further capable of being recorded on a non-transient, computer-readable medium. The computer-readable medium is, for example, a medium capable of storing electronic instructions and being connected to a bus of a computer system. For example, the readable medium is an optical disc, a magneto-optical disc, a ROM, a RAM, any type of non-volatile memory (e.g., EPROM, EEPROM, FLASH, NVRAM), a magnetic card, or an optical card.

[0052] In an alternative not shown, modules 32, 34, 36 are each implemented as programmable logic components, such as FPGAs (Field Programmable Gate Arrays), microprocessors, GPGPUs (General-purpose processing on graphics processing), or dedicated integrated circuits, such as ASICs (Application-Specific Integrated Circuits).

[0053] Figure 3 is a synoptic diagram of the main steps in one embodiment of the method for displaying extended images of the lower front of a vehicle implemented by the extended image display system 20.

[0054] The method includes a step 40 of estimating the position of the eyes of the vehicle user.

[0055] The position of the eyes of the vehicle is preferably expressed in a three-dimensional reference frame associated with the vehicle, for example the reference frame R.

[0056] According to a first embodiment, the eye position is a fixed position, previously recorded. For example, the fixed position is an average position estimated for all users, making a fixed angle between the projection direction D and the ground, the fixed angle being a predefined value between 20° and 45° depending on the vehicle model.

[0057] In other words, the estimated eye position is fixed in this embodiment for a given vehicle model, and it is read into a memory at step 40.

[0058] According to a second variant, when the vehicle is equipped with an interior camera, the estimated eye position is calculated from images captured by the interior camera.

[0059] The estimated eye position then allows the projection to be calculated, the direction D delimiting the portion of the lower front of the vehicle visible to the user, the direction D being a function of the position of the user's eyes, as can be clearly seen from the representation of [Fig.1].

[0060] The process then includes a step 42 of processing the images captured by the external camera.

[0061] The processing involves, for each image captured by the external camera, superimposing an outline of the vehicle's front hood. The outline representation depends on the driver's eye position, and more specifically on the direction D that delimits the portion of the lower front of the vehicle visible to the user. Indeed, the outline representation is positioned according to the projection along direction D in the image captured by the external camera.

[0062] Preferably, the shape of the hood contour is calculated based on information relating to the vehicle model, so as to realistically represent a silhouette of the vehicle 2.

[0063] For example, contour shapes of the hood are pre-calculated and recorded, depending on the vehicle model and the user's eye positions.

[0064] Advantageously, step 42 is fast and consumes few computing resources.

[0065] For example, the contour representation is drawn, for example as a colored line, on the image captured by the external camera. Preferably, the colored line is a smooth curve with a green color or with several colors, and the The width of the smooth curve is predefined according to the vehicle model or can be modified by the user.

[0066] Advantageously, the contour thus represented allows the user to easily and without cognitive effort determine which part of the extended image includes the area masked by the hood of the vehicle, in a realistic manner.

[0067] According to an optional variant, processing 42 further includes registration, enabling the acquisition, from initial images obtained by the external camera, of second images recentered with respect to the user's eye position. The extended images are then obtained by projecting the outline of the vehicle's front hood onto the second recentered images. Advantageously, the extended images thus obtained correspond to the user's eye position.

[0068] The method then includes a step 44 of displaying the extended image thus obtained on a chosen on-board display device.

[0069] Optionally, the method includes a correction step 43 of the extended images to obtain corrected extended images before the display step 44, the correction step 43 being configured to transform said field of view of said at least one camera into a user field of view according to said user eye position.

[0070] Advantageously, the proposed extended image display method consumes few computing resources but nevertheless provides a realistic and informative representation for the vehicle user. Driving safety and comfort are thus improved.

Claims

Demands

1. A method for displaying extended images of the lower front end of a vehicle (2) having a front hood (6), each extended image (10) comprising a visible area (12) by a user (4) of the vehicle (2) and a masked area (14) by the front hood of the vehicle, said vehicle (2) being equipped with at least one external camera (8) having a field of view oriented towards the lower front end of said vehicle, and including said visible area (12) and masked area (14), the method being implemented by a computing processor (28) of an embedded programmable electronic device (22) connected to said at least one external camera (8), and characterized in that it comprises steps of: - estimating (40) a position of the eyes of the vehicle user, and calculating a projection direction (D) delimiting a portion of the lower front end of the vehicle visible to the user, the direction (D) being a function of the user's eye position,- processing (42) of images captured by said at least one external camera to obtain extended images of the lower front of the vehicle, implementing a superimposition, on each image captured by said at least one external camera, of a representation of a contour (16) of the front hood (6) of the vehicle, said representation being made according to said projection direction (D) delimiting the portion of the lower front of the vehicle visible to the user, - display (44) on an on-board display device (24) of said extended images.

2. A method according to claim 1, wherein said processing (42) implements a calculation of said representation of the contour (16) of the front hood of the vehicle further using previously recorded information (38) relating to the vehicle model.

3. A method according to any one of claims 1 or 2, wherein the position of the user's eyes is predetermined and previously recorded.

4. A method according to claim 1 or 2, implemented in a vehicle (2) further comprising an interior camera (26) directed towards an interior of the vehicle (2), the estimation (40) of a position of the user's eyes including a determination of the user's eye position using images captured by said internal camera.

5. A method according to any one of claims 1 to 4, comprising a correction step (43) of said extended images to obtain corrected extended images before the display step (44), the correction step (43) being configured to transform said field of view of said at least one camera into a user field of view according to said user eye position.

6. A computer program comprising software instructions which, when executed by a programmable electronic device, implement a method for displaying extended images of a lower front end of a vehicle in accordance with claims 1 O / 1

7. ia 4. A system (20) for displaying extended images of the lower front of a vehicle (2) having a front hood (6), each extended image (10) comprising an area visible (12) to a user of the vehicle and an area masked (14) by the front hood of the vehicle, said vehicle (2) being equipped with at least one exterior camera (8) having a field of view directed towards the lower front of said vehicle, and including said visible area (12) and masked area (14), the display system (20) comprising said at least one exterior camera (8), an on-board programmable electronic device (22) connected to said at least one exterior camera (8), the on-board programmable electronic device (22) comprising a computing processor (28) and an on-board display device (24),the system being characterized in that the computing processor is configured to implement: - a module (32) for estimating the user's eye position in the vehicle and calculating a projection direction (D) delimiting a portion of the lower front of the vehicle visible to the user, the direction (D) being a function of the user's eye position, - a module (34) for processing images captured by said at least one camera to obtain extended images of the lower front of the vehicle, implementing a superimposition, on each image captured by said,

8.

9.

10.

11. at least one camera, of a representation of a contour (16) of the front hood of the vehicle, said representation being made according to said direction (D) of projection delimiting the portion of the lower front of the vehicle visible to the user, - a display module (36) on an on-board display device for said extended images. System according to claim 7, wherein said processing module (34) implements a calculation of said representation of the contour (16) of the front hood of the vehicle further using previously recorded information (38) relating to the model of the vehicle. System according to any one of claims 7 or 8, further comprising an interior camera (26) directed towards an interior of the vehicle, the user eye position estimation module (32) implementing a determination of the user's eye position using images captured by said interior camera (26). System according to any one of claims 7 to 9, wherein said on-board display device (24) is a display screen disposed on a dashboard of said vehicle. Vehicle (2) comprising a system for displaying extended images of the lower front of said vehicle according to any one of claims 7 to 10.

Images