METHOD FOR DISPLAYING ENHANCED IMAGES, SYSTEM, COMPUTER PROGRAM AND VEHICLE THEREFOR

Abstract

The present invention relates to a method for displaying enhanced images of a lower front end of a vehicle (2) having a front hood (6), each enhanced image (10) comprising an area (12) visible to a user (4) of the vehicle (2) and an area (14) obscured by the front hood of the vehicle, the vehicle (2) being equipped with at least one external camera (8) having a field of view directed towards the lower front end of the vehicle, comprising the steps of: - Estimating a position of the eyes of the vehicle user, - processing images captured by the at least one exterior camera to obtain enhanced images of the lower front end of the vehicle, wherein a representation of an outline (16) of the front hood (6) of the vehicle is superimposed onto each captured image, the representation being created as a function of the position of the eyes of the user of the vehicle, and displaying the enhanced images on an in-vehicle display device.

Description

The present invention relates to a method for displaying augmented images of a lower front end of a vehicle having a front hood, each augmented image including an area visible to a user of the vehicle and an area occluded by the front hood of the vehicle.The invention also relates to a system for displaying augmented images of a lower front end of an associated vehicle, an associated computer program and an associated vehicle.The invention is in the field of vehicles, particularly motor vehicles, equipped with enhanced functionalities, particularly to increase safety and user satisfaction.More and more motor vehicles are equipped with external cameras which can provide a driving aid, in particular by viewing hidden regions or blind spots, and thus contribute to avoiding collisions or preventing damage to the motor vehicle during operation.In particular, as is known, motor vehicles are equipped with a front hood, which is located for this purpose at the lower front end of the vehicle and blocks the view of an area at the lower front end of the vehicle, which is thus concealed. Such a hidden area obstructs the view of a vehicle user, e.g. the driver, on low obstacles, whether stationary objects or moving obstacles, e.g. small animals. This lack of visibility can lead to collisions or accidents depending on the situation.There is therefore a need to improve the visibility of the areas at the lower front end of the motor vehicle, in particular of the areas which are covered by the presence of the front hood.A known solution to this problem is to display images captured by an exterior camera placed outside the vehicle, the field of view of which captures the lower front end of the vehicle. However, simultaneously viewing the real image of the lower front end of the vehicle including the range visible only to the user and the images captured by an outside camera requires a lot of effort from the user to conform the two views, and is therefore not always satisfactory in terms of comfort and reduction of cognitive burden.Complex image processing solutions are also known, including storing images captured by multiple exterior cameras to calculate the images from occluded areas. Such solutions require additional hardware components such as computing processors and memory and are therefore costly.It is an object of the invention to provide an augmented image display method and system that enables the user to view an area occluded by the front hood of the vehicle, thereby consuming less hardware resources and simultaneously providing comfort and an improved user experience.To this end, the object of the invention is to provide a method for displaying augmented images of a lower front end of a vehicle having a front hood, each augmented image comprising an area visible to a user of the vehicle and an area concealed by the front hood of the vehicle, the vehicle being equipped with at least one exterior camera having a field of view, aligned with the lower front end of the vehicle, that includes the visible area and the concealed area, the method being implemented by a computing processor of an in-vehicle programmable electronic device connected to the at least one exterior camera. The system comprises the steps of:estimating a position of the eyes of the user of the vehicle,processing images captured by the at least one exterior camera in order to obtain enhanced images of the lower front end of the vehicle, wherein a representation of an outline of the front hood of the vehicle is overlaid on each image captured by the at least one exterior camera, wherein the representation is created depending on the position of the eyes of the user of the vehicle, anddisplaying the expanded images on an in-vehicle display device.Advantageously, the proposed process requires only one or more exterior cameras located at the lower front end of the vehicle and a conventional computing processor, thereby not requiring additional costly hardware resources. In addition, the blending of a representation of the outline of the front hood of the vehicle according to the position of the user's eyes assists the user in visualizing the boundary between the area covered by the hood and the visible area and thus spatially orienting itself without particular cognitive effort and understanding the environment. This improves user convenience and experience.According to further advantageous aspects of the invention, the method for displaying augmented images of a lower front end of a vehicle comprises one or more of the following features, individually or in any technically feasible combination.The processing calculates the representation of the outline of the front hood of the vehicle, and thereby additionally uses previously stored information regarding the vehicle model.The position of the user's eyes is predetermined and recorded in advance.The method is implemented in a vehicle further comprising an interior camera directed toward the interior of the vehicle, wherein estimating a position of the user's eyes comprises determining the position of the user's eyes using images captured by the interior camera.The method further includes, before the display step, a step of correcting the augmented images to obtain corrected augmented images, wherein the correction step is configured to convert the field of view of the at least one camera into a field of view of a user according to the position of the eyes of the user.The invention further relates to a system for displaying augmented images of a lower front end of a vehicle having a front hood, each augmented image comprising an area visible to a user of the vehicle and an area occluded by the front hood of the vehicle, the vehicle being equipped with at least one exterior camera having a field of view aligned with the lower front end of the vehicle, including the visible area and the occluded area, the method being implemented by a computing processor of an in-vehicle programmable electronic device connected to the at least one exterior camera. This system is arranged such that the computing processor is configured to implement:a module for estimating a position of the eyes of the user of the vehicle,a module for processing images captured by the at least one camera to obtain augmented images of the lower front end of the vehicle, wherein a representation of an outline of the front hood of the vehicle is overlaid on each image captured by the at least one camera, wherein the representation is created depending on the position of the eyes of the user of the vehicle,a module for displaying the augmented images on an in-vehicle display device.According to one characteristic, the processing module calculates the representation of the outline of the front hood of the vehicle and additionally uses previously stored information relating to the vehicle model.According to one feature, the system further comprises an interior camera directed towards the interior of the vehicle, wherein the module for estimating a position of the eyes of the user implements a determination of the position of the eyes of the user using images captured by the interior camera.According to one feature, the in-vehicle display device is a display screen disposed on an instrument panel of the vehicle.The invention further relates to a computer program comprising software instructions which, when executed by a computer, implement a method for displaying augmented images of a lower front end of a vehicle as defined above.The invention further relates to a vehicle comprising a system for displaying augmented images of a lower front end of a vehicle as defined above.The invention will be better understood from reading the following description given by way of non-limiting example only and made with reference to the drawings, in which:[FIG. 1 ] [FIG. 1 ] is a schematic illustration of a motor vehicle equipped with an augmented image display system;[FIG. 2 ] [FIG. 2 ] is a schematic diagram of an augmented image display system according to an embodiment;[FIG. 3] [FIG. 3] is a diagram of the main steps of a method for displaying augmented images according to an embodiment.[FIG. 1 ] schematically shows a motor vehicle 2 and a user 4 of the motor vehicle, for example the driver of the motor vehicle.In the following description, reference is made to the vehicle 2.The vehicle 2 is assigned a three-dimensional reference frame R which consists of three perpendicular axes (X, Y, Z). The eyes of the user 4 are at a height hi in the reference frame R, the height hi being considered as the position of the eyes of the user 4, whereby a direction D defining a spatial region P 1 visible to the user at the lower front end of the vehicle can be defined. The visible area P 1 is defined by the front hood 6 of the vehicle 2. In other words, the direction D limits the user's angle of view. The direction D is represented by a vector in the three-dimensional reference frame R, for example.Another spatial area P 2, which is also located at the lower front end of the vehicle 2, is an area that is concealed by the front hood 6 of the vehicle 2.The vehicle 2 is also equipped with one or more exterior cameras 8, not shown in the figures, whose field of view is directed toward the lower front end. The exterior camera 8 is configured to capture images including an area 12 visible to the user 4 (corresponding to the spatial area P 1 in the captured image) and an area 14 hidden by the front cover 6 (corresponding to the spatial area P 2 in the captured image).In addition, the invention advantageously proposes the display of augmented images 10 which are formed from the images captured by the external camera 8 and additionally comprise an overlaid representation of an outline 16 of the front hood 6 of the vehicle 2. This representation depends on the position of the user's eyes and therefore the direction D. In other words, the outline 16 of the front hood that is overlaid on each augmented image 10 is a representation of the outline of the front hood 6 as seen by the user 4 of the vehicle 2. By way of example, the contour line 16 is represented by a smooth curve with a color, preferably with a green color, and the width of the smooth curve is predefined or can be changed by the user.[FIG. 2 ] schematically illustrates an augmented image display system 20 of a lower front end of a vehicle 2.The system 20 is installed in a vehicle, such as an automobile.The system 20 includes a first camera 8, which is an exterior camera, the field of view of which is directed toward the lower front end of the vehicle 2.Depending on the variant, a plurality of exterior cameras 8 are positioned at the lower front end of the vehicle 2 in order to capture a broader field of view by the combination of the fields of view of the individual exterior cameras.The exterior camera 8 is connected to an in-vehicle programmable electronic device 22 that is part of the system 20.The programmable electronic device 22 includes or is connected to an in-vehicle display device 24.For example, the display device 24 is a display screen disposed on an instrument panel (not shown) of the vehicle 2.Optionally, the vehicle 2 further includes an interior camera 26 positioned within the passenger compartment of the vehicle and oriented toward the interior of the passenger compartment of the vehicle.The interior camera 26 is assigned to or comprises a unit for determining the position of the eyes of a user 4 of the vehicle 2. In this way, the position of the user's eyes can be determined in real time and this eye position information is used to dynamically determine the contour line position of the vehicle.For example, the interior camera 26 is part of a driver monitoring system (DMS) connected to the in-vehicle programmable electronic device 22.The in-vehicle programmable electronic device 22 is, for example, an in-vehicle electronic computing unit (ECU), and includes at least one computing processor 28 and an electronic storage unit 30.The in-vehicle programmable electronic device 22 includes:a module 32 for estimating a position of the eyes of the user of the vehicle,a module 34 for processing images captured by the at least one exterior camera 8 to obtain augmented images of the lower front end of the vehicle, wherein a representation of an outline of the front hood of the vehicle is overlaid on each image captured by the camera, wherein the representation is created depending on the position of the eyes of the user of the vehicle,a augmented image display module 36 on the in-vehicle display device 24.For example, the position of the driver's eyes is expressed as the height in the spatial reference frame R associated with the vehicle.When an interior camera 26 is present, the module 32 is configured to estimate the position of the user's eyes from images provided by the interior camera 26.Preferably, the electronic storage unit 30 is configured to store information 38 relating to the model of the vehicle 2, this information 38 comprising in particular information relating to the shape of the vehicle 2, this information being particularly usable to generate the outline of the front hood of the vehicle, which is represented in the expanded images.Preferably, the augmented images are displayed substantially in real time after the capturing of the images by the exterior camera 8, whereby the user 4 can advantageously view the augmented images of the lower front end of the vehicle, including the occluded area, in real time.The computing processor 28 is configured to implement the modules 32, 34, 36 when the programmable electronic device 22 is booted up.In one embodiment, the modules 32, 34, 36 are embodied in the form of software instructions forming a computer program which, when executed by a computer, implements a method of displaying augmented images according to the invention.The computer program comprising software instructions may be further recorded on a non-transitory computer readable medium. The computer readable medium is, for example, a medium in which the electronic instructions are stored and can be connected to a bus of a computer system. As an example, the readable medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (e.g., EPROM, EEPROM, FLASH, NVRAM), a magnetic card, or an optical card.In a variation not shown, the modules 32, 34, 36 are each fabricated as programmable logic components, such as field programmable gate arrays (FPGA), microprocessors, general purpose processing on graphics processing (GPGPU) components, or dedicated integrated circuits, such as application specific integrated circuits (ASIC).[FIG. 3 ] is a diagram of the main steps of an embodiment of the method for displaying augmented images of the lower front end of a vehicle implemented by the augmented image display system 20.The method comprises a step 40 of estimating the position of the eyes of the user of the vehicle.The position of the eyes of the vehicle is preferably expressed in a three-dimensional reference frame associated with the vehicle, for example, the reference frame R.According to a first variant, the eye position is a fixed, prestored position. For example, the fixed position is an average position estimated for all users, which establishes a fixed angle between the projection direction D and the ground, the fixed angle being a predefined value between 20° and 45° depending on the vehicle model.In other words, the estimated eye position is fixed for a particular vehicle model in this embodiment and is read from a memory in step 40.According to a second variant, when the vehicle is equipped with an interior camera, the estimated eye position is calculated from images captured by the interior camera.The projection is then calculated from the estimated eye position, with the direction D delimiting the portion of the lower front end of the vehicle visible to the user and the direction D depending on the position of the user's eyes, as can be seen from the representation of [FIG. 1 ].The method then comprises a step 42 for processing the images captured by the external camera.The processing includes blending each image captured by the exterior camera with the outline of the front hood of the vehicle. The outline representation depends on the position of the driver's eyes and is in particular dependent on the direction D delimiting the part of the lower front end of the vehicle visible to the user. In fact, the outline representation along the projection in direction D is positioned within the image captured by the external camera.Preferably, the shape of the hood outline line is calculated based on vehicle model information to realistically present a silhouette of the vehicle 2.For example, the contour line shapes of the front hood are calculated and stored in advance based on the vehicle model and the position of the user's eyes.Advantageously, step 42 is fast and consumes only a few computational resources.For example, the outline image is drawn on the image captured by the outside camera, for example, as a colored line. Preferably, the colored line is a smooth curve having a green color or having multiple colors, wherein the width of the smooth curve is predefined or can be changed by the user depending on the vehicle model.Advantageously, the outline represented in this way allows the user to determine, simply and without cognitive effort, which part of the expanded image comprises the area hidden in truth by the front hood of the vehicle.According to an optional variant, the processing 42 additionally comprises a realignment making it possible to obtain, from first images captured by the external camera, second images which are recenter centered with respect to the position of the user's eyes. Augmented images are then obtained by displaying the outline of the front hood of the vehicle on the re-focused second images. Advantageously, the extended images thus obtained correspond to the position of the user's eyes.The method then includes a step 44 of displaying the extended image thus obtained on a selected in-vehicle display device.Optionally, the method further comprises a step of correcting 43 the augmented images to obtain corrected augmented images before the display step 44, wherein the correction step 43 is configured to convert the field of view of the at least one camera into a field of view of a user according to the position of the user's eyes.Advantageously, the proposed method of displaying augmented images consumes only a few computing resources, and yet provides a realistic and informative representation to the vehicle user. This improves safety and ride comfort.

Claims

Method for displaying augmented images of a lower front end of a vehicle (2) having a front hood (6), each augmented image (10) comprising an area (12) visible to a user (4) of the vehicle (2) and an area (14) concealed by the front hood of the vehicle, the vehicle (2) being equipped with at least one exterior camera (8) having a field of view aligned with the lower front end of the vehicle, which includes the visible area (12) and the concealed area (14), the method being implemented by a computer processor (28) of an in-vehicle programmable electronic device (22) connected to the at least one exterior camera (8), and characterised in that it comprises the steps of: - estimating (40) a position of the eyes of the user of the vehicle, processing (42) images captured by the at least one exterior camera in order to obtain augmented images of the lower front end of the vehicle, wherein a representation of an outline (16) of the front hood (6) of the vehicle is overlaid on each image captured by the at least one exterior camera, wherein the representation is created depending on the position of the eyes of the user of the vehicle, - displaying (44) the augmented images on an in-vehicle display device (24).The method of claim 1, wherein the processing (42) implements a calculation of the representation of the vehicle front hood outline (16) using additional pre-stored information (38) relating to the vehicle model.The method according to any one of claims 1 or 2, wherein the position of the user's eyes is predetermined and recorded beforehand.The method according to any one of claims 1 or 2, implemented in a vehicle (2), further comprising an interior camera (26) directed towards the interior of the vehicle (2), wherein estimating (40) a position of the user's eyes comprises determining the position of the user's eyes using images captured by the interior camera.The method according to any one of claims 1 to 4, comprising, before the display step (44), a step of correcting (43) the augmented images to obtain corrected augmented images, wherein the correction step (43) is configured to convert the field of view of the at least one camera into a field of view of a user according to the position of the eyes of the user.A computer program comprising software instructions which, when executed by a programmable electronic device, implement a method for estimating a distance between a motor vehicle and an external object according to claims 1 to 4.A system (20) for displaying augmented images of a lower front end of a vehicle (2) having a front hood (6), each augmented image (10) comprising an area (12) visible to a user of the vehicle and an area (14) occluded by the front hood of the vehicle, the vehicle (2) being equipped with at least one exterior camera (8) having a field of view aligned with the lower front end of the vehicle including the visible area (12) and the occluded area (14), the display system (20) comprising at least one exterior camera (8), an in-vehicle programmable electronic device (22) connected to the at least one exterior camera (8), the in-vehicle programmable electronic device (22) comprising a computing processor (28) and an in-vehicle display device (24), the system characterized by, the computing processor being configured to implement: - a module (32) for estimating a position of the eyes of the user of the vehicle, - a module (34) for processing images captured by the at least one camera to obtain augmented images of the lower front end of the vehicle, wherein a representation of an outline (16) of the front hood of the vehicle is overlaid on each image captured by the at least one camera, wherein the representation is created depending on the position of the eyes of the user of the vehicle, - a module (36) for displaying the augmented images on an in-vehicle display device.The system of claim 7, wherein the processing module (34) implements a calculation of the representation of the vehicle front hood outline (16) using additional pre-stored information (38) relating to the vehicle model.The system of any of claims 7 or 8, further comprising an interior camera (26) directed toward the interior of the vehicle, wherein the module (32) for estimating a position of the user's eyes implements a determination of the position of the user's eyes using images captured by the interior camera (26).The system according to any one of claims 7 to 9, wherein the in-vehicle display device (24) is a display screen disposed on an instrument panel of the vehicle.A vehicle (2) comprising an augmented image display system of the lower front end of the vehicle according to any one of claims 7 to 10.

Images