Method for displaying, on a screen of a motor vehicle, an area hidden by a trailer hitched to the vehicle

Abstract

Disclosed is a method for displaying an area of interest, the vehicle comprising a first image sensor and at least one second image sensor, the area of interest (ZI) comprising an area called the masked area (ZM), which is masked by a trailer (200) hitched to the vehicle, the method comprising: • a first step (E1) of capturing, by means of the first sensor (1), a first image (IMG_1) of the area of interest (ZI), • a second step (E2) of capturing, by means of the second sensor (2), a second image (IMG_2) of the area of interest (ZI) comprising an image (IMG_ZM) of the masked area (ZM), then, • a third step of creating a corrected image (IMG_C) by integrating, into the first image (IMG_1), the image (IMG_ZM) of the masked area (ZM) obtained from the second image (IMG_2).

Description

[0001] DESCRIPTION

[0002] TITLE: Method of displaying, on a screen of a motor vehicle, an area hidden by a trailer attached to the vehicle.

[0003] The invention relates to a method for displaying, on a screen of a motor vehicle, an area hidden by a trailer coupled to the vehicle. The invention also relates to a device for displaying, on a screen of a motor vehicle, an area hidden by a trailer of the motor vehicle. The invention also relates to a computer program implementing the mentioned method. The invention finally relates to a recording medium on which such a program is recorded.

[0004] When a motor vehicle is equipped with a trailer, the trailer obstructs the field of view of the motor vehicle's rearview camera. The driver cannot therefore see the road markings located under or behind the trailer, and lacks reference points to better manage the parking of his vehicle equipped with the trailer.

[0005] The aim of the invention is to provide a method for displaying an area hidden by a trailer which is simple and reliable and which allows the driver of the vehicle to maneuver the trailer efficiently.

[0006] To this end, the invention relates to a method for displaying, on a screen arranged in a passenger compartment of a motor vehicle coupled to a trailer, an area of ​​interest located at the rear of the motor vehicle, the vehicle comprising a first image sensor, in particular a so-called reversing camera, and at least one second image sensor, in particular at least one exterior rearview camera, the area of ​​interest comprising an area said to be masked by the trailer.

[0007] Further, the method comprises

[0008] • a first step of capturing, by the first sensor, a first image of the area of ​​interest on which the trailer masks the masked area,

[0009] • a second step of capturing, by the second sensor, a second image of the area of ​​interest, the second image comprising an image of the masked area, then,

[0010] • a third step of creating a corrected image by integrating, into the first image, the image of the masked zone from the second image, then

[0011] • a fourth step of displaying the corrected image on the screen.

[0012] In one embodiment, the integration of the image of the masked area into the first image relates to elements of the area of ​​interest which are hidden by the trailer, said elements being located on the ground, in particular said elements being ground markings.

[0013] In one embodiment, the vehicle is equipped with two rearview cameras, a first focal distance of the two rearview cameras being small, in particular substantially equal to 1 millimeter, and

[0014] • the at least one second sensor is of a first type if the second capture step is carried out with the first focal length, and

[0015] • the at least one second sensor is of a second type if the second capture step is carried out with a second focal length of the rearview cameras, the second focal length being strictly greater than the first focal length, in particular the second focal length being substantially equal to 10 millimeters.

[0016] In one embodiment, the masked area comprises an area located under the trailer.

[0017] In one embodiment, an image of a first area located at a first given time under the trailer is constructed from at least one prior image of said first area captured at a second given time occurring before the trailer occludes said first area.

[0018] In one embodiment, the integration into the corrected image of the image of the masked area comprises a deletion of pixels from the second image representing sidewalks located at a distance from the motor vehicle and the trailer.

[0019] In one embodiment, the at least one second sensor is of the second type, and the third creation step comprises enlarging an image of an area behind the trailer and replacing in the corrected image pixels representing the trailer with pixels from the enlarged image.

[0020] In one embodiment, the third creation step comprises adding to the corrected image a line delimiting an area of ​​the corrected image containing the pixels of the enlarged image.

[0021] The invention further relates to a display device, on a screen arranged in a passenger compartment of a motor vehicle coupled to a trailer, of an area of ​​interest located at the rear of the vehicle and / or the trailer, the device comprising hardware and / or software elements implementing the method according to the invention, in particular hardware and / or software elements designed to implement the method according to the invention.

[0022] The invention also relates to a computer program product comprising program code instructions recorded on a computer-readable medium for implementing the steps of the method according to the invention when said program is running on a computer. The invention also relates to a motor vehicle, comprising the display device according to the invention and / or the computer program product according to the invention.

[0023] The invention also relates to a data recording medium, readable by a computer, on which is recorded a computer program comprising program code instructions for implementing the method according to the invention.

[0024] The attached drawing represents, by way of example, an embodiment of a display device according to the invention and an embodiment of a display method according to the invention.

[0025] Figure 1 represents a motor vehicle equipped with a display device according to the invention.

[0026] Figure 2 defines a direct orthonormal reference frame of the motor vehicle according to the invention.

[0027] Figure 3 shows an image captured by a rearview camera of the motor vehicle equipped with a trailer.

[0028] Figure 4 shows an image captured by a rearview camera of a motor vehicle without a trailer.

[0029] Figure 5 represents an image captured by a first rearview camera of a first type located on a right side of the motor vehicle.

[0030] Figure 6 shows an image captured by a second rearview camera of the first type located on a left side of the motor vehicle.

[0031] Figure 7 illustrates a first processing applied to the image captured by the first rearview camera of the first type located on the right side of the motor vehicle. Figure 8 illustrates the first processing applied to the image captured by the first rearview camera of the first type located on the left side of the motor vehicle.

[0032] Figure 9 illustrates a second processing applied to the image captured by the first rearview camera of the first type located on the right side of the motor vehicle.

[0033] Figure 10 illustrates the second processing applied to the image captured by the first rearview camera of the first type located on the left side of the motor vehicle.

[0034] Figure 11 illustrates a first sub-step of a step of creating a corrected image.

[0035] Figure 12 illustrates an image creation of a masked area located under the trailer.

[0036] Figure 13 illustrates a first embodiment of a corrected image intended to be displayed on a screen in a vehicle interior.

[0037] Figure 14 shows an image captured by a first rearview camera of a second type located on a right side of the motor vehicle.

[0038] Figure 15 shows an image captured by a second rearview camera of the second type located on a left side of the motor vehicle.

[0039] Figure 16 illustrates a first processing applied to the images captured by rearview cameras of the second type.

[0040] Figure 17 illustrates a second embodiment of a corrected image intended to be displayed on a screen in the vehicle interior.

[0041] Figure 18 is a flowchart of an embodiment of a display method according to the invention.

[0042] An example of a motor vehicle 100 coupled to a trailer 200 and of a device 10 according to the invention, for displaying on a screen 3 arranged in a passenger compartment 4 of the vehicle a zone of interest ZI located at the rear of the motor vehicle 100 is described below with reference to FIG. 1. The motor vehicle 100 may be a vehicle of any type, for example a passenger vehicle or a utility vehicle.

[0043] With reference to Figure 2, a first direct orthonormal reference frame RO (X0, YO, ZO) represented by Figure 2 is defined, the axis XO being parallel to the longitudinal axis of the motor vehicle 100 and oriented towards the rear of the motor vehicle 100, the axis YO being transverse and directed towards the right of the vehicle, and the axis ZO being vertical and oriented towards the top of the vehicle.

[0044] In the remainder of the document, the terms "left" and "right" are defined with reference to the direction of the YO axis of the RO reference frame.

[0045] In the remainder of the document, the term “zone of interest” designates an area in which a maneuver of the motor vehicle 100 coupled to a trailer 200 takes place. In particular, the device 10 according to the invention relates more specifically to a reversing maneuver of the motor vehicle 100 coupled to a trailer 200 and the term “zone of interest” refers to the road infrastructure located behind the motor vehicle 100 or the trailer 200. In particular, in the embodiment described, the zone of interest ZI relates to ground markings located behind the motor vehicle 100, a portion of the ground markings being masked by the trailer 200.

[0046] The motor vehicle 100 comprises a first image sensor 1 and a second image sensor 2 which capture images periodically. Advantageously, the captures can be carried out simultaneously on the first and second sensors 1, 2. In the remainder of the document, preferably but not limited to, the second sensor 2 comprises two sensors 21, 22 respectively located on the right side and the left side of the motor vehicle 100. In the remainder of the document, the terms “sensor 2” or “sensors 21, 22” are used interchangeably.

[0047] At each capture time T, the first and second sensors 1, 2 make it possible to capture the following images:

[0048] - a first image IMG_1 captured by the first sensor 1, of an area of ​​interest ZI on which the trailer 200 masks a so-called masked area ZM, and

[0049] - a second image captured by the second sensor 21 and / or 22, of the area of ​​interest ZI, the second image comprising an image of the masked area IMG_ZM.

[0050] As a note, the second image IMG_ZM advantageously comprises two distinct images IMG_2_D, IMG_2_G corresponding respectively to an image captured by a second sensor 21 arranged on the right side of the vehicle and to an image captured by a second sensor 22 arranged on the left side of the vehicle.

[0051] In an embodiment described in this document, the first sensor 1 is a digital camera 1, called a reversing camera, capable of capturing images intended to supply images to a screen 3, in particular arranged at the level of a reversing mirror.

[0052] An example of a first image IMG_1 captured by the first sensor 1 is illustrated by figures 3 and 4. Figure 3 represents an image captured by a reversing camera 1 of the motor vehicle coupled to a trailer 200. In figure 3, the area of ​​interest ZI is partially masked by the trailer 200, and the first image IMG_1 does not make it possible to know whether there are markings on the ground under the trailer and / or behind the trailer. Thus, from the image IMG_1 alone, the driver does not have the possibility of guiding the trajectory of the trailer with respect to possible markings on the ground. Figure 4 further represents an image captured by a reversing camera 1 of the motor vehicle 100 while it is not coupled to a trailer 200.

[0053] In the embodiment presented, the second sensor 2 comprises two exterior rearview cameras 21, 22. In the remainder of the document, two distinct embodiments of the sensor 2 are described.

[0054] In a first embodiment of the invention illustrated by figures 3 to 13, the two cameras 21, 22 are capable of applying a single focal distance F1 which is small, in particular substantially equal to one millimeter. In the remainder of the document, such cameras are called “cameras of the first type”.

[0055] In a second embodiment of the invention illustrated by figures 13 to 16, the two cameras 21, 22 are able to alternately apply a first focal length F1 which is small, in particular substantially equal to one millimeter, or a second focal length F2 which is greater than the first focal length F1. In the remainder of the document, such cameras are called “cameras of the second type”. In the second embodiment of the invention, only the second focal length F2 is used to capture the second image IMG_2_D', IMG_2_G'. The first focal length F1 is used for the rearview function. Advantageously, the motor vehicle 100 can automatically detect the start of a reversing maneuver initiated by the driver of the motor vehicle 100. The vehicle can then automatically change the focal length of the cameras 21, 22 so that it is equal to the distance F2 during the reversing maneuver of the vehicle.Whatever the embodiment of the invention, the second images captured by the at least one sensor 2 are used to correct the first image IMG_1, so as to obtain a corrected image IMG_C at least partially defining the masked zone ZM.

[0056] Advantageously, during the movement of the vehicle, the images captured by the different cameras are continuously recorded in a database 5. The device 10 comprises means 6 for aggregating the different images contained in the database 5 to create a representation of an environment of the motor vehicle 100. In addition, the device 10 advantageously comprises an odometry system 7 making it possible to locate the motor vehicle 100 in its current environment.

[0057] In one embodiment, the device 10 further comprises at least two ultrasonic sensors 8 located on the right and left lateral sides of a rear bumper of the motor vehicle, which make it possible to determine a position of the sidewalks relative to the vehicle.

[0058] The device 10 further comprises a microprocessor 9. The device 10, and particularly the microprocessor 9, allow the implementation of a method for displaying an area of ​​interest located at the rear of the motor vehicle 100, the method mainly comprising the following modules:

[0059] - a module 91 for capturing, by the first sensor 1, a first image IMG_1 of the area of ​​interest ZI on which the trailer 200 masks the masked area ZM, the module 91 collaborating with the first sensor 1,

[0060] - a module 92 for capturing, by the second sensor 2, a second image IMG_2 of the area of ​​interest ZI, the second image IMG_2 comprising an image IMG_ZM of the masked area ZM, the module 92 collaborating with the second sensor 2,

[0061] - a module 93 for creating a corrected image IMG_C by integrating, into the first image IMG_1, the image IMG_ZM of the masked zone ZM from the second image IMG_2, the module 93 collaborating with the database 5, the aggregation means 6, the odometry system 7 and the ultrasonic sensors 8,

[0062] - a module 94 for displaying the corrected image IMG_C on screen 3, the module 94 collaborating with screen 3.

[0063] The motor vehicle 100, in particular the definition device 10, preferably comprises all the hardware and / or software elements configured so as to implement the method defined in the subject of the invention or the method described below.

[0064] An embodiment of the display method is described below with reference to Figures 3 to 18. In the embodiment represented by Figures 3 to 18, the method comprises four steps E1 to E4 which are executed successively.

[0065] In step E1, a first image IMG_1 of a zone of interest ZI is captured via the first sensor 1, in which the trailer 200 masks a so-called masked zone ZM. In one embodiment, the image capture is carried out periodically, at a given frequency defined for the sensor 1. An example of the first image IMG_1 is provided by FIG. 3. The image IMG_1 does not allow the driver to perceive any possible ground markings which would be located under the trailer 200, and / or behind the trailer 200.

[0066] Then, in step E2, a second image IMG_2 of the area of ​​interest ZI is captured via the second sensor 2, the second image IMG_2 comprising an image IMG_ZM of the masked area ZM.

[0067] An example of an image captured by a camera of the first type is represented by Figures 5 and 6, Figure 5 representing an image IMG_2_D captured by the camera 21 located on the right side of the motor vehicle 100, and Figure 6 representing an image IMG_2_G captured by the camera 22 located on the left side of the motor vehicle 100.

[0068] In each of the images IMG_2_D, IMG_2_G, a portion of the image represents a lateral portion of the motor vehicle 100. The processing carried out on the images IMG_2_D, IMG_2_G is illustrated by figures 7 to 12.

[0069] When the sensor 2 is of the first type, the second step E2 further integrates processing of the second image IMG_2 comprising:

[0070] - a first simplification, illustrated by figures 7 to 10, of the image IMG_2 to keep only the pixels representing the ground (as illustrated by the images IMG_2_D_1, IMG_2_G_1 of figures 7 and 8), and by deleting the pixels representing the motor vehicle (as illustrated by the images IMG_2_D_2, IMG_2_G_2 of figures 9 and 10), and / or

[0071] - a second simplification of the image IMG_2 consisting of deleting the pixels representing sidewalks 300 located at a distance from the vehicle and the trailer, the sidewalks being detected and located by the ultrasonic sensors 8 and / or

[0072] - a generation, illustrated by figure 12, of an IMG_ODO image of the area located under the trailer.

[0073] The image IMG_ODO of a first area located at a first given time T1 under the trailer 200 is constructed from at least one previous image of said first area captured at a second given time T2 occurring before the trailer hides said first area. Thus, to generate the image IMG_ODO,

[0074] - as the vehicle moves, from the images captured by the cameras and recorded in the database 5, the data aggregation means 6 make it possible to create a representation of an environment in which the motor vehicle 100 is moving,

[0075] - thanks to the odometry system 7, the vehicle and trailer are located in their environment,

[0076] - we thus determine an image IMG_ODO of the first zone located under the trailer at time T 1 .

[0077] Thus, when the second sensor 2 is of the first type, the image of the masked zone IMG_ZM is constructed by aggregating the images IMG_2_D_2, IMG_2_G_2, and IMGJDDO.

[0078] An example of an image IMG_2_D', IMG_2_G' captured by a camera of the second type is shown in Figures 14 and 15.

[0079] When the second sensor 2 is of the second type, in step E2, it is detected that the motor vehicle 100 is equipped with a trailer and that it initiates a reversing phase. When these conditions are verified, the sensor 2 - that is to say the cameras 21, 22 - captures images IMG_2_D', IMG_2_G' with the second focal distance F2, the images IMG_2_D', IMG_2_G' being illustrated by figures 14 and 15.

[0080] In addition, when sensor 2 is of the second type, the second step E2 also integrates a simplification of the images from sensor 2 to keep only the pixels representing the ground, and by deleting the pixels representing the motor vehicle and the trailer.

[0081] Then, in step E3, a corrected image IMG_C is created by integrating, into the first image IMG_1, the image IMG_ZM of the masked zone ZM from the second image IMG_2. In the corrected image IMG_C, the trailer 200 is almost erased from the image. As a note, the drawbar 201 of the trailer 200 is represented in the image IMG_C, so that the driver has an indication of the orientation of the trailer relative to the axis XO of the reference frame RO.

[0082] The method of performing step E3 differs depending on the embodiment of the second sensor 2.

[0083] When the sensor 2 is of the first type, the corrected image IMG_C_1, illustrated by figure 13, comprises a representation of a first zone IMG_C_1 1 of the roadway located under the trailer 200 and of a second zone IMG_C_12 of the roadway located behind the trailer 200.

[0084] When sensor 2 is of the second type, the corrected image IMG_C_2, illustrated in Figure 17, includes an enlargement of the image of the masked area IMG_ZM. Indeed, since the focal length F2 used to capture the images of the masked area is large, the representation of the masked area IMG_ZM is too small to be readable by the driver, as can be seen in Figure 16.

[0085] In the embodiment shown in Figure 17, the image of the masked area IMG_ZM is enlarged and then the pixels of the image IMG_1 representing the trailer are replaced by the pixels of the enlarged masked area IMG_ZM_A.

[0086] Inserting an enlarged area IMG_ZM_A creates a discontinuity in the corrected image IMG_C. To make it easier for the driver to understand the image, the discontinuity in the corrected image IMG_C is highlighted by a dotted line showing the borders of the enlarged masked area IMG_ZM_A.

[0087] Then we move on to a fourth step E4 in which the corrected image IMG_C_1, IMG_C_2 is displayed on the screen 3. Finally, the display method according to the invention makes it possible to facilitate a parking maneuver of a vehicle coupled to a trailer, by allowing the driver to orient himself in relation to a marking on the ground located under the trailer or behind the trailer.

[0088] The implementation of the invention is simple and inexpensive, and it only requires equipment already present on the vehicle: a reversing camera and two exterior rearview cameras.

Claims

CLAIMS 1. Method for displaying, on a screen (3) arranged in a passenger compartment (4) of a motor vehicle (100) coupled to a trailer (200), a zone of interest (ZI) located at the rear of the motor vehicle, the vehicle comprising a first image sensor (1), in particular a so-called reversing camera, and at least one second image sensor (2), in particular at least one exterior rearview camera (21, 22), the zone of interest (ZI) comprising a so-called masked zone (ZM) by the trailer (200). the method comprising: • a first step (E1) of capturing, by the first sensor (1), a first image (IMG_1) of the area of ​​interest (ZI) on which the trailer (200) masks the masked area (ZM), • a second step (E2) of capturing, by the second sensor (2), a second image (IMG_2) of the area of ​​interest (ZI), the second image (IMG_2) comprising an image (IMG_ZM) of the masked area (ZM), then, • a third step of creating a corrected image (IMG_C) by integrating, into the first image (IMG_1), the image (IMG_ZM) of the masked zone (ZM) from the second image (IMG_2), then • a fourth step of displaying the corrected image (IMG_C) on the screen (3).

2. Display method according to the preceding claim, characterized in that the integration of the image of the masked zone (ZM) in the first image (IMG_1) relates to elements of the zone of interest (ZI) which are hidden by the trailer (200), said elements being located on the ground, in particular said elements being ground markings.

3. Display method according to one of the preceding claims, the vehicle being equipped with two rear-view cameras (21, 22), a first focal distance (F1) of the two rear-view cameras (21, 22) being small, in particular substantially equal to 1 millimeter, characterized in that: • the at least one second sensor (2) is of a first type if the second capture step (E2) is carried out with the first focal distance (F1), and • the at least one second sensor (2) is of a second type if the second capture step (E2) is carried out with a second focal length (F2) of the rearview cameras (21, 22), the second focal length (F2) being strictly greater than the first focal length, in particular the second focal length (F2) being substantially equal to 10 millimeters.

4. Display method according to one of the preceding claims, characterized in that the masked zone (ZM) comprises a zone (ZS) located under the trailer (200).

5. Display method according to one of the preceding claims, characterized in that an image (IMG_ODO) of a first zone located at a first given instant (T1) under the trailer (200) is constructed from at least one previous image of said first zone captured at a second given instant (T2) occurring before the trailer hides said first zone.

6. Display method according to one of the preceding claims, characterized in that the integration into the corrected image (IMG_C) of the image (IMG_ZM) of the masked zone (ZM) comprises a deletion of pixels from the second image (IMG_2) representing sidewalks (300) located at a distance from the motor vehicle (100) and the trailer (200).

7. Display method according to claim 3, characterized in that the at least one second sensor (2) is of the second type, and in that the third creation step (E3) comprises an enlargement of an image (IMG_ZM) of an area located behind the trailer (200) and a replacement in the corrected image (IMG_C) of pixels representing the trailer (200) by pixels of the enlarged image (IMG_ZM_A).

8. Display method according to the preceding claim, characterized in that the third creation step comprises an addition in the corrected image (IMG_C) of a line delimiting a zone of the corrected image (IMG_C) containing the pixels of the enlarged image (IMG_ZM_A).

9. Device (10) for displaying, on a screen (3) arranged in a passenger compartment of a motor vehicle (100) coupled to a trailer (200), a zone of interest located at the rear of the vehicle and / or the trailer (ZI), the device comprising hardware and / or software elements (1, 2, 3, 4, 5, 6, 7, 8, 9, 91, 92, 93, 94) implementing the method according to one of claims 1 to 8, in particular hardware elements (1, 2, 3, 4, 5, 6, 7, 8, 9) and / or software designed to implement the method according to one of claims 1 to 8.

10. A computer program product comprising program code instructions recorded on a computer-readable medium for implementing the steps of the method according to any one of claims 1 to 8 when said program is running on a computer. 1 1. Motor vehicle, characterized in that it comprises the display device (10) according to claim 9 and / or the computer program product according to claim 10.

Images