System and method for displaying a view of a ground area covered by a vehicle

The method and system use on-board cameras to create a transparent view of the vehicle's surroundings, addressing the lack of comprehensive visibility during maneuvers by displaying a dynamic, detailed view of the ground area under and around the vehicle, improving maneuvering safety.

FR3149262A1Active Publication Date: 2024-12-06RENAULT SA
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
FR2023005365
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-06
Estimated Expiration
2043-05-30

AI Technical Summary

Technical Problem

Existing vehicle systems fail to provide a comprehensive view of the immediate surroundings, including the area under the vehicle, during maneuvers, leading to potential obstacles being unseen by the driver, and existing obstacle detection systems lack detail and accuracy.

Method used

A method and system using multiple on-board cameras to capture images from various angles, construct meta-images, and display a proximity scene on a screen, superimposing representations of the vehicle's front wheels and shadows, providing a real-time, dynamic view of the ground area under and around the vehicle.

Benefits of technology

Enables the driver to visualize the obscured area as if it were transparent, enhancing maneuverability by providing detailed, real-time information on obstacles and trajectory, without relying on external signals or mapping.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A method and system for assisting the driving of a motor vehicle, configured to provide a display of a reconstruction of a proximity scene (PS), said proximity scene comprising at least one portion of ground covered (PSC) by said motor vehicle, in particular its front portion, the reconstruction being developed from previous images captured by at least one on-board camera covering the proximity scene, the reconstruction including an overlay of a representation of the vehicle's front wheels. Figure from the abstract: Fig. 1
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: System and method for displaying a view of a ground area covered by a vehicle

[0001] The invention relates to a method and a system for displaying a view of a ground area covered by a motor vehicle. Such a display of a ground area covered by a vehicle is part of the driving assistance systems providing assistance to a driver in particular for carrying out maneuvers, in particular at low speed.

[0002] Generally speaking, when a vehicle driver performs maneuvers, for example to park the vehicle, he does not have a total view of the vehicle's surroundings. Indeed, certain elements of the vehicle prevent a direct view, for example the hood of the vehicle prevents the elements that are just in front of the vehicle from being seen, but more generally the entire immediate perimeter of the vehicle does not benefit from a direct view by the driver.

[0003] If we consider a van, a utility vehicle and even more so a truck, the direct non-visibility zone is even more important.

[0004] The direct view or the views provided by the rearview systems do not allow the entire immediate perimeter of the vehicle to be seen.

[0005] A fortiori, the elements which are located under the vehicle cannot be seen directly by the driver.

[0006] There are obstacle detection systems based on the use of ultrasonic signals or radar signals, but this type of system does not provide information on the nature of the obstacles, and they are subject to certain false detections or certain non-detections.

[0007] In an urban environment, where the driver is trying to park, the parking space may be equipped with bollards, or there may be a curb higher than 12 or 15 cm to take into account or a pothole to avoid, or some other pitfall to manage.

[0008] In an off-road or track environment, there may be stones or potholes to avoid.

[0009] Some have tried to enrich the so-called bird's eye views (from the English terminology "Bird View" or "Bird Eye View"), but the information provided to the driver remains general and does not have a sufficient level of detail.

[0010] The inventors sought to improve the situation. There is indeed an interest in supplementing the direct vision that the driver can have live, with information which informs him about the position of objects or pitfalls to avoid in the immediate vicinity of the vehicle (or even underneath) for which the driver does not have direct vision when maneuvering the vehicle.

[0011] For this purpose, a method for assisting in driving a motor vehicle is proposed here. (VH), configured to provide a display of a restitution of a proximity scene (SP), said proximity scene comprising at least a portion of ground covered (PSC) by said motor vehicle, in particular its front part, the method comprising: a- acquiring images via at least one on-board camera (40,41,42,43,44), preferably four on-board cameras, b- recording in memory images, captured at a plurality of time stamps (tl, t2,... tn) and / or positions of the vehicle, relating at least to a front zone of the vehicle (ZA), and / or meta-images (eg 360° or Bird view), constructed from the images captured at a plurality of time stamps (tl, t2,... tn) and / or positions of the vehicle, c- recording by odometry the movements of the vehicle, d- determining a current position of the vehicle, and the perimeter of the current covered portion of ground (PSC2), in a local reference frame, e- determine in the image or meta-image memory at least one item which covers the perimeter of the current covered portion of ground (PSC2), and determine the source image(s) or meta-image(s) to be used, f- display, on a screen, a restitution of the source image to be displayed by superimposing a representation of the front wheels of the vehicle.

[0012] Thanks to the arrangements presented above, the driver is provided with visual information concerning the underside of the vehicle without using a camera directed towards the ground or installed under the chassis of the vehicle. The driver can thus maneuver as if he had a view of the area under the engine hood. The driver has the impression of "seeing through" the hood and the engine.

[0013] It should be noted that to achieve graphic rendering, it may be necessary to make one or more possible corrections, such as vertical pivoting, straightening, removal of wide-angle distortion, etc.

[0014] In this document, the term "meta-image" refers to a coherent assembly of data constructed from images captured by cameras. For example, the term meta-image here refers to: a panoramic view, a 360° graphic rendering or a view constructed by considering a virtual viewpoint from above, such as for example the "Bird View", without these examples being limiting.

[0015] In this document, it should be understood that the term "position of the vehicle" encompasses both the geographical position of a reference point of the vehicle and the current heading of the vehicle, i.e. the orientation, for example the orientation relative to north. The geographical position is referenced relative to a spatial reference point, which can be considered local here. It is considered that the vehicle is moving on the ground in 2 dimensions so the geographic position can be considered as a local map position with two coordinates.

[0016] Regarding the phrase "the plurality of timestamps and / or vehicle positions", it is understood that the image marking reference can be temporal or spatial. According to a purely temporal logic, the images, and where appropriate the meta-images, are time-stamped regardless of the vehicle's forward speed, even if it is stationary. According to an essentially spatial logic, the images are identified by the geographical / cartographic position of the viewpoint that captured them or, in this case, the vehicle's position (heading included) at the time of capture. The temporal logic is simpler but can lead to storing large quantities of data unnecessarily when the vehicle is not moving. According to this logic, images can only be recorded for different vehicle positions.Spatial logic is more relevant from this point of view, but a little more complex to implement because it is necessary to follow and manage the successive positions of the vehicle.

[0017] In step e-, we note that the process is similar to a return to previous configurations where / when at least one camera had direct vision of the portion of ground now covered. It is therefore the storage and subsequent recovery of a relevant image which allows the restitution of the image corresponding to the portion of ground now covered by the vehicle.

[0018] It should also be noted that the proposed method works both day and night, because at night we cleverly benefit from the frontal lighting of the vehicle by its front projectors, thus the lighting of the frontal scene is suitable for the use of the images of the scene which are made of it subsequently.

[0019] According to one aspect, the restitution of the proximity scene is carried out with a point of view corresponding substantially to a point of view of a driver (C) in the driving position of the vehicle. This presents the driver with a view that resembles what he would see if the hood and the vehicle elements between the front wheels were transparent.

[0020] According to a particular embodiment, at least four cameras are provided, and a 360° or Bird view meta-image construction is used, i.e. 'bird's eye view'.

[0021] We thus use an already existing function to extract relevant images or a relevant part. The existing 360° function is called in the jargon of the trade AVM ("Around View Monitor").

[0022] According to one aspect, the proximity scene (SP) comprises the covered ground portion (PSC) and its perimeter in the form of a peripheral band (BP). Thus, not only is the ground portion covered directly by the vertical shadow of the vehicle, but also the immediate surroundings of it, to the left, to the right and in front, such as for example edges such as sidewalks, space limits, shrubs or vegetation.

[0023] According to one aspect, the rendering of a proximity scene further comprises an added display of the possible trajectory of each front wheel in the event of forward or backward movement of the vehicle, based on instantaneous steering wheel angle information.

[0024] By means of which, the graphical rendering provides the driver with a visual indication of the trajectory of the wheels. In addition, this rendering is carried out in real time, so that the driver can visualize what will happen, in particular if he moves forward in relation to any obstacles that the wheels will encounter. The trajectory represented is recalculated in real time with the instantaneous steering wheel angle, consequently the graphical rendering is dynamic depending on the movements made by the driver on the steering wheel even if the vehicle is not moving forward.

[0025] According to one embodiment, the representation of the front wheels of the vehicle comprises a footprint / footprint on the ground and / or a 3-dimensional avatar. This provides an element of understanding making the graphic rendering intuitive for the driver.

[0026] By "avatar" here is meant a symbolic representation. According to one embodiment, the wheel avatar is partially translucent, i.e. partially transparent, so that the visual rendering also allows one to see through the wheel.

[0027] According to one aspect, the rendering of a proximity scene further comprises an added display of a template and / or the shadow cast by the vehicle in vertical projection. By means of which, the driver visualizes the front limit of the vehicle, this limit corresponding to the overall dimension of the vehicle. The driver can thus know if the vehicle is going to contact an obstacle located in the immediate vicinity. Similarly, the driver also visualizes the limit to the left and the limit to the right of the vehicle.

[0028] According to one aspect, when using a single front camera, the front zone of the vehicle (ZA) extends axially (longitudinally), from the front edge of the vehicle, between a minimum front distance and a maximum front distance, and extends laterally according to an open cone having a half vertex angle noted a2, a2 ​​being at least equal to 60°.

[0029] The image capture area is very wide, at least 120°, and all possible future positions are thus captured. We note that in this single camera configuration, we can cleverly use a front camera used elsewhere in distance control systems with respect to preceding vehicles.

[0030] According to one aspect, steps a- to f- are repeated, preferably with an iteration frequency at least equal to 5 Hz. This provides real-time image restitution.

[0031] According to one aspect, the iteration frequency is at least equal to 25 Hz. Whereby, a conventional smooth video type rendering is provided for viewing through the hood.

[0032] According to one aspect in step e- the item chosen is the most recent item covering the perimeter of the current covered ground portion (PSC2).

[0033] The present invention also relates to a motor vehicle driving assistance system, configured to provide a display of a restitution of a proximity scene (SP), said proximity scene comprising at least a portion of ground covered (PSC) by said motor vehicle, in particular its front part, - the restitution being developed from previous images captured by at least one on-board camera, covering the nearby scene, - the restitution including an inlay of a representation of the front wheels of the vehicle.

[0034] According to one embodiment, the representation of the front wheels of the vehicle comprises a footprint on the ground and / or a 3-dimensional avatar.

[0035] According to one embodiment, the assistance system also includes an added display of the template and / or the shadow cast by the vehicle in vertical projection.

[0036] The present invention also relates to a vehicle comprising at least one screen, a control unit, at least one on-board camera (40, 41, 42, 43, 44), and a driving assistance system configured to implement a method described previously.

[0037] The invention will be further detailed by the description of non-limiting embodiments, and on the basis of the appended figures illustrating variants of the invention, in which: - [Fig.l] illustrates schematically in elevation an example of a vehicle in which the invention is implemented, which moves on a ground; - [Fig.2] schematically illustrates a top view of a vehicle in accordance with the invention; - [Fig.3] schematically illustrates a parking maneuver situation seen in elevation; - [Fig.4] schematically illustrates a parking maneuver situation seen in above ; - [Fig.5] schematically illustrates an example of visual rendering provided to the driver on a screen concerning the configuration illustrated in figures 3 and 4; - [Fig.6] schematically illustrates the portion of ground covered in the proximity scene, itself within the perimeter of the bird's-eye view; - [Fig.7] schematically illustrates the diversity of possible trajectories; - [Fig.8] schematically illustrates an example of a functional diagram of implementation of the process; - [Fig.9] illustrates a side view of a vehicle with the ground portion covered inside the proximity scene; [Fig. 10] illustrates a top view of a vehicle with the ground portion covered within the proximity scene; [Fig. 11] schematically illustrates an example of a parking maneuver trajectory; - [Fig. 12] illustrates a second embodiment of the method based on the use of a front windshield camera; - [Fig. 13] illustrates an example of the sequence of process steps.

[0038] In the various figures, the same references designate identical or similar elements. For reasons of clarity of the description, certain elements are not necessarily represented to scale.

[0039] Generalities and system

[0040] In [Fig. 1], a vehicle denoted VH is moving on a roadway 60. In the middle of the figure, a vehicle silhouette in solid lines illustrates the current position of the vehicle at the present time t2, on the left a vehicle silhouette in dotted lines illustrates a position at an earlier time t1, and on the right another silhouette in dotted lines illustrates a position at a later time t3.

[0041] Generally speaking, a motor vehicle has a reference frame in space comprising a longitudinal axis X along a direction of movement of said motor vehicle, a lateral axis Y perpendicular to the longitudinal axis X, and a vertical axis Z perpendicular to the longitudinal axis X and to the lateral axis Y, the vertical axis being directed from bottom to top.

[0042] In the illustrated example, the roadway 60 is dotted with small imperfections, for example holes 61, 63 and bumps 62, 64, 65, generically called here “reefs”. Some reefs can be mobile like an animal, for example a passing cat, or even a ball.

[0043] At time t1, driver C does not see reef 61 because it is located below his direct low line of vision 71, but driver C sees reefs 62, 63, 64, 65 a little further away.

[0044] At time t2, the pitfalls 61, 62 are located under the front portion of the vehicle in an area called here the covered ground portion which we will see in detail later. The driver C does not see the pitfalls 63, 64, because they are located below his direct low line of vision 72, but the driver C sees the pitfall 65.

[0045] At time t3, the reefs 64, 65 are located under the front portion of the vehicle and are no longer visible to the driver C.

[0046] Thus, as the vehicle moves forward, portions of the roadway, which were visible at a given moment, are no longer visible at a later moment by direct vision from a point of view corresponding to the driver's head in the driving position.

[0047] Furthermore, the vehicle is equipped with one or more cameras whose function is to periodically capture images to restore them directly or indirectly to the driver. In particular, at least one front camera 41 is provided which may be located in the front bumper (or according to another configuration in the upper zone of the windshield on the inside.

[0048] Still with reference to [Fig.l], by placing ourselves at time t2, one or more images which were captured at time tl, in which the pitfalls 62 63 can be seen are used for a presentation of information to the driver at time t2.

[0049] Similarly, a little later, at time t3, one or more images that were captured at time t2, in which the pitfalls 64, 65 can be seen, are used to present information to the driver at time t3. This principle will be explained in more detail in the following sections.

[0050] In the example illustrated in [Fig.2], the vehicle is equipped with 4 cameras including a front camera 41 arranged in the front bumper. The vehicle is equipped with a left side camera 43, a right side camera 42 and a rear camera 44.

[0051] In certain configurations, there may be two complementary cameras, making a total of six cameras.

[0052] The cameras in question are wide-angle cameras, e.g. so-called 'fish eye' cameras. The cameras capture color images. It is not excluded to use cameras sensitive in the infrared band.

[0053] The front camera 41 has a field of vision denoted FOV1. The right side camera 42 has a field of vision denoted FOV2. The left side camera 43 has a field of vision denoted FOV3. The rear camera 44 has a field of vision denoted FOV4. As known per se, to produce a 360° vision, the different adjacent fields of vision have their extreme angular zones which intersect each other. This provides continuous coverage of the entire angular space around the vehicle VH, therefore 360° coverage around the vehicle.

[0054] The overlapping areas are identified, for example, on the basis of contour recognition, or any other geometric or photometric method. The identification of the overlapping areas allows the connection of two adjacent images, two by two. This results in the formation of a 360° meta-image. The concatenated 360° meta-image is then a continuous meta-image, without duplicates or gaps. This 360° function is called AVM (“Around View Monitor”) in the profession, and may already exist before the addition of the virtually transparent hood function.

[0055] From four images, it is also possible to form a Bird View type view as known per se. The reader may refer, for example, to document US 11021104.

[0056] We note TRV the trajectory of the vehicle.

[0057] The vehicle comprises a 3g left front wheel and a 3d right front wheel, these are of steering wheels. The YO axis passes through the front wheel axis. The XO axis corresponds to the longitudinal centerline of the vehicle. The reference position, marked RO, is taken for example at the intersection of the XO and YO axes.

[0058] Interposed between the front wheels is a powertrain known as GMP, whether the vehicle is equipped with an internal combustion engine or is electric or hybrid.

[0059] Example

[0060] The method proposed by the present invention will now be explained on the basis of a non-limiting example illustrated in figures 3, 4 and 5.

[0061] Driver C of the vehicle maneuvers the vehicle into a parking space. The parking space is bordered on the right by a sidewalk 6.

[0062] The sidewalk 6 is of a substantial height, for example 15 to 20 cm. In front of the parking space, there are small concrete blocks marked 5. There is also in an intermediate portion of the parking space a dome-shaped presence detection device 7. Parking space boundary markings 8 are also provided.

[0063] In the situation illustrated in [Fig. 3], the driver C does not see the markers 5 which are located just in front of the front bumper of the vehicle, he also does not see the pavement 6 to the immediate right of the vehicle and he also does not see the detection device 7 which is already located under the vehicle. Certain parts of the parking space limit markings 8 may also not be directly visible.

[0064] [Fig. 5] shows an example of a visual rendering provided to the driver on a screen. This screen shows the vision that the driver would have if all the elements in front of him were transparent.

[0065] This graphical restitution comprises a restitution element 5a representing the plots 5, a restitution element noted 6a representing the sidewalk 6, a restitution element noted 7a representing the detection device 7.

[0066] The front wheels of the vehicle are represented in the form of avatars. Avatar 1a represents the left wheel 3g and avatar 2a represents the right wheel 3d. The avatars are represented as if the driver saw the wheels directly by transparency through the hood and other elements of the engine compartment. In addition, the wheel avatar itself may be partially translucent, i.e. partially transparent, so that the visual rendering allows one to see through it. For example, driver C can see the position of the curb 6 through the wheel.

[0067] Line 24 is the representation of the vehicle's template. This template or overall cast shadow (excluding rearview mirrors) is calculated from the current position of R0 (at the intersection of the X0 and Y0 axes).

[0068] The line marked 22 shows the ground clearance of the left wheel, turned to the right, the line 21 represents the trajectory of the corresponding wheel. We have represented several ground locations to illustrate the dynamic aspect of the display. Even if the vehicle is not moving forward or backward, the angular position of the wheels is reflected in real time. Line 23 shows another orientation of the front wheels turned to the left. This illustrates the real-time highlighting of the impact of steering wheel rotation on the orientation of the wheels and the resulting trajectory.

[0069] Functionality

[0070] As illustrated in Figures 4, 6 and 10, a covered portion of ground, marked PSC, is defined, which corresponds to the portion of ground covered by the front part of the motor vehicle,

[0071] We also define a proximity scene, marked SP, which is a region of interest that encompasses the portion of ground covered PSC.

[0072] In the longitudinal direction X, the covered ground portion PSC extends from the rear of the front wheels to the front limit of the front bumper, over a length marked PSCX in Figures 9 and 10. In the transverse or lateral direction Y, the covered ground portion PSC extends from the left limit of the vehicle body to the right limit of the vehicle body, over a width marked PSCY in [Fig. 10].

[0073] Expressed differently, the covered ground portion PSC corresponds to the vertical projection (vertical cast shadow) of the front portion of the vehicle from a line 80 joining the rear of the front wheels. Note that the front rearview mirrors are excluded from this covered ground portion.

[0074] To obtain the proximity scene SP of interest for the present invention, the covered ground portion PSC is completed by a peripheral border noted BP.

[0075] The proximity scene SP extends forward by a few tens of centimeters relative to the covered portion of ground, the forward overhang is noted DLX. Similarly, the proximity scene SP extends to the left and to the right relative to the covered portion of ground PSC. The proximity scene SP has a length along X noted SP-LX and a width, along the transverse direction Y, noted SP-LY. The proximity scene SP may extend backward beyond the covered portion of ground but this is not obligatory, in a variant the two rear limits may coincide, without overhang.

[0076] The forward overhang DLX is a parameter that can be adjusted depending on the geometry of the vehicle and in particular the more or less plunging aspect of the hood in relation to the direct visibility provided to the driver of the vehicle in the driving position.

[0077] This DLX setting can also take into account the variability of drivers' anthropometric measurements. Average anthropometric measurements and the 10% percentile or even the 5% percentile can be taken into account.

[0078] It is also not excluded to have values ​​of lateral overtaking distance adjustable or configurable for the same reasons as previously.

[0079] The perimeter of the proximity scene corresponds to what we wish to display available to driver C, such as for example the visual rendering illustrated in [Fig.5].

[0080] The vehicle VH is also equipped with means for recording the vehicle's movements, for example by odometry.

[0081] For example, in the vehicle there are sensors delivering wheel rotation signals, used in particular by the braking system. The wheel rotation signals are also used here for the odometry function. In particular, the rotation signals of the rear wheels can be recorded, and by integration, the displacement of the vehicle in position and orientation can be deduced.

[0082] For example, the point R0, the intersection of the axes X0 and Y0, can be chosen as the reference point for the vehicle position. Alternatively, it is not excluded to take the center of gravity of the vehicle or the midpoint of the rear axle as the reference position.

[0083] Furthermore, the vehicle is equipped with a steering wheel angular position sensor, also called a steering wheel angle sensor for short.

[0084] This information completes the calculation of integration of the rear wheel rotation signals to provide reliable information on the position of the vehicle, including orientation (also called 'heading').

[0085] In a very simple case illustrated in [Fig.6], the vehicle moves forward in a straight line at a speed VS, the steering wheel angle is equal to 0 (driving in a straight line).

[0086] On the left is represented the situation at a first instant, and on the right is represented the situation at a second instant.

[0087] At the first instant, the portion of ground covered by the front of the vehicle is noted PSC0. The camera(s) capture images at least on a disc 20 of radius R centered on a central point of the vehicle. This disc 20 moves as the vehicle moves forward.

[0088] In particular, the cameras capture one or more images of the area which will be covered by the vehicle in the future, in particular the area(s) noted PSC1 which extend(s) from the front edge of the vehicle between a distance DI and a distance D2.

[0089] In practice, one can choose (D2 - Dl) > PSCX, or even (D2 - Dl) > SP-LX. However, in the opposite case, the successively captured images are aggregated by an image connection system (for example, the missing pixels in the old image are added from the new image).

[0090] The captured images are recorded regularly. The iteration frequency of image recording (or meta-images) is noted F2.

[0091] At the second instant (right part of [Fig.6]), the vehicle has advanced a distance DT, the portion of ground covered is now PSC2. The computing unit on board the The vehicle will therefore search the memory for previously captured images that cover the PSC2 portion of the ground.

[0092] In the image or meta-image memory, the one that most coincides geometrically and in orientation with the covered ground portion PSC2 is chosen. In addition, if there are several suitable time-stamped images, the most recent image or meta-image is chosen.

[0093] [Fig.7] illustrates, from a position at a given time, a part of the plurality of possible future positions of the vehicle, which will be able to cover a plurality of covered ground portions, noted PSCi.

[0094] It is expected that the images collected and recorded will cover all possible possibilities in the immediate future.

[0095] In [Fig.8], the functional diagram of the system illustrates four cameras, with their respective correction block.

[0096] Reference 45 illustrates the setting of each camera; reference 46 illustrates the coordinates of the pose point of view, which then make it possible to make the correction made necessary by the wide-angle distortions of the camera, as known per se.

[0097] In the block marked 9, the images are concatenated with the overlap removed to obtain a 360° panoramic meta-image. In the block marked 9, the bird's eye view 'Bird View' can also be constructed, using transformations known per se and not explained in detail here. The 360° panoramic meta-image can be displayed in real time on a vehicle screen, illustrated by the 'Display 360°' block, using the AVM function already mentioned above.

[0098] The block marked 10 represents a calculation unit and / or the calculation functions which allow the graphic restitution of vision through the virtually transparent cover according to the presentation.

[0099] The images and meta-images are stored in a local mass memory MEM marked 12.

[0100] According to one embodiment of the method, the images from the front camera are used directly without using meta-image portions. According to another embodiment of the method, mainly meta-image portions are used.

[0101] The block marked 11 represents the function for calculating the current position of the vehicle based on odometry logic. The block marked 11 periodically receives in real time the rotation signals of the wheels, for example the rear wheel, and also receives in real time the angle of the steering wheel. As known per se, an integration of these signals makes it possible to calculate step by step, as and when, the different successive positions of the vehicle, including orientation.

[0102] The block marked 13 represents the function of graphic restitution of the vision through the cover made virtually transparent.

[0103] Advantageously, the proposed method comprises the following steps, as shown in [Fig. 13].

[0104] An iterative image acquisition step (step a-) is provided via at least one on-board camera (40,41,42,43,44). As already mentioned above, there may be 4 cameras arranged on the vehicle, but in another embodiment, there may be a single front camera.

[0105] Furthermore, a step of recording the images in memory 12 MEM (step b-) is provided. The images are captured periodically. The images are captured at a plurality of time stamps (tl, t2,... tn) and / or positions of the vehicle.

[0106] The captured images relate to at least one front zone of the vehicle ZA. The front zone of the vehicle ZA extends longitudinally from the front edge of the vehicle, between a minimum frontal distance DI and a maximum frontal distance D2. The front zone of the vehicle ZA extends laterally along an open cone having a half vertex angle noted a2, a2 ​​being at least equal to 60°, i.e. 120° minimum opening.

[0107] As an overlay to the raw images, meta-images (eg 360° or Bird view) can be constructed and recorded, constructed from images captured at a plurality of time stamps (tl, t2,... tn) and / or vehicle positions.

[0108] As already mentioned above, the image recording can be done on a time basis, i.e. for example every xx milliseconds or on a displacement basis, i.e. at each displacement of at least yy millimeters.

[0109] In parallel (step c-), the vehicle's movements are recorded by odometry. This is done in a local reference point without the need for external signals such as geolocation signals. There is also no need to use mapping.

[0110] Then, for a current position of the vehicle, the current covered portion of ground noted PSC2 is determined, in a local reference frame (step d-).

[0111] From the definition of the portion of ground covered at the current time, at least one item is determined in the image or meta-image memory which covers the perimeter of this current portion of ground covered PSC2 (step e-).

[0112] The source image(s) or meta-image(s) to be used are determined. As already mentioned, the image or portion of meta-image that most closely matches the portion of ground covered at time t is chosen.

[0113] A restitution of the source image to be displayed is displayed on a screen by superimposing a representation (1a, 2a) of the front wheels of the vehicle.

[0114] To achieve graphic rendering, it may be necessary to make one or more possible corrections, such as vertical pivoting or straightening. Some corrections are made upstream in the processing of the captured images, others are made at the time of graphic rendering.

[0115] [Fig. 13] shows the different stages set out above.

[0116] The method is carried out iteratively. There may be one or more recurrence frequencies.

[0117] For example, the acquisition of images can be done at a frequency Fl. For example, the recording of meta images can be done with a frequency F2. For example, the calculations of odometry and current position can be carried out with a frequency F3. For example, the display of the graphic restitution can be done with a frequency F4.

[0118] The recurrence frequencies F1, F2, F3 and F4 may all be different in an asynchronous operating logic. According to another example, the recurrence frequencies are linked together with or without sub-sampling. Generally, all the frequencies mentioned are greater than 5 Hz. Preferably, at least the restitution steps are carried out at a frequency of at least 25 Hz.

[0119] [Fig. 11] represents a parking space approach trajectory.

[0120] The reference OG(i) denotes the successive positions of the vehicle reference points for odometry calculations.

[0121] The SP(i) reference denotes the successive positions of the delimitation of the proximity scene for the successive positions of the vehicle.

[0122] [Fig. 12] represents a second embodiment where essentially a single image capture camera, namely the front camera 40 located in the upper area of ​​the windshield inside the vehicle.

[0123] The operating principle itself is the same as that explained above.

[0124] Advantageously, according to the present invention, the system and the method are autonomous, i.e. the proposed functionality is ensured in areas not covered by cellular networks. In addition, the proposed functionality is ensured in locations where geolocation signals are not available. It is also noted that there is no need for mapping for the proposed function.

Claims

Claims

1. Method for assisting in driving a motor vehicle (VH), configured to provide a display of a restitution of a proximity scene (SP), said proximity scene comprising at least one portion of ground covered (PSC) by said motor vehicle, in particular its front part, the method comprising: a- acquiring images via at least one on-board camera (40, 41, 42, 43, 44), preferably four on-board cameras, b- recording in memory images, captured at a plurality of timestamps (tl, t2,... tN) or positions of the vehicle, relating at least to a front zone of the vehicle (ZA), and / or meta-images constructed from the images captured at a plurality of timestamps (tl, t2,...tN) and / or vehicle positions, c- record the movements of the vehicle by odometry, d- determine a current position of the vehicle, and the perimeter of the current covered portion of ground (PSC2), e- determine in the image or meta-image memory at least one item which covers the perimeter of the current covered portion of ground (PSC2), and determine the source image(s) or meta-image(s) to be used, f- display, on a screen, a restitution of the source image to be displayed by superimposing on it a representation of the front wheels of the vehicle.

2. Method according to claim 1, in which the restitution of the proximity scene is carried out with a point of view corresponding substantially to a point of view of a driver (C) in the driving position of the vehicle.

3. Method according to any one of claims 1 to 2, in which at least four cameras are provided, and a 'bird's eye view' type meta-image construction is used.

4. Method according to any one of claims 1 to 3, in which the proximity scene (SP) comprises the covered ground portion (PSC) and its periphery in the form of a peripheral strip (BP).

5. Method according to any one of claims 1 to 4, in which the rendering of a proximity scene further comprises an added display of the possible trajectory of each front wheel in the event of forward or backward movement of the vehicle, based on instantaneous steering wheel angle information.

6. Method according to any one of claims 1 to 5, in which the rendering of a proximity scene further comprises a display of a template and / or the shadow cast by the vehicle in vertical projection.

7. Method according to any one of claims 1 to 6, in which, when using a single front camera, the front zone of the vehicle (ZA) extends axially, from the front edge of the vehicle, between a minimum front distance and a maximum front distance, and extends laterally according to an open cone having a half vertex angle noted a2, a2 being at least equal to 60°.

8. A method according to any one of claims 1 to 7, wherein steps a- to f- are repeated.

9. Motor vehicle driving assistance system, configured to provide a display of a rendering of a proximity scene (SP), said proximity scene comprising at least one portion of ground covered (PSC) by said motor vehicle, in particular its front part, - the rendering being developed from previous images captured by at least one on-board camera, covering the proximity scene, - the rendering comprising an overlay of a representation of the front wheels of the vehicle.

10. Vehicle comprising at least one screen, a control unit, at least one on-board camera (40,41,42,43,44), and a driving assistance system configured to implement a method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Around view monitoring system and operating method thereof

    US11021104B2

  • Apparatus and method for displaying a blind spot

    US20130300872A1

  • Sensor fusion based perceptually enhanced surround view

    US20210075959A1

  • Vehicular trailering assist system

    US20210094473A1

  • System and method for providing an image depicting a transparent rear vehicle portion

    WO2022261671A1