System and method for generating an under-body view for a vehicle

Abstract

A method and system for generating an under-body view for a vehicle is described. The method comprises receiving, from one or more image capturing units, a plurality of images related to a terrain having at least one object. Further, the method comprises generating a fused image by fusing the plurality of images based on one or more vehicle dynamics parameters associated with the vehicle, the fused image being associated with the terrain being traversed by the vehicle. Further, the method comprises generating an under-body view of the vehicle including the fused image, the at least one object, and a pre-stored view of the vehicle.

Description

SYSTEM AND METHOD FOR GENERATING AN UNDER-BODY VIEW FOR A VEHICLE

[0001] The present invention generally relates to vehicle cameras, and more particularly relates to systems and methods for generating an under-body view for a vehicle that enables hazard avoidance, collision detection, and diagnosis.

[0002] Generally, roads and other terrains where vehicles are driven may have some hazards which, if collided with the vehicles, can cause damage to an under-body of the vehicles. The hazards can be rocks, tree branches, potholes, and the like. In case the dimensions of the hazards are not large enough to pose a collision risk to the vehicles, the vehicles can pass over the hazards. However, hazards with larger dimensions pose a collision risk for the vehicles. The under-body (under-chassis) of vehicles is generally a weak spot and is not designed to sustain heavy collisions. Such collisions may cause damage to components of the vehicles associated with the under-body. For example, oil tanks, fuel tanks, and battery banks may collide with hazards on roads. Moreover, vehicle tires may be driven over the hazards that may cause tire chunking and reduction in tire life.

[0003] Further, in case of a collision, there is no way for a driver to localize the damage and find out about the components that are damaged and what is the extent of damage. Existing techniques to generate under-body views include placing sensors under the chassis of the vehicle, however, such techniques are not effective. With sensors under the chassis of the vehicle, visibility is limited due to dust, low light, and limited field of view, and as a result, the sensors provide noisy data which is ineffective. Moreover, a two-dimensional view of the terrain where the vehicle is moving is not beneficial to the driver as such a view does not provide information regarding whether vehicle will pass over the hazards or possible damage that can be caused by the hazards.

[0004] Accordingly, there is a need for systems and methods that overcome at least some of the above-mentioned limitations. In particular, there is a need for systems and methods that generate a real-time under-body view of the vehicle which allows drivers to avoid hazards and be alerted regarding any collisions.

[0005] This summary is provided to introduce a selection of concepts, in a simplified format, that are further described in the detailed description of the invention. This summary is neither intended to identify key or essential inventive concepts of the invention and nor is it intended for determining the scope of the invention.

[0006] According to one embodiment of the present disclosure, a method for generating an under-body view for a vehicle is described. The method comprises receiving, from one or more image capturing units associated with the vehicle, a plurality of images related to a terrain being traversed by the vehicle, wherein the terrain includes at least one object. Further, the method comprises generating a fused image by fusing the plurality of images based on one or more vehicle dynamics parameters associated with the vehicle, the fused image being associated with the terrain being traversed by the vehicle. Furthermore, the method comprises generating an under-body view of the vehicle including the fused image, the at least one object, and a pre-stored view of the vehicle.

[0007] According to another embodiment of the present disclosure, a system to generate an under-body view for a vehicle is described. The system comprises a memory and a processor communicatively coupled with the memory. The processor is configured to receive, from one or more image capturing units associated with the vehicle, a plurality of images related to a terrain being traversed by the vehicle, wherein the terrain includes at least one object. Further, the processor is configured to generate a fused image by fusing the plurality of images based on one or more vehicle dynamics parameters associated with the vehicle, the fused image being associated with the terrain being traversed by the vehicle. Furthermore, the processor is configured to generate an under-body view of the vehicle based on the fused image, the at least one object, and a pre-stored view of the vehicle.

[0008] To further clarify the advantages and features of the present invention, a more particular description of the invention will be rendered by reference to specific embodiments thereof, which is illustrated in the appended drawings. It is appreciated that these drawings depict only typical embodiments of the invention and are therefore not to be considered limiting of its scope. The invention will be described and explained with additional specificity and detail with the accompanying drawings.

[0009] These and other features, aspects, and advantages of the present invention will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:

[0010] FIG. 1A illustrates an exemplary overview of an environment comprising a vehicle and a system associated with the vehicle, according to an embodiment of the present disclosure;

[0011] FIG. 1B illustrates a real-life example of the environment comprising the vehicle and the system, according to an embodiment of the present disclosure;

[0012] FIG. 2 illustrates a detailed block diagram of the system for generating an under-body view for the vehicle, according to an embodiment of the present disclosure;

[0013] FIG. 3 illustrates a block diagram depicting operational flow associated with the plurality of modules, according to an embodiment of the present disclosure;

[0014] FIGS. 4A-4E illustrate exemplary images associated with operations of the plurality of modules, according to an embodiment of the present disclosure; and

[0015] FIGS. 5A-5E illustrate process flows for generating an under-body view for the vehicle, according to an embodiment of the present disclosure.

[0016] Further, skilled artisans will appreciate that elements in the drawings are illustrated for simplicity and may not have necessarily been drawn to scale. For example, the flow charts illustrate the method in terms of the most prominent steps involved to help to improve understanding of aspects of the present invention. Furthermore, in terms of the construction of the device, one or more components of the device may have been represented in the drawings by conventional symbols, and the drawings may show only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the drawings with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.

[0017] For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the various embodiments and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended, such alterations and further modifications in the illustrated system, and such further applications of the principles of the invention as illustrated therein being contemplated as would normally occur to one skilled in the art to which the invention relates.

[0018] It will be understood by those skilled in the art that the foregoing general description and the following detailed description are explanatory of the invention and are not intended to be restrictive thereof.

[0019] Reference throughout this specification to “an aspect”, “another aspect” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrase “in an embodiment”, “in another embodiment” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0020] The terms “comprise”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such process or method. Similarly, one or more devices or sub-systems or elements or structures or components proceeded by “comprises... a” does not, without more constraints, preclude the existence of other devices or other sub-systems or other elements or other structures or other components or additional devices or additional sub-systems or additional elements or additional structures or additional components.

[0021] The terms “multi-party conversation”, “human-to-human conversation”, and “conversation”, may be used interchangeably throughout the description. The terms “user device”, “device”, and “electronic device” along with their inherent variations may be used interchangeably throughout the description.

[0022] FIG. 1A illustrates an exemplary overview of an environment 100 comprising a vehicle 110 and a system 120 associated with the vehicle 110. FIG. 1B illustrates a real-life example of the environment 100 comprising the vehicle 110 and the system 120. The system 120 may be configured to generate an under-body view of the vehicle 110 and facilitate hazard detection, collision prevention, and collision diagnosis. The vehicle 110 may include any type of vehicle such as a car, a truck, a trailer, a scooter, and the like.

[0023] The vehicle 110 may be associated with an Electronic Control Unit (ECU) 112 configured to monitor and control vehicle parameters such as vehicle state, velocity, steering, braking, throttle, and the like. In an embodiment, the ECU 112 may be associated with a vehicle dynamics model 114 configured to predict a vehicle dynamics and current trajectory of the vehicle 110 based on the vehicle parameters and based on one or more vehicle components (such as, wheels, suspension, etc.). In an embodiment, the vehicle dynamics model 114 may be a quarter car dynamics model.

[0024] The vehicle 110 may be associated with image capturing unit(s) 116 configured to capture images of a terrain being traversed by the vehicle 110. The image capturing units 116 may comprise a front camera and / or a rear camera. The vehicle 110 may further be associated with an interface 118 configured to enable a user within the vehicle to view and interact with information regarding the vehicle 110 and information regarding possible hazards and collisions, as will be described in detail further below. In an embodiment, the interface 118 may be a Graphical User Interface (GUI). In an embodiment, the interface 118 may be provided at a dashboard of the vehicle 110 and / or may be associated with an instrument cluster of the vehicle 110. For the sake of brevity, the architecture and standard components related to the vehicle 110 are not discussed in detail.

[0025] The vehicle 110 may be configured to traverse a terrain 130, in that, the vehicle 110 may move on the terrain 130. The terrain 130 may be any type of terrain which the vehicle 110 can be driven on, such as, straight roads, curved roads, etc. The terrain 130 may have at least one object 132 which may or may not be a hazard for the vehicle 110. In non-limiting examples, the at least one object 132 may include various types of objects such as potholes, wooden branches, leaves, rocks, and the like.

[0026] As depicted, the system 120 may be in communication with the vehicle 110. The system 120 may receive images captured by the image capturing units 116 indicative of the terrain 130 and the at least one object 132. In some embodiments, the system 120 may be a standalone entity located at a remote location and connected to the vehicle 110 via any suitable network. For example, the system 120 may be implemented on a physical server (not shown in FIG. 1) or in a cloud-based architecture and communicably coupled to the vehicle 110. In some embodiments, the system 120 may be integrated within the vehicle 110. In some embodiments, the system 120 may be implemented in a distributed manner, in that, one or more components of the system 120 may be implemented within the vehicle 110, while one or more components of the system 120 may be implemented within a cloud-based server or a physical server. Although the image capturing units 116, the ECU 112, the vehicle dynamics model 114, and the interface 118 are depicted as part of the vehicle 110, in some embodiments, one or more of the image capturing units 116, the ECU 112, the vehicle dynamics model 114, and the interface 118 may form a part of the system 120 without departing from the scope of the invention.

[0027] The system 120 may be configured to generate an under-body view of the vehicle and facilitate hazard detection, collision prevention, and diagnosis. The system 120 may be configured to perform operations and achieve the technical advantages by performing one or more operations as explained in detail at least with reference to FIGS. 1A-5E.

[0028] Reference is made to FIG. 2 which illustrates a detailed block diagram of the system 120, according to an embodiment of the present disclosure. The system 120 may be configured to receive and process images captured by the image capturing units associated with the vehicle 110. The system 120 may include a plurality of modules 201, a processor 202, an Input / Output (I / O) interface 203, a memory 204, and a transceiver 205. Further, in some embodiments where the system 120 is implemented as a standalone entity at a server / cloud architecture, the system 120 may be in communication with multiple vehicles, and the details provided below with respect to the system 120 and the vehicle 110 are applicable for the system 120 and the multiple vehicles as well.

[0029] In an exemplary embodiment, the processor 202 may be operatively coupled to each of the I / O interface 203, the plurality of modules 201, the transceiver 205, and the memory 204. In one embodiment, the processor 202 may include a graphical processing unit (GPU) and / or an Artificial Intelligence Engine (AIE). In one embodiment, the processor 202 may include at least one data processor for executing processes in virtual storage area network. The processor 202 may include specialized processing units such as, integrated system (bus) controllers, memory management control units, floating point units, graphics processing units, digital signal processing units, etc. In one embodiment, the processor 202 may include a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 202 may be one or more general processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now known or later developed devices for analyzing and processing data. The processor 202 may execute a software program, such as code generated manually (i.e., programmed) to perform the desired operation.

[0030] The processor 202 may be disposed in communication with one or more input / output (I / O) devices via the I / O interface 203. In some embodiments, the processor 202 may communicate with the vehicle 110 (for instance, with the ECU of the vehicle) using the I / O interface 203. In some embodiments, the I / O interface may be implemented within the vehicle 110 and may correspond to the interface 118 of the vehicle 110. The I / O interface 203 may employ communication code-division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long-term evolution (LTE), WiMax, or the like, etc.

[0031] Using the I / O interface 203, the system 120 may communicate with one or more I / O devices, specifically, the user devices being used to capture images. For example, the input device may be an antenna, microphone, touch screen, touchpad, storage device, transceiver, video device / source, etc. The output devices may be a video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, Plasma Display Panel (PDP), Organic light-emitting diode display (OLED) or the like), audio speaker, etc.

[0032] The processor 202 may be disposed in communication with a communication network via a network interface. In an embodiment, the network interface may be the I / O interface 203. The network interface may connect to the communication network to enable connection of the system 120 with the vehicle 110 and / or outside environment. The network interface may employ connection protocols including, without limitation, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.11a / b / g / n / x, etc. The communication network may include, without limitation, a direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Using the network interface and the communication network, the system 120 may communicate with other devices. The network interface may employ connection protocols including, but not limited to, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.11a / b / g / n / x, etc.

[0033] In some embodiments, the memory 204 may be communicatively coupled to the processor 202. The memory 204 may be configured to store data and instructions executable by the processor 202. In one embodiment, the memory 204 may be provided within vehicle 110. In another embodiment, the memory 204 may be provided within the system 120 being remote from the vehicle 110. In yet another embodiment, the memory 204 may communicate with the processor 202 via a bus within the system 120. In yet another embodiment, the memory 204 may be located remote from the processor 202, and may be in communication with the processor 202 via a network. The memory 204 may include, but not limited to, a non-transitory computer-readable storage media, such as various types of volatile and non-volatile storage media including, but not limited to, random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media and the like. In one example, the memory 204 may include a cache or random-access memory for the processor 202. In alternative examples, the memory 204 is separate from the processor 202, such as a cache memory of a processor, the system memory, or other memory. The memory 204 may be an external storage device or database for storing data. The memory 204 may be operable to store instructions executable by the processor 202. The functions, acts or tasks illustrated in the figures or described may be performed by the programmed processor 202 for executing the instructions stored in the memory 204. The functions, acts or tasks are independent of the particular type of instructions set, storage media, processor or processing strategy and may be performed by software, hardware, integrated circuits, firmware, micro-code and the like, operating alone or in combination. Likewise, processing strategies may include multiprocessing, multitasking, parallel processing, and the like.

[0034] In some embodiments, the memory 204 may include an image buffer 204A, a velocity buffer 204B, and a vehicle model unit 204C. The image buffer 204A and the velocity buffer 204B may be cyclic buffers. The vehicle model unit 204C may be configured to store a 3-dimensional (3D) model of the vehicle 110.

[0035] In some embodiments, the plurality of modules 201 may be included within the memory 204. The memory 204 may further include a database to store data. The plurality of modules 201 may include a set of instructions that may be executed to cause the system 120, in particular, the processor 202 of the system 120, to perform any one or more of the methods / processes disclosed herein. The plurality of modules 201 may be configured to perform the steps of the present disclosure using the data stored in the database. For instance, the plurality of modules 201 may be configured to perform the steps disclosed in Figs. 5A-5E. In an embodiment, each of the plurality of modules 201 may be a hardware unit which may be outside the memory 204. Further, the memory 204 may include an operating system for performing one or more tasks of the system 120, as performed by a generic operating system.

[0036] The transceiver 205 may be configured to receive and / or transmit signals to and from the vehicle 110. In one embodiment, the database may be configured to store the information as required by the plurality of modules 201 and the processor 202 to perform one or more functions as disclosed in Figs. 5A-5E.

[0037] The plurality of modules 201 may include, but not limited to, an image selector 210, a deblur module 212, an ortho-rectification module 214, a normalization module 216, an image stitching module 218, a depth module 220, a segmentation module 222, a hazard detector 224, a planner module 226, an image synthesizer 228, a collision predictor 230, a diagnostics module 232, and a visualizer 234. The plurality of modules 201 may be implemented by way of suitable hardware and / or software applications.

[0038] In some embodiments, at least one of the plurality of modules 201 may use an AI model. The AI model may also be used for enhancing the suggesting and recommending configurations for every subsequent image capture event. A function associated with AI may be performed through the non-volatile memory, the volatile memory, and the processor 202.

[0039] The processor 202 may include one or a plurality of processors. At this time, one or a plurality of processors may be a general purpose processor, such as a central processing unit (CPU), an application processor (AP), or the like, a graphics-only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and / or an AI-dedicated processor such as a neural processing unit (NPU).

[0040] The one or a plurality of processors control the processing of the input data in accordance with a predefined operating rule stored in the non-volatile memory or may employ a suitable artificial intelligence (AI) model executed from a server or a local memory module.

[0041] The AI model may consist of a plurality of neural network layers. Each layer has a plurality of weight values, and performs a layer operation through calculation of a previous layer and an operation of a plurality of weights. Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann Machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GAN), and deep Q-networks.

[0042] The learning technique is a method for training a predetermined target device (for example, a robot) using a plurality of learning data to cause, allow, or control the target device to make a determination or prediction. Examples of learning techniques include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, active learning and reinforcement learning. The processor 202 may perform pre-processing operations on the data to convert it into a form appropriate for use as an input for the artificial intelligence (AI) model.

[0043] Reasoning prediction is a technique of logically reasoning and predicting by determining information and includes, e.g., knowledge-based reasoning, optimization prediction, preference-based planning, or recommendation.

[0044] Further, the present disclosure contemplates a computer-readable medium that includes instructions or receives and executes instructions responsive to a propagated signal. Further, the instructions may be transmitted or received over the network via a communication port or interface or using a bus (not shown). The communication port or interface may be a part of the processor 202 or may be a separate component. The communication port may be created in software or may be a physical connection in hardware. The communication port may be configured to connect with a network, external media, the display, or any other components in system, or combinations thereof. The connection with the network may be a physical connection, such as a wired Ethernet connection or may be established wirelessly. Likewise, the additional connections with other components of the system 120 may be physical or may be established wirelessly. The network may alternatively be directly connected to a bus. For the sake of brevity, the architecture and standard operations of the memory 204, the processor 202, the transceiver 205, and the I / O interface 203 are not discussed in detail.

[0045] FIG. 3 illustrates a block diagram 300 depicting operational flow associated with the plurality of modules 201. It is appreciated that the details described with reference to the plurality of modules 201 may be performed in conjunction with the processor 202 and the memory 204. The details of the present invention will now be described by collectively referring to FIGS. 1A-3. It is appreciated that the term “stitching” and “fusing” are used interchangeably in the present disclosure. It is appreciated that the terms “pre-stored model” and “pre-stored view” are used interchangeably in the present disclosure.

[0046] In some embodiments, the processor 202 may be configured to determine whether a velocity of the vehicle 110 is lesser than a velocity threshold. The processor 202 may receive the velocity of the vehicle 110 from the ECU 112 of the vehicle 110. The operations by the plurality of modules 201 may be triggered when the processor 202 determines that the velocity of the vehicle 110 is lesser than a velocity threshold.

[0047] The image selector 210 may be configured to receive a plurality of images related to the terrain 130 being traversed by the vehicle 110. The terrain 130 may include at least one object 132, which may or may not form a hazardous element for the vehicle 110. The image selector 210 may be configured to receive the plurality of images from the image capturing units 116 of the vehicle 110. The plurality of images may be a series of images captured and sent sequentially. In an embodiment, the plurality of images may each be captured at pre-determined intervals. The plurality of images may be received at the image buffer 204A and the image selector 210 may retrieve the plurality of images from the image buffer 204A.

[0048] The image selector 210, the deblur module 212, the ortho-rectification module 214, and the normalization module 216 may each process the received plurality of images to generate a plurality of processed images. In some embodiments, the image selector 210, the deblur module 212, the ortho-rectification module 214, and the normalization module 216 may each process the plurality of images in a sequential manner, i.e., one image at a time, as the series of the plurality of images are received via the image buffer 204A.

[0049] The image selector 210 may be configured to select relevant images from the received plurality of images based on one or more vehicle dynamic parameters and one or more image parameters. That is, the image selector 210 may be configured to filter the received plurality of images to select a plurality of filtered images of the vehicle 110. The one or more vehicle dynamic parameters may be received from the vehicle dynamics model 114. The one or more vehicle dynamics parameters may include at least one of a velocity of the vehicle 110, an estimated trajectory of the vehicle 110, a vehicle steering parameter, a vehicle braking parameter, and a vehicle throttle parameter. The one or more image parameters may be indicative of the quality of the image. As an example, if an image from among the plurality of images is distorted, then the image will be rejected and not used for further processing. As another example, if the vehicle is moving in reverse direction, the images from rear camera of the image capturing unit 116 will be selected. Further, the one or more image parameters may include an overlapping between subsequent images from among the plurality of images. The image selector 210 may be configured to filter the plurality of images based on the overlapping, in that, the image selector 210 may be configured to reject or filter out images having a substantial overlap with one or more previous images in the sequence of the plurality of images.

[0050] The deblur module 212 may be configured to de-blur the plurality of filtered images to generate a plurality of de-blurred images. The deblur module 212 may be configured to reduce or remove blurriness in the plurality of filtered images which may be a result of the image capturing units 116 being in motion (such as, when the vehicle 110 is moving). The deblur module 212 may be configured to de-blur the plurality of filtered images based on one or more techniques including, but not limited to, Wiener deconvolution, Blind deconvolution, and Lucy-Richardson deconvolution.

[0051] The ortho-rectification module 214 may be configured to transform the plurality of de-blurred images and generate a plurality of ortho-rectified images. The ortho-rectification module 214 may be configured to correct geometric distortions which may be a result of viewing angle of the image capturing units 116. The ortho-rectification module 214 may be configured to transform the plurality of images perspective to a planar, map-like view where distance and angles are accurate and consistent.

[0052] The normalization module 216 may be configured to normalize the plurality of ortho-rectified images to generate the plurality of processed images. The normalization module 216 may be configured to remove or reduce variations in image brightness and contrast caused by differences in illumination, atmospheric condition, and / or settings of the image capturing units 116 when the image capturing units 116 capture the plurality of images. In some embodiments, the normalization module 216 may be configured to generate the plurality of processed images based on one or more of histogram equalization, image division, radiometric correction models, or image filters.

[0053] FIG. 4A illustrates an exemplary image, such as an image associated with the terrain 130 and at least one object 132 shown in FIG. 1B, being processed by the deblur module 212, the ortho-rectification module 214, and the normalization module 216. The image 401 may be received from the image selector 210. The image 401 may be processed by the deblur module 212 to generate the de-blurred image 402. The de-blurred image 402 may be processed by the ortho-rectification module 214 to generate the ortho-rectified image 403. The ortho-rectified image 403 may be processed by the normalization module 216 to generate the processed image 404.

[0054] Referring again to FIGS. 1A-3, the image stitching module 218 may be configured to generate a fused image, or a stitched image, by fusing the plurality of processed images based on the one or more vehicle dynamics parameters received from the vehicle dynamics model 114 and the velocity of the vehicle 110 received from the ECU 112 via the velocity buffer 204B. The fused image may be associated with the terrain 130 being traversed by the vehicle 110, in that, the fused image may indicate a view of the terrain 130 where the vehicle 110 is moving. It is appreciated that as the vehicle is moving, the image capturing units 116 may continuously capture the plurality of images which are processed and received at the image stitching module 218.

[0055] The image stitching module 218 may be configured stitch (fuse) a first image and a second image from among the plurality of processed images. The first image and the second image may be consecutive images received at the image stitching module 218. It is appreciated that although the details are provided with reference to first and second images, the image stitching module 218 may repeat the process for the successive images as well.

[0056] The image stitching module 218 may be configured to identify corresponding key-points for each of the first image and the second image from among the plurality of processed images. The second image may comprise an overlapping region with respect to the first image. The image stitching module 218 may identify the corresponding key-points based on ORB (Oriented FAST and Rotated BRIEF) feature detector. The corresponding key-points may be identified based on distinctive patterns in the first and second images.

[0057] The image stitching module 218 may be configured to determine binary descriptors associated with each of the corresponding key-points for each of the first image and the second image. The binary descriptions may be determined by use of the ORB descriptor technique. The binary descriptors may be indicative of unique characteristics of neighborhood of each corresponding key-points.

[0058] The image stitching module 218 may be configured to determine matched key-points based on a comparison of the determined binary descriptors for the first image and the second image. In an embodiment, the comparison of the determined binary descriptors may be performed based on a nearest neighbor search technique, such as, Brute-Force technique or Fast Library for Approximate Nearest Neighbors (FLANN) technique. The image stitching module 218 may be configured to filter false and / or unreliable matches. In some embodiments, the unreliable matches may be filtered based on filtering techniques such as Ratio Test and Random Sample Consensus (RANSAC).

[0059] Once the unreliable matches are filtered, a transformation (homograph) may be estimated for the remaining matched key-points. The transformation may indicate a geometric relationship between the first and second images taking into account translation, rotation, and scaling. That is, the image stitching module 218 may be configured to estimate a geometric relation between the first image and the second image based on the matched key-points.

[0060] The image stitching module 218 may be configured to merge the first image and the second image at the overlapping region. Based on the estimated geometric relation (homograph), one of the first and second images may be aligned with the other of the first and second images. Post alignment, the first and second images may be blended so as to generate the stitched image having a seamless transition at the overlapping region. As described above, the stitching process performed by the image stitching module 218 may be repeated for multiple subsequent images so as to continuously generate the stitched image, which may be a panoramic image. In some embodiments, the image stitching module 218 may be configured to adjust the stitched image, such as, by adjusting exposure, correcting color, or correcting alignment.

[0061] Based on the stitched image, one or more hazardous elements associated with the at least one object may be identified which is a hazard for the vehicle 110 and has a risk of collision with the vehicle 110. To identify the one or more hazardous elements, the depth module 220 may be configured to determine a stereo depth image corresponding to the stitched image. The stereo depth image may represent a 3D view of the stitched image, i.e., a 3D view of the terrain 130 associated with the stitched image. To generate the stereo depth image, the depth module 220 may be configured to determine an offset image corresponding to the stitched image, as shown by ‘Delay block’ in FIG. 3. The offset image may be time delayed with respect to the stitched image. Based on the stitched image and the offset image, the stereo depth image may be determined by the depth module 220. The depth module 220 may make use of temporal information captured in the sequence of images forming the stitched image to estimate the depth.

[0062] Further, the segmentation module 222 may be configured to detect, based on the stitched image, segmentation information associated with the at least one object 132 on the terrain 130 being traversed by the vehicle 110. The segmentation information may indicate semantics regarding the at least one object 132, such as, the number of objects on the terrain 130. In an embodiment, the segmentation information may be represented in a 2D image.

[0063] FIG. 4B illustrates an exemplary image 405 being analyzed by the depth module 220 and the segmentation module 222. The image 405 may correspond to a stitched image which may be generated by the image stitching module 218 based on the processed images 404 (shown in FIG. 4A) being received from the normalization module 216. The image 405 may be analyzed by the depth module 220 to generate the stereo depth image 406, which may represent a 3D view of the stitched image. The image 405 may also be analyzed by the segmentation module 222 to determine segmentation information regarding the at least one object 132, the segmentation information may be displayed in a 2D image 407. As depicted the segmentation information may refer to semantics of the at least one object 132 on the terrain 130.

[0064] The hazard detector 224 may be configured to process the stereo depth image and the segmentation information to identify the one or more hazardous elements associated with the at least one object 132. Based on the stereo depth image and the segmentation information, dimensions of each of the at least one object 132 on the terrain 130 may be determined. The hazard detector 224 may further be configured to receive a pre-stored view or a pre-stored model of the vehicle 110, which may be a 3D model of the vehicle 110 stored in the vehicle model unit 204C. The pre-stored view may indicate dimensions of various components of the vehicle 110. For instance, the pre-stored view may indicate protruding components under chassis of the vehicle 110, the ground clearance of the vehicle 110, and the associated dimensions.

[0065] The hazard detector 224 may thus identify the one or more hazardous elements associated with the at least one object 132 based on the generated stitched image and the pre-stored model of the vehicle 110. The one or more hazardous elements may be the at least one object 132 as a whole or one or more parts of the at least one object 132. The one or more hazardous elements may refer to elements which may collide with one or more protruding components of an under-body of the vehicle 110.

[0066] Further, the planner module 226 is configured to determine a suggested trajectory for the vehicle 110 based on the one or more hazardous elements detected by the hazard detector 224 and based on the one or more vehicle dynamics parameters received from the vehicle dynamics model 114. In some embodiments, the hazard detector 224 and / or the planner module 226 may be configured to predict a probability of collision of the one or more hazardous elements with the one or more protruding components of the under-body of the vehicle 110. The prediction regarding the probability of collision may be determined based on the one or more hazardous elements, the pre-stored model of the vehicle, the one or more vehicle dynamics parameters, and external factors such as edges / boundaries of terrain. Based on the prediction, the planner module 226 may be configured to estimate the vehicle trajectory, i.e., the suggested trajectory to prevent collision with the one or more hazardous elements.

[0067] Once the one or more hazardous elements are identified and the suggested vehicle trajectory is estimated, the image synthesizer 228 may be configured to generate a final image indicative of the under-body view of the vehicle 110. The final image may be indicative of the under-body view of the vehicle 110 and may be a 3D image indicating at least one of the identified one or more hazardous elements, one or more protruding components associated with the under-body of the vehicle 110, and the terrain 130 being traversed by the vehicle 110. In order to generate the final image, the image synthesizer 228 may take into account the stitched image, the identified one or more hazardous elements, the pre-stored model of the vehicle, and the one or more vehicle dynamics parameters. The final image may be a real-time image with identified hazardous elements and protruding components highlighted along with terrain information and vehicle information (such as, vehicle wheel positioning). The final image may enable a user within the vehicle to be aware of the hazardous elements and take preventive measures as per suggested trajectory to avoid collision with the hazardous elements.

[0068] In an embodiment, the final image may be displayed by the visualizer 234 on the interface 118 associated with the vehicle 110. In an embodiment, the estimated vehicle trajectory may also be displayed by the visualizer 234 on the interface 118 in conjunction with the final image.

[0069] Referring to FIG. 4C, an object 412 (corresponding to the at least one object 132 shown in FIG. 1B) is present on the terrain 130. Based on the pre-stored model 414 of the vehicle 110, the stereo-depth image (such as image 406 shown in FIG. 4B), and the segmentation information (such as the information shown in image 407 in FIG. 4B), the hazard detector 224 may determine that portions 412A and 412B of the object 412 are the one or more hazardous elements which may collide with one or more protruding components 416 of the vehicle 110. The hazard detector 224 may determine the dimensions of the object 412 and specific parts (e.g., 412A, 412B) of the object based on the stereo-depth image and the segmentation information. Further, the hazard detector 224 may have the information regarding the protruding components 416 of the vehicle 110 based on the pre-stored model. Accordingly, the hazard detector 224 is able to determine the one or more hazardous elements, in this case 412A and 412B, which may collide with the one or more protruding components 416 of the vehicle 110. Further, the image synthesizer 228 may generate the final image 418 that depicts the pre-stored model 414 of the vehicle 110, the protruding components 416 of the under-body of the vehicle 110, the at least one object 412, and the hazardous elements 412A, 412B which pose a risk of collision with the vehicle 110. The final image 418 may be displayed on the interface 118 of the vehicle 110 by the visualizer 234, thereby allowing the user within the vehicle to be aware of the hazardous elements 412A, 412B as well as possible components 416 which may come in contact with the hazardous elements 412A, 412B.

[0070] In other examples, other portions of the object 412 may also be determined as the one or more hazardous elements, the majority or entirety of the object 412 may also be determined as the one or more hazardous elements, and / or additional objects (not shown in FIG. 4B) may be determined as the one or more hazardous elements.

[0071] As described above, the estimated vehicle trajectory may also be displayed by the visualizer 234 on the interface 118 in conjunction with the final image. Referring to FIG. 4D, the image synthesizer 228 generates the final image 418 (corresponding to image 418 shown in FIG. 4C). The final image 418 may be generated by the image synthesizer 228 based on the inputs from the hazard detector 224, the planner module 226, the image stitching module 218, and pre-stored model of the vehicle 110. The final image 418 may represent the pre-stored model 414 of the vehicle 110, the protruding components 416 on the under-body of the vehicle 110, the terrain 130 being traversed by the vehicle, the at least one object 132 on the terrain 130 associated with the one or more hazardous elements (shown in FIG. 4C), and the suggested trajectory 420 for the vehicle 110 to avoid collision with the one or more hazardous elements.

[0072] Referring again to FIGS. 1A-3, the collision predictor 230 may be configured to process the final image generated by the image synthesizer 228. The collision predictor 230 may be configured to detect collision of the one or more hazardous elements with one or more protruding components associated with the under-body of the vehicle. In some embodiments, the collision predictor 230 may detect collision when the vehicle 110 does not travel on the suggested trajectory determined by the planner module 226. The collision predictor 230 may be configured to detect an overlap of the one or more protruding components with the one or more hazardous elements, thereby detecting the collision. In order to detect collision, the collision predictor 230 may take into account the dimensions of the one or more hazardous elements, the dimensions of the one or more protruding components, and the trajectory suggested by the planner module 226.

[0073] The collision predictor 230 may further be configured to estimate the location of the detected collision. Further, the collision predictor 230 may be configured to identify, based on the estimated location, probable components of the one or more protruding components that are damaged by the collision. For instance, the collision predictor 230 may be configured to determine the location of the collision based on a trajectory followed by the vehicle 110. As another example, the collision predictor 230 may be configured to determine which protruding component(s) from among the one or more protruding components overlaps with the one or more hazardous elements, and determine that protruding component is the probable component damaged by the collision. Further, the location of the detected collision may correspond to the location of the probable component.

[0074] The diagnostics module 232 may be configured to run diagnostics on the identified probable components in order to detect any issues caused by the detected collision. The diagnostics module 232 may trigger a corresponding diagnosis for probable components and determine a result of the corresponding diagnosis. The result of the corresponding diagnosis may indicate whether the probable component is damaged, the extent of the damage, and recovery measures which can be taken for the damage.

[0075] The diagnostics module 232 may further be configured to transmit an alert based on the result of the corresponding diagnosis. The alert may be sent to the visualizer 234 and the visualizer 234 may be configured to display the alert on the interface 118 of the vehicle 110. As mentioned previously, the alert is indicative of one or more of the extent of damage to the probable components and recovery measures associated with the probable components.

[0076] Referring to FIG. 4E, an exemplary final image 422 generated by the image synthesizer 228 is depicted. As seen in FIG. 4E, the object 412 comprises portions 412A, 412B (also shown in FIG. 4C) which are determined as the hazardous elements. However, in FIG. 4E, the trajectory of the vehicle 110 may be such that the collision predictor 230 determines that protruding component 416A from among the protruding components 416 may be impacted by the portions 412A of the object 412. The collision predictor 230 may determine the location of the collision and the identify the protruding component 416A impacted by the hazardous element 412A. Further, the diagnostics module 232 may be configured to trigger diagnosis for the protruding component 416A and determine the result of the diagnosis. The diagnostics module 232 may further trigger an alarm indicative of the result of the diagnosis, which may include extent of damage to the protruding component 416A and recovery measures for the caused damage. As an example, the protruding component 416A may be an oil tank of the vehicle 110 and the diagnostics module 232 may determine a decrease in oil level as a result of the collision. The diagnostics module 232 may trigger an alert regarding the damage and the alert may be displayed to the user associated with the vehicle 110 along with possible recovery measures.

[0077] FIG. 5A illustrates an exemplary process flow of a method 500 for generating an under-body view for the vehicle 110, according to an embodiment of the present disclosure. In one embodiment, the steps of the method 500 may be performed by the system 120, for instance, by the processor 202 of the system 120 in conjunction with the plurality of modules 201 and the memory 204, which may be integrated within the vehicle 110 or provided separately and are operatively coupled.

[0078] At step 502, the method 500 includes receiving, from one or more image capturing units associated with the vehicle, a plurality of images related to a terrain being traversed by the vehicle, wherein the terrain includes at least one object.

[0079] At step 504, the method 500 includes generating a fused image by fusing the plurality of images based on one or more vehicle dynamics parameters associated with the vehicle, the fused image being associated with the terrain being traversed by the vehicle.

[0080] At step 506, the method 500 includes generating an under-body view of the vehicle including the fused image, the at least one object, and a pre-stored view of the vehicle.

[0081] In some embodiments, the method 500 may further include generating a plurality of processed images based on processing of the plurality of received images. In some embodiments, to generate the plurality of processed images, the method 500 may comprise steps 508-514 as illustrated in FIG. 5B. At step 508, the method 500 comprises filtering, based on the one or more vehicle dynamics parameters and one or more image parameters, the received plurality of images to select a plurality of filtered images. At step 510, the method 500 comprises de-blurring the plurality of filtered images to generate a plurality of de-blurred images. At step 512, the method 500 comprises transforming the plurality of de-blurred images to generate a plurality of ortho-rectified images. At step 514, the method 500 comprises normalizing the plurality of ortho-rectified images to generate the plurality of processed images. In some embodiments, generating the fused image comprises fusing the plurality of processed images in order to generate the fused image.

[0082] In some embodiments, to generate the fused image, the method 500 may comprise steps 504A-504E as illustrated in FIG. 5C. At step 504A, the method 500 comprises identifying corresponding key-points for each of a first image and a second image from among the plurality of images. The second image comprises an overlapping region with respect to the first image. At step 504B, the method 500 comprises determining binary descriptors associated with each of the corresponding key-points for each of the first image and the second image. At step 504C, the method 500 comprises determining matched key-points based on a comparison of the determined binary descriptors for the first image and the second image. At step 504D, the method 500 comprises estimating, based on the matched key-points, a geometric relation between the first image and the second image. At step 504E, the method 500 comprises merging, at the overlapping region, the first image and the second image based on the estimated geometric relation to generate the fused image.

[0083] In some embodiments, the method 500 may further include identifying one or more hazardous elements associated with the at least one object based on the generated fused image and a pre-stored view of the vehicle. In some embodiments, to identify the one or more hazardous elements, the method 500 comprises steps 516-522 as illustrated in FIG. 5D. At step 516, the method 500 comprises determining an offset image corresponding to the fused image, the offset image being time delayed with respect to the fused image. At step 518, the method 500 comprises determining a stereo depth image based on the fused image and the determined offset image. At step 520, the method 500 comprises detecting, based on the fused image, segmentation information associated with the at least one object on the terrain being traversed by the vehicle. At step 522, the method 500 comprises identifying the one or more hazardous elements associated with the at least one object based on the stereo depth image and the segmentation information.

[0084] In some embodiments, the method 500 may further comprise step 524-528 as illustrated in FIG. 5E. At step 524, the method 500 comprises predicting, based on the one or more hazardous elements, the pre-stored view of the vehicle, and the one or more vehicle dynamics parameters, a probability of collision of the one or more hazardous elements and one or more protruding components associated with an under-body of the vehicle. At step 526, the method comprises estimating, based on the prediction, a vehicle trajectory to prevent collision with the one or more hazardous elements. At step 528, the method 500 comprises displaying the estimated vehicle trajectory in conjunction with the under-body view of the vehicle on a user interface of the vehicle.

[0085] While the above discussed steps in FIGS. 5A-5E are shown and described in a particular sequence, the steps may occur in variations to the sequence in accordance with various embodiments. Further, a detailed description related to the various steps of FIGS. 5A-5E is already covered in the description related to FIGS. 1A-4E and is omitted herein for the sake of brevity.

[0086] The present disclosure provides for various technical advancements based on the key features discussed above. The present invention assists a driver of the vehicle to avoid collision and / or minimize damage from hazards while driving the vehicle. In case of collisions, the driver can be alerted regarding the location of collision, probable components that may have been affected, diagnosis of the damage to the components, and possible recovery measures and precautions that can be taken. A 3D under-body view enables drivers to accurately avoid the tyres and protruding under chassis parts from colliding with the hazards through real-time visual feedback and guidance in vehicle cabin. Moreover, in case of collision, drivers can be altered regarding the impact, damage, and precautions. Drivers are thus not only enabled to avoid collisions and avoid damage to vehicle, but also are made aware of damage location and recovery measures. Using the present invention, drivers can easily manoeuvre the vehicle to avoid hazards proactively. Further, no hardware modifications and additions are required to implement the present invention. With minimal image processing computations, drivers can be provided with alerts and avoidance cues. Vehicle safety is thus improved by prognostics and diagnostic measures.

[0087] While specific language has been used to describe the present subject matter, any limitations arising on account thereto, are not intended. As would be apparent to a person in the art, various working modifications may be made to the method in order to implement the inventive concept as taught herein. The drawings and the foregoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment.

Claims

1.A method for generating an under-body view for a vehicle, the method comprising:receiving, from one or more image capturing units associated with the vehicle, a plurality of images related to a terrain being traversed by the vehicle, wherein the terrain includes at least one object;generating a fused image by fusing the plurality of images based on one or more vehicle dynamics parameters associated with the vehicle, the fused image being associated with the terrain being traversed by the vehicle; andgenerating an under-body view of the vehicle including the fused image, the at least one object, and a pre-stored view of the vehicle.2.The method as claimed in claim 1, comprising:displaying the under-body view of the vehicle on a user interface associated with the vehicle.3.The method as claimed in claim 1, comprising generating a plurality of processed images based on processing of the plurality of received images, wherein generating the plurality of processed images comprises:filtering, based on the one or more vehicle dynamics parameters and one or more image parameters, the received plurality of images to select a plurality of filtered images;de-blurring the plurality of filtered images to generate a plurality of de-blurred images;transforming the plurality of de-blurred images to generate a plurality of ortho-rectified images; andnormalizing the plurality of ortho-rectified images to generate the plurality of processed images, wherein generating the fused image comprises fusing the plurality of processed images in order to generate the fused image.4.The method as claimed in claim 1, comprising identifying one or more hazardous elements associated with the at least one object based on the generated fused image and a pre-stored view of the vehicle, wherein identifying the one or more hazardous elements associated with the at least one object comprises:determining an offset image corresponding to the fused image, the offset image being time delayed with respect to the fused image;determining a stereo depth image based on the fused image and the determined offset image;detecting, based on the fused image, segmentation information associated with the at least one object on the terrain being traversed by the vehicle; andidentifying the one or more hazardous elements associated with the at least one object based on the stereo depth image and the segmentation information.5.The method as claimed in claim 4, wherein the under-body view of the vehicle is a three-dimensional (3D) image indicating at least one of the identified one or more hazardous elements, one or more protruding components associated with an under-body of the vehicle, and the terrain being traversed by the vehicle.6.The method as claimed in claim 1, wherein generating the under-body view of the vehicle comprises generating the under-body view based on the one or more vehicle dynamics parameters, wherein the one or more vehicle dynamics parameters include at least one of a velocity of the vehicle, an estimated trajectory of the vehicle, a vehicle steering parameter, a vehicle braking parameter, and a vehicle throttle parameter.7.The method as claimed in claim 1, wherein generating the fused image by fusing the plurality of images comprises:identifying corresponding key-points for each of a first image and a second image from among the plurality of images, wherein the second image comprises an overlapping region with respect to the first image;determining binary descriptors associated with each of the corresponding key-points for each of the first image and the second image;determining matched key-points based on a comparison of the determined binary descriptors for the first image and the second image;estimating, based on the matched key-points, a geometric relation between the first image and the second image; andmerging, at the overlapping region, the first image and the second image based on the estimated geometric relation to generate the fused image.8.The method as claimed in claim 4, comprising:predicting, based on the one or more hazardous elements, the pre-stored view of the vehicle, and the one or more vehicle dynamics parameters, a probability of collision of the one or more hazardous elements and one or more protruding components associated with an under-body of the vehicle;estimating, based on the prediction, a vehicle trajectory to prevent collision with the one or more hazardous elements; anddisplaying the estimated vehicle trajectory in conjunction with the under-body view of the vehicle on a user interface of the vehicle.9.The method as claimed in claim 4, comprising:detecting collision of the one or more hazardous elements with one or more protruding components associated with an under-body of the vehicle;estimating a location of the detected collision; andidentifying, based on the estimated location, probable components of the one or more protruding components that are damaged by the collision.10.The method as claimed in claim 9, comprising:triggering, based on the identified probable components, a corresponding diagnosis for the probable components; andtransmitting an alert to be displayed on a user interface of the vehicle based on a result of the corresponding diagnosis, the alert being indicative of one or more of an extent of damage to the probable components and recovery measures associated with the probable components.11.A system to generate an under-body view for a vehicle, the system comprising:a memory; anda processor communicatively coupled with the memory, the processor being configured to:receive, from one or more image capturing units associated with the vehicle, a plurality of images related to a terrain being traversed by the vehicle, wherein the terrain includes at least one object;generate a fused image by fusing the plurality of images based on one or more vehicle dynamics parameters associated with the vehicle, the fused image being associated with the terrain being traversed by the vehicle; andgenerate an under-body view of the vehicle including the fused image, the at least one object, and a pre-stored view of the vehicle.12.The system as claimed in claim 11, wherein the processor is configured to:display the under-body view of the vehicle on a user interface associated with the vehicle.13.The system as claimed in claim 11, wherein the processor is configured to generate a plurality of processed images based on processing of the plurality of received images, wherein to generate the plurality of processed images, the processor is configured to:filter, based on the one or more vehicle dynamics parameters and one or more image parameters, the received plurality of images to select a plurality of filtered images;de-blur the plurality of filtered images to generate a plurality of de-blurred images;transform the plurality of de-blurred images to generate a plurality of ortho-rectified images; andnormalize the plurality of ortho-rectified images to generate the plurality of processed images, wherein to generate the fused image, the processor is configured to fuse the plurality of processed images.14.The system as claimed in claim 11, wherein the processor is configured to identify one or more hazardous elements associated with the at least one object based on the generated fused image and a pre-stored view of the vehicle, wherein to identify the one or more hazardous elements associated with the at least one object, the processor is configured to:determine an offset image corresponding to the fused image, the offset image being time delayed with respect to the fused image;determine a stereo depth image based on the fused image and the determined offset image;detect, based on the fused image, segmentation information associated with the at least one object on the terrain being traversed by the vehicle; andidentify the one or more hazardous elements associated with the at least one object based on the stereo depth image and the segmentation information.15.The system as claimed in claim 14, wherein the under-body view of the vehicle is a three-dimensional (3D) image indicating at least one of the identified one or more hazardous elements, one or more protruding components associated with an under-body of the vehicle, and the terrain being traversed by the vehicle.

Images