Cross-traffic detection and alert in an fdm system

Abstract

An environmental monitoring system for a vehicle includes at least one imager module having a field of view configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary that has a width smaller than the field of view. A control system is configured to generate integral images, the integral images including segmentations each with an input pixel data. The integral images are then reviewed for the presence of an object within the field of view but outside the outer graphic boundary. If the object is within the field of view but outside the outer graphic boundary, generates an alert to a user.

Description

CROSS-TRAFFIC DETECTION AND ALERT IN AN FPM SYSTEM FIELD OF THE DISCLOSURE

[0001] The present disclosure generally relates to an environmental monitoring system for a vehicle that monitors the environment relative to an associated vehicle for the presence of an object.SUMMARY OF THE DISCLOSURE

[0002] According to one aspect of the present disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary that has a width smaller than the field of view. A control system is configured to generate integral images from the captured images, the integral images including segmentations, each with an input pixel data. The integral images are then reviewed for the presence of an object within the field of view but outside the outer graphic boundary. If the object is within the field of view but outside the outer graphic boundary, generate an alert to a user.

[0003] According to another aspect of the present disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary. A control system is configured to generate integral images from the captured images, the integral images including segmentations, each with an input pixel data. The integral images are then reviewed for the presence of an object within the field of view. If the object is within the field of view, determine an initial size of the object on at least one of the images or the graphic. After a predetermined period, the control system is further caused to determine a subsequent size of the object on at least one of a subsequent one of the integral images or a subsequent graphic, determine if the object is moving, and, if the object is not moving, extrapolate an approximate distance of the object based on differences between the initial size and the subsequent size.

[0004] According to yet another aspect of the present disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of viewand configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary that has a width smaller than the field of view. A control system is configured to generate integral images from the captured images, the integral images including segmentations each with an input pixel data. The integral images are then reviewed for the presence of an object and relative location of the object. If the object is detected proximate a path of travel, moving in a direction of the path of travel of the vehicle, or the vehicle is moving in a direction of the object generate an alert to a user.

[0005] According to various aspects of the disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary that has a width smaller than the field of view. In this manner, objects captured by the imager module may not be shown on the display. As such, a control system of the environmental monitoring system may utilize the computational efficiency of integral images from images captured by the imager module to detect objects outside of the outer graphic boundary.

[0006] These and other features, advantages, and objects of the present disclosure will be further understood and appreciated by those skilled in the art by reference to the following specification, claims, and appended drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In the drawings:

[0008] FIG. 1 is a top plan view of a vehicle in an environment utilizing an environmental monitoring system, according to an aspect of the disclosure;

[0009] FIG. 2A is a front view of a full display mirror with a warning indicia generated on a display module of cross-traffic relative to a vehicle, according to an aspect of the disclosure;

[0010] FIG. 2B is a front view of a full display mirror with a generated graphic on a display module of cross-traffic relative to a vehicle, according to an aspect of the disclosure;

[0011] FIG. 3A is a schematic view of an image segmented into a grid pattern with input pixel values, according to an aspect of the disclosure;

[0012] FIG. 3B is a schematic view of an integral image segmented into a grid pattern with input pixel value sums, according to an aspect of the disclosure;

[0013] FIG. 3C is a partially schematic front view of an integral image segmented into a grid pattern, according to an aspect of the disclosure;

[0014] FIG. 3D is a partially schematic front view of an integral image segmented into vertical columns, according to an aspect of the disclosure;

[0015] FIG. 4 is a schematic view of an environmental monitoring system extrapolating an approximate distance of an object based on an integral image, according to an aspect of the disclosure; and

[0016] FIG. 5 is a schematic view of a control system of an environmental monitoring system, according to an aspect of the present disclosure.DETAILED DESCRIPTION

[0017] The present illustrated embodiments reside primarily in combinations of method steps and apparatus components related to an environmental monitoring system for a vehicle that monitors the environment relative to an associated vehicle for the presence of an object. Accordingly, the apparatus components and method steps have been represented, where appropriate, by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present disclosure so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. Further, like numerals in the description and drawings represent like elements.

[0018] For purposes of description herein, the terms "upper," "lower," "right," "left," "rear," "front," "vertical," "horizontal," and derivatives thereof, shall relate to the disclosure as oriented in FIG. 1. Unless stated otherwise, the term "front" shall refer to the surface of the device closer to an intended viewer of the device, and the term "rear" shall refer to the surface of the device further from the intended viewer of the device. However, it is to be understood that the disclosure may assume various alternative orientations, except where expressly specified to the contrary. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments of the inventive concepts defined in the appended claims.Hence, specific dimensions and other physical characteristics relating to the embodiments disclosed herein are not to be considered as limiting, unless the claims expressly state otherwise.

[0019] The terms "including," "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "comprises a . . . " does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0020] Referring initially to FIGS. 1-3B, reference numeral 10 generally designates an environmental monitoring system of a vehicle 12. The environmental monitoring system 10 includes at least one imager module 14 having a field of view 19 and configured to capture a series of images 16A of an environment 18 defined by the field of view 19 relative to (e.g., outside of and / or behind, etc.) an associated vehicle 12. A display module 20 is configured to generate a graphic 22 of the images 16A and defines an outer graphic boundary 24 that has a width smaller than the field of view 19. A control system 100 under some implementations may include a processor 104 and a memory 106 or, alternatively, one or more integrated controllers. In some implementations, the memory 106 includes instructions that, when executed by the processor 104, cause the processor 104 to perform various steps. In some implementations, an integrated circuit includes architecture to perform the various steps. More particularly, the various steps may include generating integral images 16B from the captured images 16A, the integral images 16B including segmentations S each with an input pixel data. The integral images 16B are reviewed for the presence of an object 26 within the field of view 19 but outside the outer graphic boundary 24. If the object 26 is within the field of view 19 but outside the graphic boundary 24, an alert 28 is generated to a user.

[0021] With reference now to FIG. 1, throughout traveling, the vehicle 12 oftentimes needs to travel backwards, out of driveways, parking spots, and any environment 18 where object 26 activity is pervasive. In addition, the vehicle 12 may also be traveling forwards, while one or more objects 26 may be in a blind spot. For example, while the vehicle 12 travels forwards, another vehicle (e.g., the object 26) may be traveling forwards at a faster rate and approaching a rear of the vehicle 12. In other examples, while the vehicle 12 travels forwards,another vehicle or other object 26 may be stationary or traveling at the same speed and merging into a blind spot. While the driver can typically rely on the images 16A captured by the one or more imager modules 14 in detecting the activities on the display module 20, the generated graphic 22 may have a limited graphic boundary 24 that typically cannot capture all activity within the field of view 19. For example, the at least one imager module 14 may include a wide-angle lens configured such that the field of view 19 is larger (e.g., wider) than ideal for a user to accurately monitor during traveling. For example, the field of view 19 may be 120° or more and the generated graphic 22 cannot adequately relay all the information without shrinking the environment 18 and the objects 26 within the environment 18 until the sizes in the generated graphic 22 are smaller than ideal for a user's vision. Indeed, a user must be able to rely on the generated graphic without undue attention while still operating the vehicle 12. In this manner, the control system 100 may monitor regions of the images 16A that are outside of the outer graphic boundary 24 for the presence of objects 26 in order to identify and relay information (e.g., via the alert 28) to the user (e.g., the driver). For example, the closer the object 26 is to a rear 30 of the vehicle 12 the more likely that the object 26 will be outside of the outer graphic boundary 24. In some implementations, in order to save processing power and increase efficiency of the control system 100, the integral image 16B can be utilized, the principles and methods will be discussed further in reference to FIGS. 3A and 3B.

[0022] However, it should be appreciated that, without departing from the scope of the subject disclosure, the environmental monitoring system 10 may also be utilized for monitoring activities that are currently displayed on the display module 20 (e.g., on the graphic 22 within the outer graphic boundary 24). More particularly, in some implementations, the field of view 19 may substantially match the outer graphic boundary 24 or the object 26 may otherwise be detected within the generated graphic boundary 24. Regardless of if the field of view 19 substantially matches or is larger than the outer graphic boundary 24, the methodologies described here may be utilized for detection of the object 26, whether the object 26 is in motion, and if the object 26 is moving towards, or has the possibility of moving towards, a path of travel of the vehicle ("PoT"). The methodologies described here may be utilized for detection of the object 26 and if the path of travel of the vehicle 12 or some other indicator (e.g., a turn signal) indicates the possibility of the path of travel of the vehicle 12 moving towards the object 26. In this manner, the control system 100may be configured to generate the integral images 16B from the captured images 16A, the integral images 16B including segmentations S each with an input pixel data. The integral images 16B are reviewed for the presence of an object 26 within the field of view 19 only or within the field of view 19 and the outer graphic boundary 24. If the object 26 is detected, the alert 28 (e.g., via the control system 100) is generated to a user. Generally speaking, the integral image 16B can be utilized in any implementation and / or any configuration of the control system 100 in order to save processing power and increase efficiency of the control system 100.

[0023] With continued reference to FIG. 1, the at least one imager module 14 may include one, two, or more imager modules 14 located around an interior cabin 32 or an exterior of the vehicle 12. For example, the at least one imager module 14 may be located on the rear 30 of the vehicle 12, side mirrors 34 of the vehicle 12, a top surface 36 of the vehicle 12, a full display mirror assembly 38 (e.g., a full display rearview mirror assembly) of the vehicle 12, and / or combinations thereof. If more than one imager module 14 is utilized, the control system 100 may be configured to, upon detecting the object 26 with one of the imager modules 14, generating a graphic 22 of images 16A associated with that imager module 14. For example, the control system 100 may be configured to automatically switch to the graphic 22 of the images 16A associated with the imager module 14 with an object 26 detected within the field of view 19. In some implementations, the control system 100 may be configured to generate a recommendation to switch to the graphic 22 of the images 16A associated with the imager module 14 with an object 26 detected within the field of view 19. Regardless of if switching the graphic 22 is automatic or recommended, the alert 28 may be generated. In some scenarios, upon detecting the object 26 with two of the imager modules 14, the control system 100 may prioritize which imager module 14 to utilize on the generated graphic 22. For example, if the object 26 is detected in the field of view 19 only (e.g., outside of the outer graphic boundary 24) from one imager module 14, but detected in both the field of view and within outer graphic boundary 24 with another imager module 14, the control system 100 may recommend and / or automatically switch to the imager module 14 where the object is also detected within the outer graphic boundary 24.

[0024] With continued reference to FIG. 1, in some implementations, the display module 20 may be located in the full display mirror assembly 38. In other implementations, the display module 20 may be located anywhere within the vehicle 12 (e.g., a center module, a heads-updisplay, a mobile device, and / or the like). Regardless of the locations of the at least one imager module 14 and the display module 20, the control system (e.g., the processor 104) may be configured to determine if the object 26 is moving towards a path of travel of the vehicle 12 or the path of the vehicle 12 is moving towards the object 26. In some implementations, when the field of view 19 is larger than the outer graphic boundary 24, the control system (e.g., the processor 104) may be configured to determine if the object 26 is moving towards the outer graphic boundary 24 (e.g., the vehicle 12) before generating the alert 28, or, alternatively, before generating a second or subsequent alert 28 that is different than the alert 28. However, as previously noted, the control system 100 may more generally determine if the object 26 is in, proximate (e.g., within a predetermined distance of 20 feet or less, 10 feet or less, 5 feet or less), or moving towards the path of travel of the vehicle 12 or the path of the vehicle 12 is moving towards the object 26 before generating the alert 28. In some implementations, the control system 100 may generate the first alert 28 if the object 26 is within the predetermined distance and generate the second and subsequent alert 28 if the object 26 is moving towards the outer graphic boundary 24.

[0025] For purposes of the disclosure, when an object is moving towards the outer graphic boundary 24, the object 26 itself may be moving towards the outer graphic boundary 24 as defined within the field of view 19, the vehicle 12 may be moving towards the object 26 that causes the outer graphic boundary 24 as defined within the field of view 19 to move towards the object 26, and / or both the object 26 and the vehicle 12 may be in relative motion. In some implementations, the alert 28 (e.g., initial or subsequent) may be associated with a prediction that the object 26 itself may be about to move towards the outer graphic boundary 24 as defined within the field of view 19, the vehicle 12 may be about to move towards the object 26 that causes the outer graphic boundary 24 as defined within the field of view 19 to move towards the object 26, and / or both the object 26 and the vehicle 12 may be in relative motion towards one another. For example, the control system 100 may be configured to determine if the vehicle 12 is about to turn (e.g., through a vehicle control system 150) based, for example, on one or more of a turn signal activation, GPS directions to turn into the path of the object 26, steering wheel rotation, orientation of the wheels or steering gear, and / or the like. Likewise, when the object 26 is another vehicle, the control system 100 may be configured to detect turn signals indicating that the other vehicle is about to turn into the path of travel.

[0026] With reference now to FIGS. 2A-3D, the images 16A and integral images 16B are depicted. More particularly, FIG. 2A depicts a first image 16A prior to the object 26 (i.e., a vehicle) being within the graphic boundary 24 but subsequent of the object 26 being captured by the field of view 19 of the imager module 14. Because the object 26 is detected (e.g., within the field of view 19 but outside the outer graphic boundary 24 or within the field of view 19 and inside the outer graphic boundary 24), the alert 28 is generated to the user. In some embodiments, the alert 28 may be audible, for example, via a sound module in the fully display mirror assembly 38 or a vehicular audio system (e.g. through the vehicle control system 150). In some embodiments, the alert 28 may be haptic, for example, via vibration of a steering wheel, a seat, or the full display mirror assembly 38. In some embodiments, the alert 28 may be visual, for example, via generating a graphic on the display module 20. For example, the graphic may be generated on a side of the display module 20 associated in a direction of relative positioning of the object 26. Further, the graphic may be generated as an arrow pointing in a direction of relative travel and / or location of the object 26. In some embodiments, the alert 28 may include two or more of audible, haptic, and visual ques. In some embodiments, the alert 28 may include an initial alert (e.g., a first audible, haptic, and / or visual que) when the object 26 is detected and a second or subsequent alert (e.g., a second audible, haptic, and / or visual que) if the object 26 is moving into the path of travel of the vehicle 12 and / or the vehicle 12 moves within a predetermined distance of the object 26. In some implementations, sounds, haptic, and / or visual ques may be different between the first alert 28 and the subsequent alert 28. For example, the first alert 28 may include one of the audible, haptic, and / or visual ques and the second alert 28 may include a different one of the audible, haptic, and / or visual ques. In other implementations, the first alert 28 may include one of the audible, haptic, and / or visual ques and the second alert 28 may include a two or more of the audible, haptic, and / or visual ques. In some implementations, the first alert 28 may include one of the audible, haptic, and / or visual ques and the second alert 28 may the same of the audible, haptic, and / or visual ques, but at louder volumes, more intense vibrations, and / or larger or different (e.g., more vibrant) colors. FIG. 2B depicts a second image 16A after the object 26 is within the graphic boundary 24 as previously described, the alert 28 may be generated regardless of if the object 26 is detected outside or within the outer graphic boundary 24 based on the objects 26 proximity to the path of travel or the objects 26 movement towards the path of travel.

[0027] With reference now to FIGS. 3A-3C, to save processing power and increase efficiency of the control system 100, the integral image 16B can be generated and reviewed (e.g., via the control system 100) for increased computational efficiency. The image 16A is a monochrome image with measurable image intensity. The integral image 16B is derived (e.g., via the control system 100) from the image 16A and contains generally the same information of the image 16A that can, therefore, be utilized (e.g., via the control system 100) to recreate the image 16A. Under the principles of integral imaging, the image 16A can be broken down into individual segments S. The segments S can be any shape (e.g., pixel by pixel dimensions or x, y or dimensions in the integral image x', y') and some varieties of which will be explained in further detail below. The input pixel values, input pixel intensities, input pixel luminance, and / or pixel values of each segment are populated in a summed area table (e.g., on a segment-by-segment basis), where each segment S holds the sum of all pixel values to the left and / or top of the segment S and the segment S itself. Further, utilizing the summed area table, any subset of segments S can be calculated and monitored. In this manner, object detection on any portion of the integral image 16B can be accomplished by utilizing, for example, four or less (e.g., three or less) sum of pixel values of any given segment S, rather than computing each individual segment. Each sum of all pixel values associated with a segment S can be represented as motion vectors that can be quickly summed (e.g., with four or less sum pixel values) without having to quantify each segment S individually. It is important to note that the computational steps (e.g., steps required by the control system 100) to detect the object 26 are largely independent of the size of the integral image 16B, but may be affected, although significantly less than relying simply on the image 16A, by the number of segments S and the number of pixels (i.e., image resolution).

[0028] With reference now particularly to FIG. 3C, the integral image 16B is depicted in accordance with a first implementation that is utilized under the principles as those described in reference to FIGS. 3A and 3B, where the segments S of the integral image 16B are patterned into the grid defined by both columns and rows.

[0029] With reference now to FIG. 3D, the integral image 16B is depicted in accordance with a first implementation, where the segments S of the integral image 16B are patterned into the columns for cumulating a sum of the corresponding input pixel for each column in a horizontal (e.g., cross-car) direction. For example, each segment S may include 50 pixels in the vertical direction and 100 pixels in the horizontal direction. Using for example, left-to-right segments S, each segment S holds the sum of all pixel values (e.g., of segments) of the segments S to the left. Utilizing the integral image 16B the control system 100 may only need to calculate (e.g., two additions and one subtraction) of adjacent segments S. Because the segments S depicted in FIG. 3A may be columns (i.e., greater pixel count from an up-down direction rather than a left-right direction), the segments S may be at least partially overlapped. Utilizing the sum of all pixel values in each segment S, motion vectors (e.g., changes to or differences in the sum of all pixel values in select segments S) can be quickly ascertained to detect the object 26 and movement of the object 26 based on correlative calculations. The integral image 16B can be generated for incoming frames (i.e., images 16A) of data, utilizing the linearized, white-balanced, Bayer pixel values as they exit color-balance operation. The control system 100 utilizes the integral image 16B with luminance values as input, so a simple bi-linear demosaic operation on the linearized pixels can be measured and obtained prior to the computation of the integral image 16B. Once the bi-linear interpolated pixels are obtained, the integral image 16B will line buffer the bi-linear interpolated pixels as the difference equation s(x,y-l) - s(x-l,y-l) (e.g., for input to the line buffer). The control system 100 utilizes this information on the next line (before overwriting with new data) by adding in i(x,y) and s(x-l,y), to create a final s(x,y) integral image pixel or segment S.

[0030] With reference now to FIG. 4, the control system 100 may utilize the images 16A and / or integral images 16B to determine an approximate distance of the object 26. For example, if the vehicle 12 is in motion and the object 26 is stationary, the vehicle 12 is stationary and the object 26 is in motion, or both the vehicle 12 and the object 26 are in motion relative changes in size detected by the control system 100 in the images 16A or inverse images 16B can be utilized to extrapolate the approximate distance of the object 26 and if the object 26 is becoming relatively further or closer to the vehicle 12. More particularly, the control system 100 may be configured to generate integral images 16B from the captured images 16A, the integral images 16B including segmentations S each with an input pixel data (e.g., pixel value sums). The integral images 16B or images 16A may then be reviewed for the presence of the object 26 within the field of view 19. If the object 26 is within the field of view 19, the control system 100 may determine an initial size of the object 26 on at least one of the images 16A, 16B or the graphic 22. After a predetermined period, the control system 100 (e.g., the processor 104) may further determine a subsequent size of the object 26 on at least one of a subsequent one of the images 16A, 16B or a subsequentgraphic 22, and extrapolate an approximate distance of the object 26 based on differences between the initial size and the subsequent size. More particularly, in scenarios the object 26 is static (i.e. not moving), it will appear in approximately the same location from one image 16A, 16B frame to the next. In order for the control system 100 to obtain a relative sense of how close the object 26 is to the rear of the vehicle 12, the rate of change of the size of the object 26 can be used to provide approximate distances to the vehicle 12. In this manner, the control system 100 may be configured to determine if the object 26 is static and not moving. For example, using a speed of the vehicle 12 (e.g., via the vehicular control system 150), the approximate distance based on the size of the object 26 can be determined based on frame- to-frame measurements. Of note, the closer the object 26 is, the measurement will generally change (e.g., expand) at a more rapid or greater rate while, when the object 26 is farther away, the measurements will change (e.g., expand) at a slower rate. Further, in some implementations, the control system 100 may be configured to detect the rate of change of the measurement to extrapolate a relative speed of movement between the vehicle 12 and the object 26. In some implementations, the control system 100 may be configured to generate the alert 28 when the approximate distance is within a predetermined threshold distance. In some implementations, the control system 100 may be further configured to generate a subsequent alert 28 when the approximate distance is within a second predetermined threshold distance that is closer than the first predetermined threshold distance.

[0031] With reference to FIG. 5, the control system 100 is associated, for example, with the one or more components of the environmental monitoring system 10, such as the full display mirror assembly 38, such as an electro-optic assembly (not shown), the imager module 14, and the display module 20. The control system 100 may include an electronic control unit (ECU) 102. The ECU 102 may include the processor 104 and the memory 106. The processor 104 may include any suitable processor 104. Additionally, or alternatively, the ECU 102 may include any suitable number of processors (e.g., located in one or more components of the full display mirror assembly 38 and / or the vehicle 12), in addition to or other than the processor 104. The memory 106 may comprise a single disk or a plurality of disks (e.g., hard drives), and includes a storage management module that manages one or more partitions within the memory 106. In some embodiments, the memory 106 may include flash memory, semiconductor (solid state) memory or the like. The memory 106 may include Random AccessMemory (RAM), a Read-Only Memory (ROM), or a combination thereof. The memory 106 may include instructions that, when executed by the processor 104, cause the processor 104 to, at least, perform the functions and method steps as described herein. The imager module 14 and the display module 20 may, therefore, be controlled, receive inputs, and / or transmit inputs in relation to the ECU 102. The ECU 102 may receive and / or the memory 106 may save software 108, an integral image module 110, image data 112 (images 16A) captured by the imager module 14, and an alert module 114. The software 108 may include application data upgradable via user inputs or a remote device, the integral image module 110 may include instructions for performing the integral image calculations described in reference to FIGS. 3A- 3D, the image data 112 may include an image 16A or a series of images 16A (e.g., a video) captured by the imager module 14, and the alert module 114 may include instructions for conditions, such as the presence of the object 26 and / or movement of the object 26 towards the path of travel of the vehicle 12 or the path of the vehicle 12 is moving towards the object 26 associated, for generating the alert 28 as described in the proceeding paragraphs and depicted in FIG. 2A. The control system 100 may be in operable communication with the vehicular control system 150, for example, the audio system for generating the alert 28, the steering wheel for generating the alert 28, and a speed detection system for detecting relative positioning as described in reference to FIG. 4. The control system 100 may further be configured to receive information from the vehicle control system 150 such as a turn signal activation, GPS directions to turn into the path of the object 26, steering wheel rotation, orientation of the wheels or steering gear, and / or the like as described above.

[0032] In further implementations, the control system 100 may include other configurations, in addition to or alternatively from the above, for performing the methods and functionalities described herein. For example, the control system 100 may include and operate under the principles of microprocessors, microcontrollers, application-specific integrated circuits (ASIC), or other circuitry configured to perform instructions, computations, and control various input / output signals. For example, in some implementations, the control system 100 may include one or more Field Programmable Gate Arrays ("FPGAs"), which are a type of programmable integrated circuits capable of performing the methods and functionalities described herein. In some implementations, the modules (e.g., the integral imager module 110 and / or the alert module 114) may be configured as integrated circuits, for example, in communication with other components of the control system 100. More particularly, in someembodiments, the integral image 16B is generated (e.g., calculated) by an integrated circuit, such as one or more FPGAs.

[0033] The disclosure herein is further summarized in the following paragraphs and is further characterized by combinations of any and all of the various aspects described therein.

[0034] According to one aspect of the present disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary that has a width smaller than the field of view. A control system is configured to generate integral images from the captured images, the integral images including segmentations, each with an input pixel data. The integral images are then reviewed for the presence of an object within the field of view but outside the outer graphic boundary. If the object is within the field of view but outside the outer graphic boundary, generate an alert to a user.

[0035] According to another aspect, a control system is configured to determine if an object is moving towards an outer graphic boundary before generating the alert.

[0036] According to yet another aspect, a control system is configured to segment the integral images into the columns for cumulating a sum of the corresponding input pixel for each column in a horizontal direction.

[0037] According to still another aspect, a display module is located in a full display mirror assembly.

[0038] According to another aspect, an alert is generated graphically on a display module.

[0039] According to yet another aspect, an alert generated includes a graphic on a display module on a side of the display module associated in a direction of relative positioning of an object.

[0040] According to another aspect, an alert is at least one of audible or haptic.

[0041] According to still another aspect, a control system is configured to, if the object is within the field of view but outside the outer graphic boundary, generate the alert to the user. If the object is moving towards the outer graphic boundary, generate a subsequent alert that is different than the alert.

[0042] According to yet another aspect, a control system is configured to, if the object is within the field of view but outside the outer graphic boundary, generate the alert to the user.If the vehicle is moving towards the object based on the object's position relative to the outer graphic boundary, generate a subsequent alert that is different than the alert

[0043] According to still yet another aspect, a control system is configured to, if the object is within the field of view but outside the outer graphic boundary, generate the alert to the user. The control system is further configured to generate a subsequent alert that is different than the alert upon receipt, from a vehicular control system, an indication that the vehicle will move towards the object.

[0044] According to another aspect, the indication includes at least one of a turn signal activation, a GPS direction, or a steering wheel rotation.

[0045] According to still another aspect, the alert is visual and the subsequent alert is one of audible or haptic.

[0046] According to yet another aspect, the control system includes one or more Field Programmable Gate Arrays for generating the integral images.

[0047] According to still yet another aspect, the display is located in a full display rearview mirror assembly.

[0048] According to another aspect of the present disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary. A control system includes is configured to generate integral images from the captured images, the integral images including segmentations each with an input pixel data. The integral images are then reviewed for the presence of an object within the field of view. If the object is within the field of view, determine an initial size of the object on at least one of the images or the graphic. After a predetermined period, the control system is further caused to determine a subsequent size of the object on at least one of a subsequent one of the integral images or a subsequent graphic, determine if the object is moving, and, if the object is not moving, extrapolate an approximate distance of the object based on differences between the initial size and the subsequent size.

[0049] According to another aspect, a control system is configured to determine a relative speed between the object and the vehicle based on a rate of change from the differences between the initial size and the subsequent size based on the review the integral images.

[0050] According to still another aspect, a control system is configured to receive a speed of the vehicle from a vehicular control system.

[0051] According to yet another aspect, a control system is configured to generate an alert when the approximate distance is within a predetermined threshold distance.

[0052] According to yet another aspect of the present disclosure, an environmental monitoring system for a vehicle includes at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle. A display module is configured to generate a graphic of the images and defines an outer graphic boundary that has a width smaller than the field of view. A control system is configured to generate integral images from the captured images, the integral images including segmentations each with an input pixel data. The integral images are then reviewed for the presence of an object and relative location of the object. If the object is detected proximate a path of travel, moving in a direction of the path of travel of the vehicle, or the vehicle is moving in a direction of the object generate an alert to a user.

[0053] According to another aspect, a control system is configured to generate the alert to the user if the object is detected proximate a path of travel, and generate a subsequent alert that is different than the alert if the object is moving in a direction of the path of travel of the vehicle or the vehicle is moving in the direction of the object.

[0054] It will be understood by one having ordinary skill in the art that construction of the described disclosure and other components is not limited to any specific material. Other exemplary embodiments of the disclosure disclosed herein may be formed from a wide variety of materials, unless described otherwise herein.

[0055] For purposes of this disclosure, the term "coupled" (in all of its forms, couple, coupling, coupled, etc.) generally means the joining of two components (electrical or mechanical) directly or indirectly to one another. Such joining may be stationary in nature or movable in nature. Such joining may be achieved with the two components (electrical or mechanical) and any additional intermediate members being integrally formed as a single unitary body with one another or with the two components. Such joining may be permanent in nature or may be removable or releasable in nature unless otherwise stated.

[0056] As used herein, the term "about" means that amounts, sizes, formulations, parameters, and other quantities and characteristics are not and need not be exact, but may be approximate and / or larger or smaller, as desired, reflecting tolerances, conversion factors,rounding off, measurement error and the like, and other factors known to those of skill in the art. When the term "about" is used in describing a value or an end-point of a range, the disclosure should be understood to include the specific value or end-point referred to. Whether or not a numerical value or end-point of a range in the specification recites "about," the numerical value or end-point of a range is intended to include two embodiments: one modified by "about," and one not modified by "about." It will be further understood that the end-points of each of the ranges are significant both in relation to the other end-point, and independently of the other end-point.

[0057] The terms "substantial," "substantially," and variations thereof as used herein are intended to note that a described feature is equal or approximately equal to a value or description. For example, a "substantially planar" surface is intended to denote a surface that is planar or approximately planar. Moreover, "substantially" is intended to denote that two values are equal or approximately equal. In some embodiments, "substantially" may denote values within about 10% of each other, such as within about 5% of each other, or within about 2% of each other.

[0058] It is also important to note that the construction and arrangement of the elements of the disclosure, as shown in the exemplary embodiments, is illustrative only. Although only a few embodiments of the present innovations have been described in detail in this disclosure, those skilled in the art who review this disclosure will readily appreciate that many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.) without materially departing from the novel teachings and advantages of the subject matter recited. For example, elements shown as integrally formed may be constructed of multiple parts, or elements shown as multiple parts may be integrally formed, the operation of the interfaces may be reversed or otherwise varied, the length or width of the structures and / or members or connectors or other elements of the system may be varied, and the nature or number of adjustment positions provided between the elements may be varied. It should be noted that the elements and / or assemblies of the system may be constructed from any of a wide variety of materials that provide sufficient strength or durability, in any of a wide variety of colors, textures, and combinations. Accordingly, all such modifications are intended to be included within the scope of the present innovations. Other substitutions, modifications, changes, and omissions may bemade in the design, operating conditions, and arrangement of the desired and other exemplary embodiments without departing from the spirit of the present innovations.

[0059] It will be understood that any described processes or steps within described processes may be combined with other disclosed processes or steps to form structures within the scope of the present disclosure. The exemplary structures and processes disclosed herein are for illustrative purposes and are not to be construed as limiting.

[0060] It is also to be understood that variations and modifications can be made on the aforementioned structures and methods without departing from the concepts of the present disclosure, and further it is to be understood that such concepts are intended to be covered by the following claims unless these claims by their language expressly state otherwise.

Claims

What is claimed is:

1. An environmental monitoring system for a vehicle comprising: at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle; a display module configured to generate a graphic of the images defining an outer graphic boundary that has a width smaller than the field of view; a control system configured to: generate integral images from the captured images, the integral images including segmentations each with an input pixel data; review the integral images for the presence of an object within the field of view but outside the outer graphic boundary; and if the object is within the field of view but outside the outer graphic boundary, generate an alert to a user.

2. The environmental monitoring system according to claim 1, wherein the control system is configured to determine if the object is moving towards the outer graphic boundary before generating the alert.

3. The environmental monitoring system according to claim 2, wherein the control system is configured to segment the integral images into the columns for cumulating a sum of the corresponding input pixel for each column in a horizontal direction.

4. The environmental monitoring system according to any one of claims 1-3, wherein the display module is located in a full display mirror assembly.

5. The environmental monitoring system according to any one of claims 1-3, wherein the alert is generated graphically on the display module.

6. The environmental monitoring system according to claim 5, wherein the alert generated includes a graphic on the display module on a side of the display module associated in a direction of relative positioning of the object.

7. The environmental monitoring system according to any one of claims 1-3, wherein the alert is at least one of audible or haptic.

8. The environmental monitoring system according to claim 1, wherein the control system is further configured to: if the object is within the field of view but outside the outer graphic boundary, generate the alert to the user; and if the object is moving towards the outer graphic boundary, generate a subsequent alert that is different than the alert.

9. The environmental monitoring system according to claim 1, wherein the control system is further configured to: if the object is within the field of view but outside the outer graphic boundary, generate the alert to the user; and if the vehicle is moving towards the object based on the objects position relative to the outer graphic boundary, generate a subsequent alert that is different than the alert.

10. The environmental monitoring system according to claim 1, wherein the control system is further configured to: if the object is within the field of view but outside the outer graphic boundary, generate the alert to the user; and generate a subsequent alert that is different than the alert upon receipt, from a vehicular control system, an indication that the vehicle will move towards the object.

11. The environmental monitoring system according to claim 10, wherein the indication includes at least one of a turn signal activation, a GPS direction, or a steering wheel rotation.

12. The environmental monitoring system according to any one of claims 8-11, wherein the alert is visual and the subsequent alert is one of audible or haptic.

13. The environmental monitoring system according to any one of claims 1-3, wherein the control system includes one or more Field Programmable Gate Arrays for generating the integral images.

14. The environmental monitoring system according to any one of claims 1-3, wherein the display is located in a full display rearview mirror assembly.

15. An environmental monitoring system for a vehicle comprising: at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle; a display module configured to generate a graphic of the images defining an outer graphic boundary; a control system is configured to: generate integral images from the captured images, the integral images including segmentations each with an input pixel data; review the integral images for the presence of an object within the field of view; if the object is within the field of view, determine an initial size of the object on at least one of the integral images or the graphic; after a predetermined period, determine a subsequent size of the object on at least one of a subsequent one of the integral images or a subsequent graphic; determine if the object is moving; and if the object is not moving, extrapolate an approximate distance of the object based on differences between the initial size and the subsequent size.

16. The environmental monitoring system according to claim 15, wherein the control system is further configured to determine a relative speed between the object and the vehicle based on a rate of change from the differences between the initial size and the subsequent size based on the review the integral images.

17. The environmental monitoring system according to claim 16, wherein the control system is further configured to receive a speed of the vehicle from a vehicular control system.

18. The environmental monitoring system according to any one of claims 15-17, wherein the control system is further configured to generate an alert when the approximate distance is within a predetermined threshold distance.

19. An environmental monitoring system for a vehicle comprising: at least one imager module having a field of view and configured to capture a series of images of an environment defined by the field of view relative to an associated vehicle; a display module configured to generate a graphic of the images within the field of view; a control system configured to: generate integral images from the captured images, the integral images including segmentations each with an input pixel data; review the integral images for the presence of an object and relative location of the object; and if the object is detected proximate a path of travel, moving in a direction of the path of travel of the vehicle, or the vehicle is moving in a direction of the object generate an alert to a user.

20. The environmental monitoring system according to claim 19, wherein the control system is configured to: generate the alert to the user if the object is detected proximate a path of travel; and generate a subsequent alert that is different than the alert if the object is moving in a direction of the path of travel of the vehicle or the vehicle is moving in the direction of the object.

Images