Dynamic region of interest identification for vehicles

Abstract

A method for identifying a region of interest in an environment surrounding a vehicle may include receiving data of interest using a first vehicle sensor. The data of interest includes a first measurement. The method may also include determining an expected path of the vehicle. The method may also include identifying a region of interest in the surroundings of the vehicle based at least in part on at least one of the data of interest and an expected path of the vehicle. The method may also include making a second measurement of the region of interest using a second vehicle sensor.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for identifying regions of interest in a vehicle's surroundings. Background Art

[0002] To enhance occupant awareness and convenience, vehicles may be equipped with advanced driver assistance systems (ADAS) and / or automated driving systems (ADS). ADAS and ADS systems can use a variety of sensors (such as cameras, radar, and light detection and ranging sensors) to detect and identify objects around the vehicle, including other vehicles, pedestrians, road configurations, traffic signs, and road markings. ADAS and ADS systems can take actions based on the environmental conditions surrounding the vehicle, such as applying the brakes or issuing alerts to vehicle occupants. However, the various sensors used by ADAS and ADS systems may generate a large amount of data about the entire environment surrounding the vehicle without considering the importance of certain parts of the environment relative to other parts of the environment.

[0003] Therefore, while ADAS and ADS systems and methods achieve their intended purposes, a need remains for new and improved systems and methods for identifying areas of interest in a vehicle's surroundings. Summary of the Invention

[0004] According to several aspects, a method for identifying an area of interest in a vehicle's surroundings is provided. The method may include receiving data of interest using a first vehicle sensor. The data of interest may include a first measurement. The method may also include determining an expected path of the vehicle. The method may also include identifying the area of interest in the vehicle's surroundings based, at least in part, on at least one of the data of interest and the expected path of the vehicle. The method may also include taking a second measurement of the area of interest using a second vehicle sensor.

[0005] In another aspect of the present disclosure, receiving the data of interest may further include taking a first measurement of the vehicle surroundings using a first vehicle sensor, wherein the first vehicle sensor includes at least one of a radar sensor and a light detection and ranging (LIDAR) sensor.

[0006] In another aspect of the present disclosure, identifying an area of interest in the vehicle surroundings may further include identifying an object of interest in the vehicle surroundings based at least in part on the first measurement. Identifying the area of interest in the vehicle surroundings may further include determining a predicted path of the object of interest based at least in part on the first measurement. Identifying the area of interest in the vehicle surroundings may further include calculating a time of collision between the object of interest and the vehicle based at least in part on the predicted path of the object of interest. Identifying the area of interest in the vehicle surroundings may further include determining a probability of collision based at least in part on the predicted path of the object of interest, an uncertainty in the predicted path of the object of interest, and a time of collision between the object of interest and the vehicle. Identifying the area of interest in the vehicle surroundings may further include identifying that the area of interest includes the object of interest in response to determining that the probability of collision is greater than or equal to a predetermined probability of collision threshold.

[0007] In another aspect of the present disclosure, determining the intended path of the vehicle may further include receiving one or more occupant inputs from an occupant of the vehicle using one or more vehicle input devices. Determining the intended path of the vehicle may further include performing one or more vehicle dynamics measurements using one or more vehicle dynamics sensors. Determining the intended path of the vehicle may further include determining the location of the vehicle using a global navigation satellite system (GNSS). Determining the intended path of the vehicle may further include determining the intended path of the vehicle based, at least in part, on at least one of the one or more occupant inputs, the one or more vehicle dynamics measurements, and the location of the vehicle.

[0008] In another aspect of the present disclosure, identifying a region of interest in the vehicle's surroundings may further include identifying a region of interest in the vehicle's surroundings, wherein the region of interest includes at least a portion of an expected path of the vehicle.

[0009] In another aspect of the present disclosure, receiving the data of interest may further include receiving one or more region cues from a remote server system.The first measurement is one or more region cues.

[0010] In another aspect of the present disclosure, identifying the region of interest in the vehicle surroundings may further include identifying the region of interest in the vehicle surroundings based at least in part on one or more region clues, wherein the one or more region clues include locations of missing map elements from a remote server system.

[0011] In another aspect of the present disclosure, identifying an area of interest in the vehicle's surroundings may further include identifying the area of interest in the vehicle's surroundings based at least in part on one or more area cues. The one or more area cues may include at least one crowdsourced area of interest parameter. The at least one crowdsourced area of interest parameter may be determined by a remote server system using crowdsourcing.

[0012] In another aspect of the present disclosure, performing a second measurement of the region of interest using a second vehicle sensor may further include capturing a first image of the vehicle's surroundings using the second vehicle sensor. The first image has a first image resolution. The first image includes at least the region of interest. The second vehicle sensor is a camera. Performing a second measurement of the region of interest using the second vehicle sensor may further include determining a priority for the region of interest. The priority includes at least one of a high priority and a low priority. Performing a second measurement of the region of interest using the second vehicle sensor may further include caching the first image in a non-transitory memory in response to determining that the priority is a low priority. Performing a second measurement of the region of interest using the second vehicle sensor may further include generating a second image of the vehicle's surroundings in response to determining that the priority is a high priority.

[0013] In another aspect of the present disclosure, the second image of the environment includes the region of interest. The second image of the environment is upscaled using a machine learning super-resolution algorithm.

[0014] According to several aspects, a system for identifying an area of interest in a vehicle's surroundings is provided. The system may include a first vehicle sensor. The system may also include a second vehicle sensor. The system may also include a vehicle controller in electrical communication with the first vehicle sensor and the second vehicle sensor. The vehicle controller is programmed to receive data of interest using the first vehicle sensor. The vehicle controller is further programmed to identify an area of interest in the vehicle's surroundings based at least in part on the data of interest. The vehicle controller is further programmed to take a second measurement of the area of interest using the second vehicle sensor.

[0015] In another aspect of the present disclosure, the first vehicle sensor includes a perception sensor. To receive data of interest using the first vehicle sensor, the vehicle controller is further programmed to take a first measurement using the perception sensor. The data of interest is the first measurement.

[0016] In another aspect of the present disclosure, to identify an area of interest in the vehicle's surroundings, the vehicle controller is further programmed to identify an object of interest in the vehicle's surroundings based at least in part on the first measurement. To identify the area of interest in the vehicle's surroundings, the vehicle controller is further programmed to determine a predicted path of the object of interest based at least in part on the first measurement. To identify the area of interest in the vehicle's surroundings, the vehicle controller is further programmed to calculate a time of collision between the object of interest and the vehicle based at least in part on the predicted path of the object of interest. To identify the area of interest in the vehicle's surroundings, the vehicle controller is further programmed to determine a probability of collision based at least in part on an uncertainty in the predicted path of the object of interest and a time of collision between the object of interest and the vehicle. To identify the area of interest in the vehicle's surroundings, the vehicle controller is further programmed to identify that the area of interest includes the object of interest in response to determining that the probability of collision is greater than or equal to a predetermined probability of collision threshold.

[0017] In another aspect of the present disclosure, the first vehicle sensor is a vehicle communication system. To receive data of interest using the first vehicle sensor, the vehicle controller is further programmed to receive one or more region clues from a remote server system using the vehicle communication system. The data of interest is one or more region clues. To receive the data of interest using the first vehicle sensor, the vehicle controller is further programmed to transmit at least one vehicle region of interest parameter to the remote server system using the vehicle communication system for crowdsourcing by the remote server system.

[0018] In another aspect of the present disclosure, to identify an area of interest, the vehicle controller is further programmed to identify an area of interest in the vehicle's surroundings based, at least in part, on one or more area cues. The one or more area cues include locations of missing map elements from a remote server system. To identify an area of interest, the vehicle controller is further programmed to identify an area of interest in the vehicle's surroundings based, at least in part, on the one or more area cues. The one or more area cues include at least one crowdsourced area of interest parameter. The at least one crowdsourced area of interest parameter is determined by the remote server system using crowdsourcing.

[0019] In another aspect of the present disclosure, the second vehicle sensor includes a camera. In order to perform a second measurement of the area of interest using the second vehicle sensor, the vehicle controller is further programmed to capture a first image of the vehicle's surroundings using the camera. The first image has a first image resolution. The first image includes at least the area of interest. In order to perform a second measurement of the area of interest using the second vehicle sensor, the vehicle controller is further programmed to determine a priority of the area of interest. The priority includes at least one of a high priority and a low priority. In order to perform a second measurement of the area of interest using the second vehicle sensor, the vehicle controller is further programmed to cache the first image in a non-transitory memory of the vehicle controller in response to determining that the priority is a low priority. In order to perform a second measurement of the area of interest using the second vehicle sensor, the vehicle controller is further programmed to generate a second image of the vehicle's surroundings in response to determining that the priority is a high priority.

[0020] In another aspect of the present disclosure, the system further includes a vehicle graphics processing unit (GPU) in electrical communication with the vehicle controller. To generate a second image of the vehicle's surroundings, the vehicle controller is further programmed to compare the first image resolution with the maximum image resolution of the camera. To generate the second image of the vehicle's surroundings, the vehicle controller is further programmed to generate a magnified image of the vehicle's surroundings in response to determining that the first image resolution is equal to the maximum image resolution of the camera. The magnified image includes a region of interest. The magnified image of the environment is magnified using a machine learning super-resolution algorithm executed by the vehicle's GPU. To generate the second image of the vehicle's surroundings, the vehicle controller is further programmed to capture a high-resolution image of the vehicle's surroundings in response to determining that the first image resolution is less than the maximum image resolution of the camera. The high-resolution image includes the region of interest. The high-resolution image has a second image resolution. The second image resolution is greater than the first image resolution.

[0021] According to several aspects, a method for identifying an area of interest in a vehicle's surroundings is provided. The method may include taking a first measurement of the vehicle's surroundings using a first vehicle sensor. The first vehicle sensor is a perception sensor, including at least one of a radar sensor and a light detection and ranging (LIDAR) sensor. The method may also include identifying the area of interest in the vehicle's surroundings based at least in part on the first measurement. The method may also include taking a second measurement of the area of interest using a second vehicle sensor. The second vehicle sensor is a camera. The method may also include performing a data processing task based at least in part on the second measurement.

[0022] In another aspect of the present disclosure, identifying an area of interest in the vehicle surroundings may further include identifying an object of interest in the vehicle surroundings based at least in part on the first measurement. Identifying the area of interest in the vehicle surroundings may further include determining a predicted path of the object of interest based at least in part on the first measurement. Identifying the area of interest in the vehicle surroundings may further include calculating a time of collision between the object of interest and the vehicle based at least in part on the predicted path of the object of interest. Identifying the area of interest in the vehicle surroundings may further include determining a probability of collision based at least in part on an uncertainty of the predicted path of the object of interest and a time of collision between the object of interest and the vehicle. Identifying the area of interest in the vehicle surroundings may further include identifying that the area of interest includes the object of interest in response to determining that the probability of collision is greater than or equal to a predetermined probability of collision threshold.

[0023] In another aspect of the present disclosure, performing a second measurement of the region of interest using a second vehicle sensor may further include capturing a first image of the vehicle's surroundings using a camera. The first image has a first image resolution. The first image includes at least the region of interest. Performing the second measurement of the region of interest using the second vehicle sensor may further include comparing the first image resolution to a maximum image resolution of the camera. Performing the second measurement of the region of interest using the second vehicle sensor may further include generating a magnified image of the vehicle's surroundings in response to determining that the first image resolution is equal to the camera's maximum image resolution. The magnified image includes the region of interest. The magnified image of the environment is magnified using a machine learning super-resolution algorithm. Performing the second measurement of the region of interest using the second vehicle sensor may further include capturing a high-resolution image of the vehicle's surroundings in response to determining that the first image resolution is less than the camera's maximum image resolution. The high-resolution image includes the region of interest. The high-resolution image of the environment has a second image resolution. The second image resolution is greater than the first image resolution.

[0024] Further areas of applicability will become apparent from the description provided herein.It should be understood that the description and specific examples are intended for purposes of illustration only and are not intended to limit the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are for illustration purposes only and are not intended to limit the scope of the present disclosure in any way.

[0026] Figure 1 is a schematic diagram of a system for identifying an area of interest in a vehicle's surroundings according to an exemplary embodiment;

[0027] Figure 2 is a flow chart of a first exemplary method for identifying a region of interest in a vehicle's surroundings according to an exemplary embodiment;

[0028] Figure 3 is a flow chart of a second exemplary method for identifying an area of interest in a vehicle's surroundings according to an exemplary embodiment; and

[0029] Figure 4 is a flow chart of a method for performing a second measurement of a region of interest according to an exemplary embodiment. DETAILED DESCRIPTION

[0030] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, or uses.

[0031] In various aspects of the present disclosure, certain areas of a vehicle's surroundings may be more important than others for object detection, collision avoidance, driver assistance features, and the like. For example, areas including lane markings, road signs, and the like are highly relevant for object detection and driver assistance features. Areas including threats such as objects or vehicles in a collision path are highly relevant for collision avoidance. Therefore, the present disclosure provides a new and improved system and method for identifying areas of interest in a vehicle's surroundings and performing additional or enhanced measurements on the areas of interest.

[0032] refer to Figure 1 , a system for identifying areas of interest in a vehicle's surroundings is shown, generally designated by the reference numeral 10. The system 10 is shown with an exemplary vehicle 12. Although a passenger vehicle is shown, it should be understood that the vehicle 12 may be any type of vehicle without departing from the scope of the present disclosure. The system 10 generally includes a vehicle controller 14, a vehicle graphics processing unit (GPU) 16, and a plurality of vehicle sensors 18.

[0033] The vehicle controller 14 is used to implement methods 200 and 300 for identifying areas of interest in the vehicle's surroundings, as will be described below. The vehicle controller 14 includes at least one processor 20 and a non-transitory computer-readable storage device or medium 22. The processor 20 can be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the vehicle controller 14, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a device for executing instructions. The computer-readable storage device or medium 22 can include volatile and non-volatile storage devices such as read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is persistent or non-volatile memory that can be used to store various operating variables when the processor 20 is powered off. The computer-readable storage device or medium 22 can be implemented using a variety of memory devices, such as programmable read-only memory (PROM), electrical PROM (ePROM), electrically erasable PROM (EEPROM), flash memory, or other electrical, magnetic, optical, or combination memory devices capable of storing data, some of which represents executable instructions used by the vehicle controller 14 to control various systems of the vehicle 12. The vehicle controller 14 can also be composed of multiple controllers that are in electrical communication with each other. The vehicle controller 14 can be interconnected with additional systems and / or controllers of the vehicle 12, thereby allowing the vehicle controller 14 to access data such as the speed, acceleration, braking, and steering angle of the vehicle 12.

[0034] The vehicle controller 14 is in electrical communication with a vehicle graphics processing unit (GPU) 16 and a plurality of vehicle sensors 18. In an exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (e.g., WiFi, Ethernet, etc.), a serial peripheral interface (SPI) network, etc. It should be understood that various additional wired and wireless technologies and communication protocols for communicating with the vehicle controller 14 are within the scope of the present disclosure.

[0035] The vehicle graphics processing unit (GPU) 16 is used to facilitate and / or accelerate the processing of graphics-intensive computing tasks. In an exemplary embodiment, the GPU includes one or more GPU processors and video memory. In a non-limiting example, the one or more GPU processors are configured to process multiple parallel tasks simultaneously. In a non-limiting example, the one or more processors may include multiple processing cores. In another non-limiting example, the one or more GPU processors include one or more field programmable gate arrays (FPGAs) configured to perform graphics processing tasks.

[0036] The video memory is used to store, buffer, and / or cache video data before, during, and / or after processing by one or more GPU processors. In a non-limiting example, the video memory is a volatile high-speed memory. In some embodiments, the vehicle GPU 16 may also include additional memory devices for storing software and / or firmware, such as programmable read-only memory (PROM), electrical programmable ...

[0037] It should be understood that the vehicle GPU 16 may include multiple graphics processing units in electrical communication with each other. It should be understood that the vehicle GPU 16 may be integrated with the vehicle controller 14 (e.g., on the same circuit board as the vehicle controller 14 or otherwise integrated as part of the vehicle controller 14) without departing from the scope of the present disclosure. The vehicle GPU 16 is in electrical communication with the vehicle controller 14 as described above.

[0038] Vehicle sensors 18 are used to obtain information related to vehicle 12. In the exemplary embodiment, vehicle sensors 18 include at least a camera 24, a vehicle communication system 26, a perception sensor 28, a global navigation satellite system (GNSS) 30, a vehicle input device 32, and a vehicle dynamics sensor 34.

[0039] The camera 24 is a perception sensor for capturing images and / or video of the environment surrounding the vehicle 12. In an exemplary embodiment, the camera 24 comprises a still camera and / or video camera positioned to view the environment surrounding the vehicle 12. In a non-limiting example, the camera 24 comprises a camera mounted inside the vehicle 12 (e.g., in the headliner of the vehicle 12) with a view through the windshield. In another non-limiting example, the camera 24 comprises a camera mounted outside the vehicle 12 (e.g., on the roof of the vehicle 12) with a view of the environment in front of the vehicle 12.

[0040] In another exemplary embodiment, the cameras 24 are a surround-view camera system that includes multiple cameras (also referred to as satellite cameras) arranged to provide a view of the environment adjacent to all sides of the vehicle 12. In a non-limiting example, the cameras 24 include a front-facing camera (e.g., mounted in the front grille of the vehicle 12), a rear-facing camera (e.g., mounted on the rear tailgate of the vehicle 12), and two side-facing cameras (e.g., mounted below each of the two side-view mirrors of the vehicle 12). In another non-limiting example, the cameras 24 also include an additional rear-view camera mounted near the center high-mounted brake light of the vehicle 12.

[0041] In an exemplary embodiment, the vehicle controller 14 is capable of capturing images at multiple resolutions using the camera 24. In a first embodiment, the vehicle controller 14 uses the camera 24 to capture an image at a second image resolution (e.g., 1920×1080). The vehicle controller 14 then downsamples the image to a first image resolution (e.g., 960×540) to conserve computing, network, and / or storage resources. Thus, in the first embodiment, a higher resolution image can be obtained by retaining the image originally captured at the second image resolution without downsampling. In the first embodiment, the first image resolution is less than the second image resolution, and the second image resolution is less than or equal to the maximum image resolution of the camera 24 (i.e., the maximum image resolution that the camera 24 is capable of capturing). In a second embodiment, the vehicle controller 14 uses the camera 24 to directly capture an image at the first image resolution (e.g., if the camera 24 is not capable of capturing images at the second image resolution). Therefore, in the second embodiment, it is not possible to obtain a higher resolution image without upscaling, as will be discussed in more detail below. In the second embodiment, the first image resolution is equal to the maximum image resolution of the camera 24.

[0042] It should be understood that camera systems with additional cameras and / or additional mounting locations are within the scope of the present disclosure. It should also be understood that cameras with various sensor types, including, for example, charge coupled device (CCD) sensors, complementary metal oxide semiconductor (CMOS) sensors, and / or high dynamic range (HDR) sensors, are within the scope of the present disclosure. Furthermore, cameras with various lens types, including, for example, wide angle lenses and / or narrow angle lenses, are also within the scope of the present disclosure.

[0043] The vehicle controller 14 uses the vehicle communication system 26 to communicate with other systems external to the vehicle 12. For example, the vehicle communication system 26 includes the capability to communicate with vehicles ("V2V" communication), infrastructure ("V2I" communication), remote systems at a remote call center (e.g., General Motors' ON-STAR), and / or personal devices. Generally speaking, the term "vehicle-to-everything communication" ("V2X" communication) refers to communication between the vehicle 12 and any remote system (e.g., vehicle, infrastructure, and / or remote system). In certain embodiments, the vehicle communication system 26 is a wireless communication system that is configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communication (e.g., using GSMA standards such as SGP.02, SGP.22, SGP.32, etc.).

[0044] Thus, the vehicle communication system 26 may also include an embedded universal integrated circuit card (eUICC) configured to store at least one cellular connection configuration profile, such as an embedded user identity module (eSIM) profile. The vehicle communication system 26 is also configured to communicate via a personal area network (e.g., Bluetooth), near field communication (NFC), and / or any additional type of radio frequency communication. However, additional or alternative communication methods such as dedicated short-range communication (DSRC) channels and / or mobile telecommunication protocols based on the Third Generation Partnership Project (3GPP) standards are also considered to be within the scope of the present disclosure. A DSRC channel refers to a one-way or two-way short-range to medium-range wireless communication channel designed specifically for automotive use and a set of corresponding protocols and standards. 3GPP refers to a partnership between multiple standards organizations that develop mobile telecommunication protocols and standards. The 3GPP standards are structured as "releases." Therefore, communication methods based on 3GPP Releases 14, 15, 16, and / or future 3GPP releases are considered to be within the scope of the present disclosure.

[0045] Thus, the vehicle communication system 26 may include one or more antennas and / or communication transceivers for receiving and / or transmitting signals, such as collaborative sensing messages (CSMs). The vehicle communication system 26 is configured to wirelessly transmit information between the vehicle 12 and another vehicle. Additionally, the vehicle communication system 26 is configured to wirelessly transmit information between the vehicle 12 and infrastructure or other vehicles. It should be understood that the vehicle communication system 26 may be integrated with the vehicle controller 14 (e.g., integrated on the same circuit board as the vehicle controller 14 or otherwise integrated as part of the vehicle controller 14) without departing from the scope of the present disclosure.

[0046] Perception sensor 28 is used to perceive objects and / or measure distances in the environment surrounding vehicle 12. In an exemplary embodiment, perception sensor 28 includes at least one of a radar sensor 36 and a light detection and ranging (LiDAR) sensor 38.

[0047] The radar sensor 36 is used to detect and measure the distance, speed, and direction of an object by transmitting radio waves and analyzing the reflections of the radio waves. In an exemplary embodiment, the radar sensor 36 includes a radar transmitter, a radar antenna, a radar receiver, and a radar signal processing unit. In a non-limiting example, the radar transmitter transmits a radio frequency (RF) signal that travels through space until it encounters an object. The RF signal bounces off the surface of the object and returns to the radar sensor 36. The radar receiver captures the reflected signal, and the radar signal processing unit analyzes the time delay, frequency shift, and amplitude of the returned RF signal to determine the distance, speed, and direction of the detected object. As described above, the radar sensor 36 is in electrical communication with the vehicle controller 14.

[0048] The LiDAR sensor 38 is used for remote sensing and environmental mapping by emitting laser pulses and measuring the time it takes for the laser pulses to return to the LiDAR sensor 38 after striking an object. In an exemplary embodiment, the LiDAR sensor 38 includes a LiDAR laser source, a LiDAR scanner or mirror, a LiDAR photodetector, and a LiDAR time-of-flight measurement system. In a non-limiting example, the LiDAR laser source emits laser pulses that travel to a target area, and the LiDAR scanner directs these pulses in different directions. The emitted laser pulses interact with objects in the environment, and their reflections are captured by the LiDAR photodetector. The LiDAR time-of-flight measurement system calculates the distance to the object based on the time between the LiDAR laser source emitting the laser pulse and the LiDAR photodetector receiving the reflected laser pulse. As described above, the LiDAR sensor 38 is in electrical communication with the vehicle controller 14.

[0049] In another exemplary embodiment, perception sensor 28 further comprises a stereo camera with distance measurement capabilities. In one example, perception sensor 28 is mounted inside vehicle 12, such as in the roof lining of vehicle 12, with a view through the windshield of vehicle 12. In another example, perception sensor 28 is mounted outside vehicle 12, such as on the roof of vehicle 12, with a view of the environment surrounding vehicle 12. It should be understood that various additional types of perception sensors, such as ultrasonic ranging sensors and / or time-of-flight sensors, are within the scope of the present disclosure. As described above, perception sensor 28 is in electrical communication with vehicle controller 14.

[0050] GNSS 30 is used to determine the geographic location of vehicle 12. In an exemplary embodiment, GNSS 30 is a global positioning system (GPS). In a non-limiting example, the GPS includes a GPS receiver antenna (not shown) and a GPS controller (not shown) in electrical communication with the GPS receiver antenna. The GPS receiver antenna receives signals from multiple satellites, and the GPS controller calculates the geographic location of vehicle 12 based on the signals received by the GPS receiver antenna.

[0051] In an exemplary embodiment, GNSS 30 also includes a map. This map includes information about infrastructure, such as municipal boundaries, roads, railways, sidewalks, buildings, and the like. Thus, the map information is used to contextualize the geographic location of vehicle 12. In one non-limiting example, the map is retrieved from a remote source using a wireless connection. In another non-limiting example, the map is stored in a database of GNSS 30.

[0052] It should be understood that various additional types of satellite-based radio navigation systems, such as the Global Positioning System (GPS), Galileo, GLONASS, and the BeiDou Navigation Satellite System (BDS), are within the scope of the present disclosure. It should be understood that the GNSS 30 can be integrated with the vehicle controller 14 (e.g., on the same circuit board as the vehicle controller 14 or otherwise integrated as part of the vehicle controller 14) without departing from the scope of the present disclosure.

[0053] Vehicle input devices 32 are used to receive input from occupants of vehicle 12 . Within the scope of this disclosure, occupants include, in non-limiting examples, the driver, passengers, and / or any other person in vehicle 12 . In exemplary embodiments, vehicle input devices 32 include mechanical and / or electromechanical buttons, switches, levers, knobs, or other control devices. In non-limiting examples, vehicle input devices 32 include brake pedals, accelerator pedals, steering wheels, turn signal switches, and the like. In exemplary embodiments, vehicle input devices 32 include electrically powered and / or software-controlled buttons or switches that use capacitive touch or other touchscreen technology to detect occupant interaction. In non-limiting examples, vehicle input devices 32 include infotainment touchscreens. In exemplary embodiments, vehicle input devices 32 include hands-free input devices, including, for example, a microphone for receiving voice commands, a camera for receiving gesture commands, or other sensors. It should be understood that vehicle input devices 32 include any device with which an occupant can interact within vehicle 12 to control the operation of any aspect or system of vehicle 12 . Vehicle input devices 32 are in electrical communication with vehicle controller 14 .

[0054] Vehicle dynamics sensors 34 are used to measure and monitor various parameters related to the motion and stability of vehicle 12. In an exemplary embodiment, vehicle dynamics sensors 34 include one or more of the following: an accelerometer, a gyroscope, and / or a magnetometer. In non-limiting examples, an accelerometer measures linear acceleration, a gyroscope measures direction and angular velocity, and a magnetometer detects the orientation of vehicle 12 relative to the Earth's magnetic field.

[0055] An accelerometer is used to measure linear acceleration. In a non-limiting example, the accelerometer is a microelectromechanical system (MEMS) that includes a mass suspended on one or more springs. When the vehicle 12 accelerates, the mass is displaced, and the displacement of the mass is converted into an electrical signal. A gyroscope is used to measure the direction and angular velocity of the vehicle 12. In a non-limiting example, the gyroscope includes a rotating disk or a vibrating crystal. When the vehicle 12 rotates, the gyroscope detects the Coriolis effect and generates an electrical signal proportional to the direction and / or angular velocity. A magnetometer is used to detect the direction of the vehicle 12 relative to the Earth's magnetic field. In a non-limiting example, the magnetometer uses a Hall effect sensor to measure changes in the magnetic field. It should be understood that the vehicle dynamics sensor 34 may include additional sensors, such as an inertial measurement unit (IMU), without departing from the scope of this disclosure. The vehicle dynamics sensor 34 is in electrical communication with the vehicle controller 14.

[0056] In another exemplary embodiment, the plurality of vehicle sensors 18 further includes sensors for determining performance data regarding the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 18 further includes at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor.

[0057] In another exemplary embodiment, the plurality of vehicle sensors 18 further includes additional sensors for determining information about the environment within the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 18 further includes at least one of a seat occupancy sensor, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, a cabin microphone, an occupant eye tracker, and the like.

[0058] In another exemplary embodiment, the plurality of vehicle sensors 18 further include additional sensors for determining information about the environment surrounding the vehicle 12. In a non-limiting example, the plurality of vehicle sensors 18 further include at least one of an ambient air temperature sensor, a barometric pressure sensor, etc. As described above, the plurality of vehicle sensors 18 are in electrical communication with the vehicle controller 14.

[0059] Continue to refer Figure 1 , a remote server system is shown and generally designated by the reference numeral 40. The remote server system 40 includes a server controller 42 in electronic communication with a server database 44 and a server communication system 46. In a non-limiting example, the remote server system 40 is located in a server farm, data center, or the like and is connected to the Internet.

[0060] The server controller 42 includes at least one server processor 48 and a server non-transitory computer-readable storage device or server media 50. The descriptions of types and configurations given above for the vehicle controller 14 also apply to the server controller 42. In some examples, the server controller 42 differs from the vehicle controller 14 in that the server controller 42 can have a higher processing speed, include more memory, include more inputs / outputs, etc. In a non-limiting example, the server processor 48 and server media 50 of the server controller 42 are similar in structure and / or function to the processor 20 and media 22 of the vehicle controller 14, as described above.

[0061] The server database 44 is used to store detailed maps of roads, including, for example, information about lane boundaries, road geometry, speed limits, traffic signs, and / or other relevant features. In an exemplary embodiment, the server database 44 includes one or more mass storage devices, such as a hard drive, a tape drive, a magneto-optical drive, an optical disk, a solid-state drive, and / or an attached device operable to store data in a persistent and machine-readable manner. In some examples, one or more mass storage devices may be configured to provide redundancy in the event of hardware failure and / or data corruption using, for example, a redundant array of independent disks (RAID). In a non-limiting example, the server controller 42 may execute software such as a database management system (DBMS) so that data stored on the one or more mass storage devices is organized and accessed.

[0062] The server communication system 46 is used to communicate with external systems (e.g., the vehicle controller 14) via the vehicle communication system 26. In a non-limiting example, as described above, the server communication system 46 is similar in structure and / or function to the vehicle communication system 26. In some examples, the server communication system 46 differs from the vehicle communication system 26 in that the server communication system 46 is capable of higher power signal transmission, more sensitive signal reception, higher bandwidth transmission, additional transmission / reception protocols, etc.

[0063] refer to Figure 2, a flow chart of a first exemplary method 200 for identifying an area of interest in a vehicle's surroundings is provided. The first exemplary method 200 begins at block 202 and proceeds to block 204. At block 204, the vehicle controller 14 receives data of interest. Within the scope of the present disclosure, the data of interest includes data related to an area of interest in the vehicle's surroundings. In an exemplary embodiment, the data of interest includes a first measurement of the vehicle's surroundings performed using the perception sensor 28. In a non-limiting example, the first measurement is performed using the radar sensor 36. In another non-limiting example, the first measurement is performed using the LiDAR sensor 38. In an exemplary embodiment, the measurement includes, for example, the position of an object of interest relative to the vehicle 12 and / or the velocity of the object of interest relative to the vehicle 12. Within the scope of the present disclosure, an object of interest is an object in the vehicle's surroundings that may be related to the vehicle 12, for example, that may cause a collision with the vehicle 12. After block 204, the first exemplary method 200 proceeds to block 206.

[0064] At block 206, the vehicle controller 14 determines a probability of collision with the object of interest based, at least in part, on the first measurement performed at block 204. In an exemplary embodiment, to determine the probability of collision, the vehicle controller 14 first identifies the object of interest based on the first measurement. In a non-limiting example, the object of interest is a remote vehicle traveling on a road near the vehicle 12. In an exemplary embodiment, the vehicle controller 14 identifies the object of interest based on one or more of the following: size, position, heading, and velocity of a plurality of objects detected in the environment surrounding the vehicle 12.

[0065] The vehicle controller 14 then determines a predicted path for the object of interest based at least in part on the first measurement. In a non-limiting example, the predicted path for the object of interest is determined based on the position, velocity, and heading of the object of interest. In an exemplary embodiment, the predicted path for the object of interest includes a corresponding uncertainty value. The uncertainty of the predicted path for the object of interest quantifies the likelihood that the predicted path for the object of interest is correct. In some embodiments, the uncertainty of the predicted path for the object of interest provides additional information, such as a mathematical function describing the statistical probability of various deviations of the object of interest from the predicted path.

[0066] The vehicle controller 14 then determines a time of collision between the object of interest and the vehicle 12 based at least in part on the speed of the vehicle 12, the speed of the object of interest, and the predicted path of the object of interest. In an exemplary embodiment, the time of collision is determined based on the longitudinal distance between the vehicle 12 and the object of interest. A probability of collision is determined based at least in part on the uncertainty of the predicted path of the object of interest and the time of collision between the object of interest and the vehicle 12. In an exemplary embodiment, the probability of collision is determined based on a predicted lateral position of the object of interest relative to the predicted path of the vehicle 12. In a non-limiting example, a deterministic algorithm (e.g., a weighted average or other statistical measure) is used to calculate the probability of collision. In another non-limiting example, a machine learning model is used to determine the probability of collision, which machine learning model is trained to determine the probability of collision based at least in part on the uncertainty of the predicted path of the object of interest and the time of collision between the object and the vehicle 12. After box 206, the first exemplary method 200 proceeds to box 208.

[0067] At block 208, the vehicle controller 14 compares the collision probability determined at block 206 to a predetermined collision probability threshold (e.g., ten percent). If the collision probability is greater than or equal to the predetermined collision probability threshold, the first exemplary method 200 proceeds to block 210. If the collision probability is less than the predetermined collision probability threshold, the first exemplary method 200 proceeds to block 212, as will be discussed in greater detail below.

[0068] At block 210, in response to determining at block 208 that the collision probability is greater than or equal to the predetermined collision probability threshold, the vehicle controller 14 identifies that the region of interest includes the object of interest. Within the scope of the present disclosure, a region of interest is an area within the environment surrounding the vehicle 12 (i.e., a portion of the environment surrounding the vehicle 12) that is of interest for further sensor measurements, as will be discussed in greater detail below.

[0069] In an exemplary embodiment, the region of interest determined by the vehicle 12 is defined by vehicle region of interest parameters. In a non-limiting example, the region of interest parameters include, for example, two or more coordinates defining a bounding box (i.e., location and size) of the region of interest and one or more GNSS coordinates to which the region of interest applies. In an exemplary embodiment, to identify that the region of interest includes an object of interest, the vehicle region of interest parameters are selected such that the location of the object of interest is within the region of interest.

[0070] In addition to adjusting the region of interest parameters to include the object of interest, the region of interest parameters may also be additionally adjusted or controlled based on additional coefficients and parameters. In an exemplary embodiment, the region of interest parameters may be adjusted based on, for example, perception measurements determined by the vehicle sensors 18, such as a number of static objects (i.e., non-moving objects) in the environment surrounding the vehicle 12, a number of dynamic objects (i.e., moving objects) in the environment surrounding the vehicle 12, scene complexity in the environment surrounding the vehicle 12, and / or analysis of pixel variations (e.g., to identify static and dynamic regions of the environment surrounding the vehicle 12).

[0071] In another exemplary embodiment, the region of interest parameters may be adjusted based on environmental measurements determined using the vehicle sensors 18, such as lighting of the environment surrounding the vehicle 12 (e.g., brightness measurements based on the CIELAB color space method), weather of the environment surrounding the vehicle 12 (e.g., windshield wiper status, fog / mist detection based on fog light activation, etc.), the number and / or type of map features in the environment surrounding the vehicle 12 (e.g., number of lanes, merge locations, intersection locations, etc.), GNSS quality at the location of the vehicle 12 (e.g., 2D absolute position error, absolute velocity error, precision degradation), and wireless connection quality at the location of the vehicle 12 (e.g., signal strength, network type, service level, etc.).

[0072] In another exemplary embodiment, the region of interest parameters can be adjusted based on additional coefficients, for example, the region of interest can have a defined maximum and minimum size. In addition, the size change of the region of interest (i.e., expansion or contraction) can be controlled by a control parameter that limits the rate of size change. For example, the control parameter can be configured to cause the region of interest to expand quickly but to contract slowly (i.e., the maximum allowable expansion rate is greater than the maximum allowable contraction rate). After block 210, the first exemplary method 200 proceeds to block 214, as will be discussed in more detail below.

[0073] At box 212, in response to determining at box 208 that the collision probability is less than a predetermined collision probability threshold, the vehicle controller 14 determines an intended path for the vehicle 12. In an exemplary embodiment, to determine the intended path for the vehicle 12, the vehicle controller 14 receives one or more occupant inputs from a vehicle occupant using the vehicle input device 32. In a non-limiting example, the one or more occupant inputs include an occupant-activated turn signal. In an exemplary embodiment, the vehicle controller 14 may determine that the intended path for the vehicle 12 includes a turn or a lane change based on the turn signal activation. In another non-limiting example, the one or more occupant inputs include a steering angle change (i.e., the occupant turns the steering wheel). In an exemplary embodiment, the vehicle controller 14 may determine that the intended path for the vehicle 12 includes a turn or a lane change based on the steering angle change. In another non-limiting example, the one or more occupant inputs include actuation of a brake pedal. In an exemplary embodiment, the vehicle controller 14 may determine that the intended path for the vehicle 12 includes a deceleration or a stop based on the brake pedal actuation.

[0074] In another exemplary embodiment, to determine the intended path of vehicle 12, vehicle controller 14 utilizes vehicle dynamics sensors 34 to perform one or more vehicle dynamics measurements. In an exemplary embodiment, the one or more measurements include acceleration measurements, angular velocity measurements, and / or heading measurements. In an exemplary embodiment, vehicle controller 14 determines the intended path of vehicle 12 based on the one or more vehicle dynamics measurements. In an exemplary embodiment, to determine the intended path of vehicle 12, vehicle controller 14 utilizes GNSS 30 to determine the location of vehicle 12. In an exemplary embodiment, vehicle controller 14 determines the intended path of vehicle 12 based on the location of vehicle 12 and / or map information stored in GNSS 30.

[0075] In another exemplary embodiment, to determine the intended path of the vehicle 12, the vehicle controller 14 uses one of the plurality of vehicle sensors 18. In an exemplary embodiment, the vehicle controller 14 uses an occupant eye tracker to determine the occupant's gaze direction. In a non-limiting example, the occupant's gaze direction is correlated with the vehicle's intended path.

[0076] It will be appreciated that the intended path of the vehicle 12 may be determined at least in part based on one or more of the following: one or more occupant inputs, one or more vehicle dynamics measurements, inputs from the vehicle sensors 18, and / or the location of the vehicle 12. It will also be appreciated that the vehicle controller 14 may utilize deterministic algorithms and / or machine learning-based algorithms (e.g., sensor fusion algorithms, etc.) to process and interpret the one or more occupant inputs, one or more vehicle dynamics measurements, and / or the location of the vehicle 12 to determine the intended path of the vehicle. Following block 212, the first exemplary method 200 proceeds to block 216.

[0077] At block 216, the vehicle controller 14 identifies that the region of interest includes the expected path of the vehicle 12 determined at block 212. In an exemplary embodiment, to identify that the region of interest includes the expected path of the vehicle 12, vehicle region of interest parameters are selected such that the location of the expected path of the vehicle 12 (or, in some examples, the portion of the field of view of the vehicle sensors 18 that is closest to the intended path of the vehicle 12) is within the region of interest. For example, if the expected path of the vehicle 12 includes a right turn, the region of interest may be configured to include a portion of the environment in front of the vehicle 12 and around the right side of the vehicle 12. In addition to adjusting the region of interest parameters to include the expected path of the vehicle 12, the region of interest parameters may also be additionally adjusted or controlled based on additional coefficients and parameters, as discussed above with reference to block 210. Following block 216, the first exemplary method 200 proceeds to block 214.

[0078] At block 214, the vehicle controller 14 uses the vehicle communication system 26 to establish a connection with the server communication system 46 of the remote server system 40 and transmits the vehicle area of interest parameters determined at blocks 210 and / or 216 and the location of the vehicle 12 (as determined using the GNSS 30) to the remote server system 40. In an exemplary embodiment, the server controller 42 uses the server communication system 46 to receive the vehicle area of interest parameters and the location of the vehicle 12 and aggregate the data in the server database 44. In a non-limiting example, the server controller 42 creates an aggregated crowdsourced database of vehicle area of interest parameters corresponding to the GNSS location, as will be discussed in more detail below. After block 214, the first exemplary method 200 proceeds to block 218.

[0079] At block 218 , the vehicle controller 14 takes a second measurement of the area of interest using the camera 24 , as will be discussed in greater detail below. Following block 218 , the first exemplary method 200 proceeds to block 220 .

[0080] At box 220, the vehicle controller 14 takes an action based at least in part on the second measurement taken at box 218. In an exemplary embodiment, the action includes performing a data processing task based at least in part on the second measurement. Within the scope of the present disclosure, the data processing task includes, for example, an object detection and / or classification algorithm. For example, the data processing task may include detecting road geometry based on the second measurement. The action may also include transmitting information about the detected features to the remote server system 40. For example, the object detection algorithm may detect the location of lane markings on the road, and the vehicle communication system 26 may be used to transmit the location of the lane markings on the road to the remote server system 40 for storage, aggregation, and transmission to other vehicles. After box 220, the first exemplary method 200 proceeds to enter a standby state at box 222.

[0081] In an exemplary embodiment, the vehicle controller 14 repeatedly exits the standby state at block 222 and restarts the method 200 at block 202. In a non-limiting example, the vehicle controller 14 exits the standby state at block 222 and restarts the method 200 on a timer, such as every three hundred milliseconds.

[0082] refer to Figure 3 , a flow chart of a second exemplary method 300 for identifying an area of interest in an environment surrounding a vehicle is provided. The second exemplary method 300 begins at box 302 and proceeds to box 304. At box 304, the vehicle controller 14 receives data of interest. In an exemplary embodiment, the data of interest includes one or more area clues transmitted by the remote server system 40 and received by the vehicle communication system 26. Within the scope of the present disclosure, the one or more area clues include information about the location and / or size of the requested area of interest. In a non-limiting example, the one or more area clues include the location of a missing map element from the remote server system 40. For example, the one or more area clues may include the approximate GNSS coordinates of a lane marking at the right edge of the road, which lane marking is known to exist, but whose detailed data (e.g., exact location, size, color, type, etc.) is missing from the map stored in the server database 44.

[0083] In another non-limiting example, the vehicle controller 14 downloads a portion of a map (also referred to as a map tile) from the remote server system 40 using the vehicle communication system 26. The vehicle controller 14 then searches the map tile for missing map elements (e.g., missing lane marking information, missing traffic sign information, missing lane configuration information, missing speed limit information, etc.) to identify the location of the missing map element.

[0084] In another non-limiting example, one or more area clues may include crowdsourced area of interest parameters. Within the scope of the present disclosure, crowdsourced area of interest parameters are area of interest parameters determined by aggregating (e.g., averaging) vehicle area of interest parameters of GNSS coordinates received from multiple vehicles. For example, due to unclear road markings on the left edge of the intersection, multiple vehicles may determine an area of interest near the left edge of the road at the intersection. The multiple vehicles then transmit the vehicle area of interest parameters along with the GNSS positions to the remote server system 40 (see the discussion of block 214 above). Thus, the remote server system 40 determines the crowdsourced area of interest parameters based on the aggregation (e.g., averaging or other statistical methods) of the vehicle area of interest parameters received from the multiple vehicles. After receiving one or more area clues at block 304, the second exemplary method 300 proceeds to block 306.

[0085] At block 306, if the one or more area clues received at block 304 do not include the location of the missing map element, the second exemplary method 300 proceeds to block 308, as discussed in greater detail below. If the one or more area clues received at block 304 include the location of the missing map element, the second exemplary method 300 proceeds to block 310.

[0086] At block 310, the vehicle controller 14 identifies that the region of interest includes the missing map element. In an exemplary embodiment, to identify that the region of interest includes the missing map element, vehicle region of interest parameters are selected such that the location of the missing map element is within the region of interest. In a non-limiting example, the vehicle controller 14 uses a computer vision algorithm (e.g., an object detection algorithm) to identify the missing map element using the vehicle sensors 18 and determine the location of the missing map element in the environment surrounding the vehicle 12. The vehicle controller 14 then determines the vehicle region of interest parameters such that the location of the missing map element is within the region of interest. In addition to adjusting the region of interest parameters to include the missing map element, the region of interest parameters may also be additionally adjusted or controlled based on additional coefficients and parameters, as discussed above with reference to block 210. Following block 310, the second exemplary method 300 proceeds to block 218, as will be discussed in more detail below.

[0087] At block 308, if the one or more region cues received at block 304 do not include crowdsourced region of interest parameters, the second exemplary method 300 proceeds to enter a standby state at block 312. If the one or more region cues received at block 304 include crowdsourced region of interest parameters, the second exemplary method 300 proceeds to block 314.

[0088] At block 314, the vehicle controller 14 identifies a region of interest based on the crowdsourced region of interest parameters. In an exemplary embodiment, two or more coordinates defining a bounding box (i.e., location and size) of the region of interest are set based on the crowdsourced region of interest parameters. In a non-limiting example, the crowdsourced region of interest parameters also include one or more GNSS coordinates to which the region of interest applies. Thus, the region of interest described by the crowdsourced region of interest parameters is only applied when the vehicle 12 is within a predetermined distance threshold from the one or more GNSS coordinates. In addition to adjusting the region of interest parameters to include the crowdsourced region of interest parameters, the region of interest parameters may also be additionally adjusted or controlled based on additional coefficients and parameters, as discussed above with reference to block 210. Following block 314, the second exemplary method 300 proceeds to block 218, as discussed in greater detail below.

[0089] At block 218 , the vehicle controller 14 uses the camera 24 to perform a second measurement of the region of interest determined at blocks 310 or 314 , as discussed in greater detail below. Following block 218 , the second exemplary method 300 proceeds to block 316 .

[0090] At box 316, the vehicle controller 14 takes an action based at least in part on the second measurement taken at box 218. In an exemplary embodiment, the action includes performing a data processing task based at least in part on the second measurement. Within the scope of the present disclosure, the data processing task includes, for example, an object detection and / or classification algorithm. For example, the data processing task may include detecting road geometry based on the second measurement. The action may also include transmitting information about the detected features to the remote server system 40. For example, the object detection algorithm may detect the location of lane markings on the road, and the vehicle communication system 26 may be used to transmit the location of the lane markings on the road to the remote server system 40 for storage, aggregation, and transmission to other vehicles. After box 316, the second exemplary method 300 proceeds to enter a standby state at box 312.

[0091] In an exemplary embodiment, the vehicle controller 14 repeatedly exits the standby state at block 312 and restarts the method 300 at block 302. In a non-limiting example, the vehicle controller 14 exits the standby state at block 312 and restarts the method 300 on a timer, such as every three hundred milliseconds.

[0092] refer to Figure 4 , a flow chart of an exemplary embodiment 218a of block 218 (i.e., a method for performing a second measurement of a region of interest) is shown. Exemplary embodiment 218a begins at block 402. At block 402, vehicle controller 14 uses camera 24 to capture a first image of the environment surrounding vehicle 12. In the exemplary embodiment, the first image is captured using a first image resolution (e.g., 960×540). The first image includes at least the region of interest. After block 402, exemplary embodiment 218a proceeds to block 404.

[0093] At box 404, the vehicle controller 14 determines the priority of the area of interest. Within the scope of the present disclosure, the priority describes the time sensitivity of further processing of the area of interest. In a non-limiting example, the priority includes, for example, a high priority or a low priority. In an exemplary embodiment, the priority is determined based on a predetermined set of rules (e.g., a lookup table). In a non-limiting example, if the area of interest is determined based on the probability of collision with the object of interest (e.g., as discussed with reference to the first exemplary method 200), the priority is determined to be a high priority. If the area of interest is determined based on missing map elements (e.g., as discussed with reference to the second exemplary method 300), the priority is determined to be a low priority. After box 404, the exemplary embodiment 218a proceeds to box 406.

[0094] At block 406 , if the priority determined at block 404 is high priority, the exemplary embodiment 218 a proceeds to block 408 , as discussed in greater detail below. If the priority determined at block 404 is low priority, the exemplary embodiment 218 a proceeds to block 410 .

[0095] At box 410, the vehicle controller 14 caches the first image captured at box 402 in the medium 22 of the vehicle controller 14. In an exemplary embodiment, the vehicle controller 14 also uses the camera 24 to capture a high-resolution image (i.e., a second image). The high-resolution image includes at least the region of interest. In an exemplary embodiment, the high-resolution image includes only the region of interest. In an exemplary embodiment, the high-resolution image is captured using a second image resolution (e.g., 1920×1080), wherein the second image resolution is higher than the first image resolution. The high-resolution image is also cached in the medium 22 of the vehicle controller 14. In another exemplary embodiment, one or more of the first image and the high-resolution image are transmitted to the remote server system 40 using the vehicle communication system 26. The remote server system 40 saves one or more of the first image and the high-resolution image in the server database 44 for subsequent processing. After box 410, the exemplary embodiment 218a ends, and the methods 200, 300 continue as described above.

[0096] At block 408, vehicle controller 14 compares the first image resolution of the first image to the maximum image resolution of camera 24. If the first image resolution is equal to the maximum image resolution of camera 24 (i.e., if vehicle controller 14 does not downsample images captured using camera 24, as described above), exemplary embodiment 218a proceeds to block 412, as discussed in greater detail below. In some embodiments, the comparison of block 408 is omitted, and exemplary embodiment 218a proceeds directly to block 412. If the first image resolution is less than the maximum image resolution of camera 24 (i.e., if vehicle controller 14 does downsample images captured using camera 24, as described above), exemplary embodiment 218a proceeds to block 414.

[0097] At block 414, vehicle controller 14 uses camera 24 to capture a high-resolution image (i.e., a second image) of the environment surrounding vehicle 12. The high-resolution image includes at least the region of interest. In an exemplary embodiment, the high-resolution image includes only the region of interest. In an exemplary embodiment, the high-resolution image is captured using a second image resolution (e.g., 1920×1080), which is higher than the first image resolution. After block 414, exemplary embodiment 218a ends, and methods 200, 300 continue as described above.

[0098] At block 412, the vehicle controller 14 uses the vehicle GPU 16 to generate a magnified image (i.e., a second image) of the environment surrounding the vehicle 12. Within the scope of the present disclosure, a magnified image is an image having a higher resolution than the original image. In an exemplary embodiment, the magnified image is generated based at least in part on the first image captured at block 402. In an exemplary embodiment, the magnified image includes only the region of interest. In a non-limiting example, the magnified image is generated using a machine learning super-resolution algorithm executed at least by the vehicle GPU 16.

[0099] In an exemplary embodiment, a machine learning super-resolution algorithm is used to enhance the resolution and quality of images using machine learning and / or artificial intelligence techniques. In a non-limiting example, the machine learning super-resolution algorithm employs a deep learning model, such as a convolutional neural network (CNN), to upscale low-resolution content while enhancing visual detail. In a non-limiting example, the machine learning super-resolution algorithm is trained using a dataset comprising image pairs of high-resolution images and corresponding low-resolution images. The machine learning super-resolution algorithm functions by taking a low-resolution image as input and generating a high-resolution output that predicts missing details in the image. After sufficient training, the machine learning super-resolution algorithm is capable of reconstructing fine textures and structures not present in the original low-resolution image. In an exemplary embodiment, the machine learning super-resolution algorithm enhances the graphics processing and / or parallelization capabilities of the vehicle's GPU 16 to improve the speed, accuracy, and / or efficiency of image processing. After block 412, the exemplary embodiment 218a ends, and methods 200, 300 continue as described above.

[0100] The disclosed system 10 and methods 200, 300 have several advantages. Using the system 10 and methods 200, 300, the region of interest is dynamically adjusted based on vehicle behavior, environmental conditions, and information received from the remote server system 40. By using the system 10 and methods 200, 300, the performance of additional vehicle systems and features, such as object detection algorithms, collision avoidance features, and the like, is improved due to the dynamic adjustment of the region of interest. The use of an adaptive region of interest allows computational resources required for image magnification or high-resolution image capture to be efficiently utilized to improve the functionality of vehicle systems and features.

[0101] The description of the present disclosure is merely exemplary in nature, and variations that do not depart from the gist of the present disclosure are intended to fall within the scope of the present disclosure. Such variations should not be regarded as a departure from the spirit and scope of the present disclosure.

Claims

1. A method for identifying a region of interest in an environment surrounding a vehicle, comprising: receiving data of interest using a first vehicle sensor, wherein the data of interest includes a first measurement; determining an intended path for the vehicle; identifying an area of interest in the vehicle's surroundings based at least in part on at least one of the data of interest and an expected path of the vehicle; and A second measurement of the region of interest is taken using a second vehicle sensor.

2. The method according to claim 1, wherein Receiving interesting data also includes: The first measurement of the vehicle surroundings is performed using the first vehicle sensor, wherein the first vehicle sensor includes at least one of a radar sensor and a light detection and ranging (LIDAR) sensor.

3. The method according to claim 2, wherein Identifying a region of interest in the vehicle's surroundings further includes: identifying an object of interest in the vehicle surroundings based at least in part on the first measurement; determining a predicted path of the object of interest based at least in part on the first measurement; calculating a time of collision between the object of interest and the vehicle based at least in part on the predicted path of the object of interest; determining a collision probability based at least in part on the predicted path of the object of interest, an uncertainty in the predicted path of the object of interest, and a time of collision between the object of interest and the vehicle; and The region of interest is identified as including the object of interest in response to determining that the collision probability is greater than or equal to a predetermined collision probability threshold.

4. The method according to claim 1, wherein Determining the expected path of the vehicle further includes: receiving one or more occupant inputs from an occupant of the vehicle using one or more vehicle input devices; performing one or more vehicle dynamics measurements using one or more vehicle dynamics sensors; determining the position of the vehicle using a Global Navigation Satellite System (GNSS); and An intended path of the vehicle is determined based at least in part on at least one of the one or more occupant inputs, the one or more vehicle dynamics measurements, and a position of the vehicle.

5. The method according to claim 4, wherein Identifying a region of interest in the vehicle's surroundings further includes: A region of interest in the vehicle's surroundings is identified, wherein the region of interest includes at least a portion of an expected path of the vehicle.

6. The method according to claim 1, wherein Receiving interesting data also includes: One or more region cues are received from a remote server system, wherein the first measurement is the one or more region cues.

7. The method according to claim 6, wherein: Identifying a region of interest in the vehicle's surroundings further includes: An area of interest in the vehicle surroundings is identified based at least in part on the one or more area cues, wherein the one or more area cues include locations of missing map elements from the remote server system.

8. The method according to claim 6, wherein: Identifying a region of interest in the vehicle's surroundings further includes: An area of interest in the vehicle surroundings is identified based at least in part on the one or more area cues, wherein the one or more area cues include at least one crowdsourced area of interest parameter, wherein the at least one crowdsourced area of interest parameter is determined by the remote server system using crowdsourcing.

9. The method according to claim 1, wherein Taking a second measurement of the region of interest using a second vehicle sensor further includes: capturing a first image of the vehicle surroundings using a second vehicle sensor, wherein the first image has a first image resolution, wherein the first image includes at least the region of interest, wherein the second vehicle sensor is a camera; determining a priority of the region of interest, wherein the priority includes at least one of a high priority and a low priority; In response to determining that the priority is a low priority, caching the first image in a non-transitory memory; and In response to determining that the priority is high, a second image of the vehicle surroundings is generated.

10. The method according to claim 9, wherein: Generating a second image of the vehicle surroundings further includes: A second image of the vehicle's surroundings is generated, wherein the second image includes the region of interest, and wherein the second image of the environment is magnified using a machine learning super-resolution algorithm.

Images