SYSTEM AND METHOD FOR ADJUSTING THE ALIGNMENT OF A CAMERA MOUNTED IN THE VEHICLE

Abstract

The present disclosure provides a system (100) and a method for adjusting the orientation of cameras (120) mounted in a vehicle (10). The method includes performing (202) a location-based scenario check based on high-resolution map data and applying (204) an orientation value to one or more cameras (120) to perform an online calibration of the cameras (120) based on the location-based scenario check (206). The method includes obtaining (208) a plurality of still images taken by the cameras (120) during the online calibration of the cameras (120).The procedure involves performing (212) a back projection onto one or more selected images from the plurality of images to determine a rotation angle for the cameras (120), and updating (214) an alignment angle of the cameras (120) based on the rotation angle of the cameras (120) to dynamically adjust the alignment of the cameras (120).

Description

The present disclosure relates to automotive technology. In particular, the present disclosure relates to a system and a method for dynamically adjusting the orientation of a camera mounted in a vehicle in real time based on localized high-resolution (HD) map data. Currently, a vehicle sensor array is inherently static and limits the cameras' field of view (FOV), especially when driving on roads with sharp curves. While 360-degree cameras provide a wide perspective, they struggle to capture detailed information about objects at greater distances, particularly when relying on side cameras. This results in a perception gap that affects both near and far vision. Complex road scenarios, such as exits, on-ramps, and dynamic environments like multi-lane highways with, for example, 10 lanes, are not fully visible to long-range cameras due to their limited FOV. While these cameras are capable of detecting objects far ahead, they may not cover the necessary width of a roadway.Conversely, cameras with a larger FOV can cover more lanes and nearby objects, but have a limited range, meaning they cannot detect distant road elements early enough for lane planning or trajectory adjustment, making them less effective. For example, lane 1 might be outside the detection range of the wide-area camera, and neither a lane split nor lane 5 might be detected by any of the cameras in the current vehicle sensor array. This limitation leads to significant problems with lane planning and decision-making, particularly in scenarios where a vehicle traveling in lane 1 needs to change lanes to lane 5. By the time the mid-range camera detects the split and lane 5, it would be too late to execute the necessary sequence of lane changes, leaving insufficient time for the lane planning or trajectory algorithms to react and safely navigate the vehicle across multiple lanes.This scenario highlights the need for a more dynamic camera system that can adjust its orientation and FOV based on the vehicle's real-time environment to ensure that critical road information is detected earlier, improve lane-changing decision-making, and increase overall driving safety. Many techniques have been developed to avoid the aforementioned problems. For example, patent US10997737B2 discloses a system and method for aligning image data from a vehicle camera. The method involves obtaining image data from the vehicle camera; obtaining sensor data from a vehicle sensor(s); and performing a yaw / pitch estimation operation to obtain information about the camera's yaw / pitch alignment, including a yaw misalignment estimation and a pitch misalignment estimation. The yaw / pitch estimation operation uses the image data and the sensor data to determine three-dimensional points of identified features within the individual frames of the image data.Following the yaw / pitch estimation process, the procedure involves performing a roll estimation process to obtain roll alignment information, which includes a roll misalignment estimation. The roll estimation process involves using information about the yaw / pitch camera alignment. The procedure involves applying camera alignment information to the image data to obtain aligned image data. The camera alignment information includes information based on the yaw / pitch camera alignment information, the roll alignment information, or both. Another patent document, US20240100947A1, discloses systems and methods for detecting that the vehicle is in reverse mode, receiving road surface topology, determining based on the road surface topology that a feature of interest is within a collected image and outside a displayed field of view (FOV) for a rear-facing camera, and adjusting the displayed FOV for the rear-facing camera to place the feature of interest within the displayed FOV. Receiving a reverse driving indication can be done via a Controller Area Network (CAN) bus in the vehicle.Receiving a road surface topology involves receiving an estimate of the road surface topology via at least one of monocular depth estimation, photogrammetric area mapping using Structure from Motion (SFM), multiview stereo, imaging radar, lidar and a sensor system on the vehicle. Although the cited document reveals various techniques for automatically adjusting the camera FOV, they do not focus on updating a camera orientation angle to dynamically adjust the camera's orientation, and there is still room for a solution for dynamically adjusting the camera's orientation. A general objective of the present disclosure is to provide a system and a method for dynamically adjusting the orientation of a vehicle-mounted camera in real time based on localized high-resolution (HD) map data. Another objective of the present disclosure is to provide a system and a method to adjust the orientation and field of view (FOV) of the camera based on the real-time environment of a vehicle, thereby ensuring that critical road information is detected earlier, improving decision-making during lane changes and increasing overall driving safety. Another objective of the present disclosure is to provide a system and a method to facilitate lane changes in advance and to enable better trajectory planning by adjusting the camera's orientation and FOV. The present disclosure relates to automotive technology. In particular, the present disclosure relates to a system and a method for dynamically adjusting the orientation of a camera mounted in a vehicle in real time based on localized high-resolution (HD) map data. One aspect of the present disclosure relates to a system for adjusting the orientation of one or more cameras mounted in a vehicle. The system may include a processor and memory coupled to the processor. The memory contains one or more processor-executable instructions which, when executed, cause the processor to perform a location-based scenario check based on high-resolution map data and apply an orientation value to the one or more cameras to perform an online calibration of the one or more cameras based on the location-based scenario check. The processor receives a plurality of still images taken by the one or more cameras during the online calibration of the one or more cameras.The processor performs a back-projection onto one or more selected frames from the majority of frames to determine a rotation angle for the one or more cameras. Based on this rotation angle, the processor updates the orientation angle of the one or more cameras to dynamically adjust their orientation. In some embodiments, the high-resolution map data can include information on road features, lane geometries, lane markings, and global positioning system (GPS) coordinates of a route oriented toward a destination. In some embodiments, the processor can perform location-based scenario checking by being configured to create a region of interest (ROI) around the vehicle's current position based on high-resolution map data. The processor can extract lane geometry and a centerline of an ego lane ahead of the vehicle based on the ROI and the path oriented toward the destination. The processor can determine changes in curvature and slope of the lane geometry within a predefined field of view (FOV) of one or more cameras by filtering out the centerline of the ego lane, and define an initial FOV for the one or more cameras based on these changes in curvature and slope. In some implementations, the processor can create the ROI based on the predefined FOV of one or more cameras. In some embodiments, after receiving one or more individual images captured by one or more cameras, the processor can send the one or more individual images, along with one or more near-range camera images and one or more mid-range camera images, to a perception system to perform camera-based perception. In some embodiments, the processor can determine the rotation angle for one or more cameras by being configured, once the initial FOV is defined, to select one or more map points within the ROI and determine an ROI-based rotation value for the one or more cameras using an extrinsic calibration matrix based on the one or more selected map points. The processor can then backproject the one or more selected map points onto the one or more selected still images using the extrinsic calibration matrix and calculate one or more image coordinates from the backprojected map points.The processor can determine that the one or more back-projected map points fit within the one or more image coordinates and that a percentage of the one or more selected map points is higher than a predetermined threshold. Based on this determination, the processor can adjust an extrinsic rotation value in the extrinsic calibration matrix to maximize the number of visible map points within the initial FOV and determine a rotation angle for the tilt and yaw of the one or more cameras based on the ROI-based rotation value and the extrinsic rotation value. In some embodiments, the processor can determine that the rotation angle matches a pre-calibration value at a current position of one or more cameras, after the rotation angle for the tilt and yaw of one or more cameras has been calculated. In some embodiments, the processor can update the orientation angle of one or more cameras by being configured to validate integrity between the ROI-based rotation value and the extrinsic rotation value, and, based on successful validation, to update the orientation angle of one or more cameras. In some embodiments, when updating the orientation angle of one or more cameras, the processor can be configured to independently determine one or more calibration parameters based on a new position of the one or more cameras and to determine whether the one or more calibration parameters match a pre-calibration value. If the one or more calibration parameters match the pre-calibration value, the processor can adjust the orientation of the one or more cameras. If the one or more calibration parameters do not match the pre-calibration value, the processor can reset the one or more cameras to an original position. One aspect of the present disclosure relates to a method for adjusting the orientation of one or more cameras mounted in a vehicle. The method may involve a processor associated with a system performing a location-based scenario check based on high-resolution map data and the processor applying an orientation value to the camera(s) to perform an online camera calibration based on the location-based scenario check. The method may also involve the processor obtaining a plurality of still images taken by the camera(s) during the online camera calibration.The procedure may involve the processor performing a back-projection onto one or more selected individual images from the majority of images to determine a rotation angle for the one or more cameras, and the processor updating a camera orientation angle based on the rotation angle of the one or more cameras to dynamically adjust the orientation of the one or more cameras. Various tasks, features, aspects and advantages of the invention will become clearer from the following detailed description of preferred embodiments together with the accompanying drawings, in which identical numbers represent identical components. The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and form part of this patent specification. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. Fig. 1 illustrates an exemplary block diagram of a vehicle that includes a system for adjusting the orientation of a camera mounted in a vehicle according to the embodiments of the present disclosure. Fig. 2A illustrates a flowchart of an exemplary method for adjusting the orientation of the camera mounted in the vehicle according to the embodiments of the present disclosure.Figure 2B illustrates an exemplary flowchart depicting a process for performing a location-based scenario check based on high-resolution map data according to embodiments of the present disclosure. Figure 2C illustrates an exemplary flowchart depicting a process for performing a backprojection of one or more selected still images according to embodiments of the present disclosure. Figure 2D illustrates an exemplary diagram depicting the pre-calibration of the camera according to embodiments of the present disclosure. A detailed description of the embodiments of the disclosure illustrated in the accompanying drawings follows. The embodiments are described in such detail to clearly convey the disclosure. However, the intention is not to limit the foreseeable variations of embodiments by providing so much detail; on the contrary, the intention is to cover all modifications, equivalents, and alternatives that fall within the scope of the present disclosures, as defined by the accompanying claims. The present disclosure relates to automotive technology. In particular, the present disclosure relates to a system and a method for dynamically adjusting the orientation of a camera mounted in a vehicle in real time based on localized high-resolution (HD) map data. One aspect of the present disclosure relates to a system for adjusting the orientation of a camera mounted in a vehicle. The system includes a processor and memory coupled to the processor. The memory contains one or more processor-executable instructions which, when executed, cause the processor to perform a location-based scenario check based on high-resolution map data and apply an orientation value to the camera to perform an online calibration of the camera based on the location-based scenario check.The processor receives a plurality of still images captured by the camera during the camera's online calibration, performs a back-projection onto one or more selected still images from the plurality of still images to determine a rotation angle for the camera, and updates a camera orientation angle based on the camera rotation angle to dynamically adjust the camera's orientation. One aspect of the present disclosure relates to a method for adjusting the orientation of a camera mounted in a vehicle. The method involves a processor associated with a system performing a location-based scenario check based on high-resolution map data and the processor applying an orientation value to the camera to perform an online calibration of the camera based on the location-based scenario check. The method also includes the processor obtaining a plurality of still images taken by the camera during the online camera calibration.The process involves the processor performing a back-projection onto one or more selected individual images from the majority of images to determine a rotation angle for the camera, and the processor updating an orientation angle of the camera based on the rotation angle of the camera to dynamically adjust the orientation of the camera. Various embodiments of the present disclosure are explained in detail with reference to Figs. 1-2D. Fig. 1 illustrates an exemplary block diagram of a vehicle 10 which includes a system 100 for adjusting the orientation of a camera 120 mounted in the vehicle 10 according to the embodiments of the present disclosure. Referring to Fig. 1, the vehicle 10 can include, but is not limited to, the system 100 and one or more cameras 120. In one embodiment, the system 100 can include one or more processors 102. The processor(s) 102 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuits, and / or any devices that process data based on operating instructions. Among other capabilities, the one or more processors 102 can be configured to retrieve and execute computer-readable instructions stored in a primary memory 104. The memory 104 can store the computer-readable instructions or routines that can be retrieved and executed to form the data units or to share them with other elements of the system 100.Memory 104 can include any non-volatile storage device, including, for example, volatile memory such as random access memory (RAM) or non-volatile memory such as erasable programmable read-only memory (EPROM), flash memory, and the like. In some embodiments, the system 100 may also include an interface 106. The interface 106 may include a plurality of interfaces, for example, interfaces for data input and output devices, referred to as I / O devices, secondary storage devices, and the like. The interface 106 may facilitate communication between the system 100 and other components of the vehicle 10 by using peripheral devices that enable wired and / or wireless communication. The interface 106 may also provide a communication path for one or more components within the system 100. Examples of such components include, but are not limited to, processing module 108 and a database 110.The database 110 can contain data that is either stored or generated as a result of functionalities implemented by one of the components of the processing engine(s) 108. In one embodiment, one or more cameras 120 can be operationally connected to the system 100. The cameras 120 can include a fixed short-range camera, a fixed mid-range camera, and a dynamically oriented long-range camera. The short-range and mid-range cameras can have a fixed field of view (FOV), and their orientation must not change. This serves to monitor dynamic changes in the vehicle's environment 10 for safety-critical perception data. The long-range cameras can have a limited FOV, and therefore their rotation is dynamic to capture information about changes in the road environment and to detect hazards at a great distance.Information about changes in the road environment at a great distance can be captured to pre-process information for the localization and lane-change modules. For localization, it is advantageous to have an image of the distant horizon, allowing for more accurate sensor and map data acquisition in advance. Lane-change modules require advance information about the distant horizon to process road environment changes, such as exit scenarios, enabling smoother lane changes. The lane-planning modules associated with System 100 also require this distant horizon information to determine a route. A distance-based trigger can be used to periodically search the road for hazards or lost cargo based on the vehicle's location. In one embodiment, the processing engine(s) 108 can be implemented as a combination of hardware and software (for example, programmable instructions) to implement one or more functionalities of the processing engine(s) 108. In the examples described herein, such combinations of hardware and software can be realized in various ways. For example, the software for the processing engine(s) 108 can consist of processor-executable instructions stored on a non-volatile, machine-readable storage medium, and the hardware for the processing engine(s) 108 can include a processing resource (for example, one or more processors) to execute such instructions. In other embodiments, the processing engine(s) 108 can be implemented by electronic circuits. In some embodiments, the one or more processors 102 can perform a location-based scenario check based on high-resolution map data via the processing engine(s) 108. The high-resolution map data can include, but are not limited to, information on road features, lane geometries, lane markings, and global positioning system (GPS) coordinates of a path oriented toward a destination. In some embodiments, the one or more processors 102 can apply an orientation value to the one or more cameras 120 via the processing engine(s) 108 to perform an online calibration of the one or more cameras 120 based on the location-based scenario check. In some embodiments, the one or more processor(s) 102 can receive, via the processing engine(s) 108, a plurality of still images captured by the one or more cameras 120 during the online calibration of the one or more cameras, and select one or more still images from the plurality of still images. In some embodiments, the one or more processor(s) 102 can, via the processing engine(s) 108, perform a backprojection onto one or more selected still images from the plurality of still images to determine a rotation angle for the one or more cameras 120. In some embodiments, the one or more processor(s) 102 can, via the processing engine(s) 108, update an orientation angle of the one or more cameras 120 based on the rotation angle of the one or more cameras 120.The alignment angle of one or more cameras 120 can be updated to dynamically adjust the alignment of one or more cameras 120. Figures 2A-2D illustrate a flowchart 200A of an exemplary method for adjusting the orientation of one or more cameras 120 mounted in the vehicle 10, an exemplary flowchart 200B representing a process for performing a location-based scenario check based on high-resolution map data, an exemplary flowchart representing a process for performing a backprojection onto one or more selected still images, and an exemplary representation representing the precalibration of one or more cameras according to the embodiments of the present disclosure. Referring to Fig. 2A, the method for adjusting the orientation of the one or more cameras 120 mounted in the vehicle 10 may include one or more steps. The one or more steps may be performed by the one or more processors 102 of the system 100. In the case of 202, the procedure can involve receiving high-resolution map data from vehicle 10. The procedure can involve performing a location-based scenario check based on the high-resolution map data. As illustrated in Fig. 2B, the location-based scenario check can be performed in the following steps. In 202-1, the method can involve extracting relevant information from high-resolution (HD) maps containing detailed data on road geometries, lane markings, and other important features required for navigation. The HD maps can be specifically designed for machine use and provide precise geocoded metadata that helps vehicles understand their surroundings better than conventional GPS systems. In 202-2, the method can involve creating a region of interest (ROI) around the current position of the vehicle 10 based on the high-resolution map data. The ROI can be created to focus on a specific area in front of the vehicle 10 where angle calculations can be performed. This ensures that the analysis focuses on the relevant data that directly affect the vehicle's trajectory and orientation. In 202-3, the procedure may involve extracting the lane geometry and centerline of an ego lane directly in front of vehicle 10. This includes identifying the lane boundaries and calculating the centerline coordinates, which serve as a reference for determining the vehicle's position within its lane. In 202-4, the procedure may involve analyzing the extracted data to identify all sub-paths adjacent to the current ego lane (the lane currently occupied by vehicle 10). This check is important for correcting the vehicle's alignment if it detects that it is deviating from its intended lane. In 202-5, once the relevant lanes have been identified, the procedure may involve facilitating a filtering mechanism to isolate the centerline data specifically for the current ego lane.This ensures that only relevant geometric information is used for further calculations, thereby minimizing potential navigation errors. In 202-6, the procedure can involve determining changes in curvature and slope of the road geometry within a predefined field of view (FOV) of camera 120 by filtering out the centerline data of the Ego lane. These metrics are important for understanding how upcoming road conditions affect vehicle dynamics, particularly steering and stability. If the change in curvature exceeds a predefined threshold, a tangent can be placed at that point. This can provide a range of orientation (ROI) distance, which is the distance between the vehicle coordinates and the location of the tangent's starting point. Furthermore, a maximum ROI can be approximately determined in both a transverse and longitudinal direction, depending on the FOV of camera 120. The ROI can be aligned by drawing a perpendicular from the tangent to the center of the road's centerline.The tangent can begin at a distance of "x" meters in front of vehicle 10. If it is determined that the FOV value in the x and y directions is greater than the curvature, the lane points furthest from the transverse and longitudinal directions within the FOV can be selected. The ROI distance can be calculated based on the speed of vehicle 10. For a typical straight road, the ROI distance can be 0. With a change in gradient, the tangent can be oriented downwards or upwards, and this can influence the gradient angle. The initial gradient angle can be determined by measuring the change in angle between the current position of the road attribute relative to vehicle 10 (near the horizon) and the position of the ROI at the distant horizon. In 202-7, the method may involve defining an initial FOV recommendation for the cameras 120 based on changes in curvature and slope of the road geometry. This recommendation can optimize visibility and detection capabilities and ensure that important information about road conditions and road geometry is effectively captured. Referring to Fig. 2A, the method in 204 may involve applying a reference value to the cameras 120 when performing the location-based scenario check. In 206, the method may involve performing an online calibration of the cameras 120 based on the location-based scenario check by applying the reference value to the cameras 120. In 208, the method may involve obtaining a plurality of still images captured by the cameras 120 during the online calibration of the cameras 120.In the case of 210, the procedure can involve selecting one or more individual images from the plurality of individual images relevant for performing camera-based perception and transmitting the one or more selected individual images to a perception system. The perception system can receive the one or more selected individual images along with one or more images from the near-field camera and one or more images from the mid-field camera to perform the camera-based perception. One or more near-field cameras and one or more mid-field cameras can be fixed without orientation, and only one or more far-field cameras can change orientation, so that the safety aspect of the vehicle 10 can remain intact due to the fixed near-field and mid-field cameras. In the 210a, camera-based perception can be performed by various stakeholders for object detection, object fusion, depth estimation, lane detection, lane fusion, localization, path planning, and more. For object detection, the captured images can be processed to identify and classify objects in the scene, such as vehicles, pedestrians, traffic signs, and lane markings. This often involves the use of machine learning and computer vision techniques. For lane detection, the 120 cameras can be used to detect lane markings on the road, providing assistance in staying in the lane and ensuring that the vehicle remains within its assigned lane.The camera data can be combined with information from other sensors, such as Light Detection and Ranging (LiDAR) sensors and radar, to obtain a more comprehensive understanding of the environment. Referring to Fig. 2A, the method at 212 can involve performing a backprojection onto one or more selected frames from the plurality of frames to determine a rotation angle for the one or more cameras 120. As illustrated in Fig. 2C, the rotation angle for the cameras 120 can be determined by the following steps. At 212-1, once the initial FOV is defined, certain map points of a most probable path, i.e., a path that leads the vehicle 10 to the selected destination, are chosen within the ROI. The map points can generally be derived from the high-resolution map data and can represent important features or landmarks that the vehicle 10 needs to recognize for navigation and decision-making.In 212-2, the procedure can involve determining a ROI-based rotation value for one or more cameras 120 based on one or more selected map points. This includes using an extrinsic calibration matrix to adjust the ROI based on the camera rotation and ensuring that the selected map points are correctly aligned with the camera perspective. If there are other partial routes in the vicinity, the vehicle's path and the most probable route can be checked to see if they are aligned with the destination. If it turns out that the vehicle 10 stays on the most probable route and does not take the partial routes, the camera alignment can be skipped, even if maximum map attributes lie in the partial routes.In 212-3, the procedure may involve using the extrinsic calibration matrix to backproject the selected map points onto the one or more selected individual images. This process translates one or more coordinates into the image coordinates and calculates the one or more image coordinates so that a maximum number of road geometry points can fit into the image coordinates. In 212-4, the procedure may involve checking whether the backprojected map points actually fit into the image coordinates. This ensures that all relevant features are visible within the camera's field of view (FOV), which is crucial for effective navigation. In 212-5, to evaluate the effectiveness of the FOV and point selection, the procedure may involve determining whether a certain percentage of the selected map points are within acceptable limits (above a predefined threshold).The procedure determines whether the maximum number of points of the selected map points lies within the image coordinates for this specific position of vehicle 10. This metric can determine whether adjustments to the camera alignment or FOV settings are necessary. In step 212-6, based on the successful verification and determination in steps 212-4 and 212-5, the procedure may involve iteratively adjusting an extrinsic rotation value in the calibration matrix to maximize the number of visible map points within the FOV. This optimization ensures that as many relevant features as possible are captured by camera 120. In step 212-7, if both the ROI-based rotation value and the rear-projection-based rotation values ​​are aligned with the pre-calibrated settings, the procedure may involve outputting the rotation values.This confirmation indicates that camera 120 is correctly aligned to the intended operating parameters. In accordance with 212-8, the procedure may involve determining a rotation angle for the tilt and yaw of camera 120 based on the ROI-based rotation value and the extrinsic rotation value, and outputting a final rotation angle that represents an optimized orientation for camera 120, ensuring that it effectively captures critical environmental data while maintaining alignment with high-resolution map data. For example, the rotation angle of camera 120 may be assumed to be 2 degrees of freedom, with the orientation changing at any given time only for yaw or tilt. The rotation angle may vary within a defined safety tolerance (for example, 5 degrees for yaw and 2 degrees for tilt in both directions). Referring to Fig. 2A, the method at 214 can include updating the orientation angle of the cameras 120 based on the rotation angle of the cameras 120. The orientation angle of the camera 120 can be updated to dynamically adjust the orientation of the camera 120. This can include validating the integrity between the ROI-based rotation value, the extrinsic rotation value, and the online calibration, and, based on successful validation, updating the orientation angle of the cameras 120. At 216, after updating the orientation angle of the cameras 120, the method can include independently determining one or more calibration parameters based on a new position of the cameras 120 and determining whether the one or more calibration parameters match a pre-calibration value.If one or more calibration parameters match the pre-calibration value, the procedure may involve adjusting the alignment of the cameras 120. If one or more calibration parameters do not match the pre-calibration value, the procedure may involve resetting the cameras 120 to their original position. Referring to Fig. 2D, all position steps (220, 222a, 222b, 224a, 224b, 226a, 226b, 228a, 228b) for camera 120 along the z-axis (yaw) and y-axis (pitch) can be defined to increase safety. Parameters influencing safety can include the overall range / distance of the mid-range camera, the rotation of camera 120 per second, and a maximum possible speed for autonomous driving. Additional steps can be provided depending on the vehicle's safety requirements. Within a unit of time, camera 120 can only perform one degree of freedom (either pitch or yaw), not both simultaneously. Extrinsic calibration can be performed for each of these positions, and the calibration values ​​for pitch and yaw can be stored.When calculating the camera's rotation angle based on map data reprojection, the procedure may include checking whether the rotation angle is aligned with the pre-calibration value at the current camera position. If the rotation angle matches the pre-calibration value, the online calibration check can be performed. Otherwise, the camera 120 can be reset to the 0-degree step position. During camera rotation, individual frames can be omitted if they are deemed unnecessary. Rotation steps along the y-axis can be calculated similarly for tilt. At a given time, the camera movement can be restricted to one axis. As illustrated in Fig. 2D, there can be four defined steps (222a, 222b, 224a, 224b, 226a, 226b, 228a, 228b) in each direction, and the rotations can be adjusted incrementally in decimal steps. By following these steps, autonomous systems can improve their perception capabilities, resulting in greater navigation accuracy and safety in dynamic driving environments. While the foregoing describes various embodiments of the present disclosure, other and further embodiments of the present disclosure may be formulated without deviating from its fundamental scope. The scope of the present disclosure is determined by the claims that follow. The present disclosure is not limited to the described embodiments, variants, or examples, which are included to enable a person skilled in the art to manufacture and use the present disclosure when combined with the information and knowledge available to such a person. The present disclosure dynamically adjusts the orientation of a vehicle-mounted camera in real time based on localized high-resolution (HD) map data. The present disclosure adjusts the camera's orientation and field of view (FOV) based on a vehicle's real-time environment, thereby ensuring that critical road information is detected earlier, improving decision-making during lane changes and increasing overall driving safety. The present disclosure allows for advance lane changes and better trajectory planning by adjusting the camera's orientation and FOV. QUOTES INCLUDED IN THE DESCRIPTION This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature US 10997737B2

[0004] US 20240100947A1

[0005]

Claims

System (100) for adjusting the orientation of one or more cameras (120) mounted in a vehicle (10), the system (100) comprising: a processor (102); and a memory (104) coupled to the processor (102), the memory (104) comprising one or more instructions executable by the processor which, when executed, cause the processor (102) to: perform a location-based scenario check based on the high-resolution map data; apply an orientation value to the one or more cameras (120) to perform an online calibration of the one or more cameras (120) based on the location-based scenario check; and obtain a plurality of still images taken by the one or more cameras (120) during the online calibration of the one or more cameras (120).Performing a back-projection of one or more selected individual images from the majority of individual images to determine a rotation angle for the one or more cameras (120); and updating an orientation angle of the one or more cameras (120) based on the rotation angle of the one or more cameras (120) to dynamically adjust the orientation of the one or more cameras (120). System (100) according to claim 1, wherein the high-resolution map data comprise information on road features, lane geometries, lane markings and global positioning system (GPS) coordinates of a path oriented towards a destination. System (100) according to claim 1, wherein the processor (102) performs the location-based scenario check by being configured to: create a region of interest (ROI) around a current position of the vehicle (10) based on the high-resolution map data; extract a lane geometry and a centerline of an ego lane in front of the vehicle (10) based on the ROI and a path directed towards a target; determine a change in curvature and a change in inclination of the road geometry within a predefined field of view (FOV) of the one or more cameras (120) by filtering out the centerline of the ego lane; and define an initial FOV for the one or more cameras (120) based on the change in curvature and the change in inclination of the lane geometry. System (100) according to claim 3, wherein the processor (102) creates the ROI based on the predefined FOV of the one or more cameras (120). System (100) according to claim 1, wherein the processor (102), after receiving one or more individual images taken by one or more cameras (120), shall send the one or more selected individual images together with one or more near-range camera images and one or more mid-range camera images to a perception system in order to perform camera-based perception. System (100) according to claim 1, wherein the processor (102) determines the rotation angle for the one or more cameras (120) by being configured to: once an initial FOV is defined, select one or more map points within a ROI; determine an ROI-based rotation value for the one or more cameras (120) using an extrinsic calibration matrix based on the one or more selected map points; backproject the one or more selected map points onto the one or more selected still images using the extrinsic calibration matrix; calculate one or more image coordinates from the one or more backprojected map points;Determine that the one or more back-projected map points fit into the one or more image coordinates and that a percentage of the one or more selected map points is higher than a predetermined threshold; based on the determination, adjust an extrinsic rotation value in the extrinsic calibration matrix to maximize a number of visible map points within the initial FOV; and determine a rotation angle for tilt and yaw of the one or more cameras (120) based on the ROI-based rotation value and the extrinsic rotation value. System (100) according to claim 6, wherein the processor (102) is to determine that the rotation angle corresponds to a pre-calibration value at a current position of the one or more cameras (120) after the rotation angle for the tilt and yaw of the one or more cameras (120) has been calculated. System (100) according to claim 1, wherein the processor (102) is to update the orientation angle of one or more cameras (120) by being configured to: validate an integrity between an ROI-based rotation value and an extrinsic rotation value; and, based on successful validation, update the orientation angle of one or more cameras (120). System (100) according to claim 8, wherein the processor (102) is configured, upon updating the orientation angle of one or more cameras (120), to: independently determine one or more calibration parameters based on a new position of one or more cameras (120); determine whether one or more calibration parameters match a pre-calibration value; if one or more calibration parameters match the pre-calibration value, adjust the orientation of one or more cameras (120); if one or more calibration parameters do not match the pre-calibration value, reset one or more cameras (120) to an original position. A method for adjusting the orientation of one or more cameras (120) mounted in a vehicle (10), comprising: (202) by a processor (102) associated with a system (100) performing a location-based scenario check based on high-resolution map data; (204) by the processor (102) applying an orientation value to the one or more cameras (120) to perform an online camera calibration based on the location-based scenario check; (208) by the processor (102) obtaining a plurality of still images taken by the one or more cameras (120) during the online camera calibration; (212) by the processor (102) performing a backprojection of one or more selected still images from the plurality of still images to determine a rotation angle for the one or more cameras (120);and update (214) by the processor (102) of a camera orientation angle based on the rotation angle of the one or more cameras (120) in order to dynamically adjust the orientation of the one or more cameras (120).

Images