Computer-implemented method for steering a vehicle taking into account a camera-based estimation of the vehicle's center of gravity

Abstract

The examples described herein provide a method that includes receiving an image from a vehicle's camera. The method further includes determining, using the image, the relative position and pose of the vehicle relative to a lane marking of a lane on a road occupied by the vehicle and on which the vehicle is traveling. The method further includes determining the relative center of gravity of the vehicle, at least partially, based on the vehicle's relative position and pose. The method also includes controlling the vehicle using an advanced driver assistance system based on a model of the vehicle, wherein the model utilizes the vehicle's relative center of gravity.

Description

BackgroundThe present invention relates to vehicles and, more particularly, to a computer-implemented method for controlling a vehicle in consideration of a camera-based vehicle centroid estimate for model-based vehicle control.For general background information, reference is made at this point beforehand to the publications DE 10 2022 201 221 A1, U.S. Pat. No. 2016 / 0 214 620 A1 and EP 3 330 147 B1.Modern vehicles (e.g., a car, a motorcycle, a boat, or any other type of automobile) may be equipped with one or more cameras that provide back-up assistance (e.g., backup-up assistance), capture images of the vehicle driver to determine the drowsiness or attention of the driver, provide images of the road while the vehicle is driving for purposes of collision avoidance, provide structure recognition (e.g., traffic signs, etc.), and / or the like, including combinations and / or multiples thereof. For example, a vehicle may be equipped with multiple cameras and images from multiple cameras (referred to as "surround view cameras") may be used to generate a "surround view" or "bird's-eye view" of the vehicle. Some of the cameras (referred to as "long-range cameras") may be used to capture long-range images (e.g., for object detection for collision avoidance, structure detection, etc.).Such vehicles may also be equipped with sensors such as a radar device(s), lidar device(s), and / or the like for perception tasks. Radio Detection and Ranging (radar) is a technology that uses radio waves to detect and determine the distance, speed, and angle of objects. Radar operates by emitting radio signals that bounce off objects and return to the radar system where the reflected waves are analyzed based on the time between emission and reception. The measured time may be used to determine the distance between the radar device and the detected one when performing perception tasks.Perception tasks may include one or more of object detection, classification, tracking, lane detection, traffic sign detection, and obstacle avoidance. Perception tasks are particularly useful for an autonomous or semi-autonomous vehicle to provide real-time awareness of its environment to the vehicle to make safe and informed driving decisions. Images from the one or more cameras of the vehicle may also be used to detect objects, track targets, and / or the like, including combinations and / or multiples thereof.The desire for precise vehicle control based on the center of gravity of the vehicle is of great importance for efficient operation of the vehicle.SummaryAccording to the invention, a computer-implemented method is presented which is distinguished by the features of claim 1. The method includes receiving an image from a camera of a vehicle. The method further includes determining, using the image, a relative position and pose of the vehicle relative to a lane marking of a lane of a road occupied by the vehicle and on which the vehicle is traveling. The method further includes determining a relative center of gravity position of the vehicle based at least in part on the relative position and pose of the vehicle. The method further comprises controlling the vehicle using an advanced driver assistance system based on a model of the vehicle, the model utilizing the relative center of gravity position of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the vehicle being mechanically coupled to a trailer, and wherein the relative center of gravity position of the vehicle is based at least in part on the trailer.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the advanced driver assistance system being an automated lane change system to cause the vehicle to make a lane change.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the advanced driver assistance system being a front collision warning system to generate a warning to an operator of the vehicle that alerts of a possible front collision.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include generating a warning indicative of a load displacement based at least in part on the relative center-of-gravity position of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include where the advanced driver assistance system is a imminent collision braking system to actuate brakes of the vehicle to reduce the speed of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include the advanced driver assistance system being an automated avoidance steering system to adjust the trajectory of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include wherein controlling the vehicle includes performing a perception task using at least the image and the model of the vehicle that uses the relative centroid position of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include determining the relative center of gravity position of the vehicle using the following equation: where ẏ c is a relative position to the lane marking, ψ c is a relative direction of travel of the vehicle to the lane marking, δ is a front wheel angle of the vehicle, v x is a longitudinal speed of the vehicle, L is a wheelbase of the vehicle determined by l f+ l r, l f is a distance of the relative center of gravity position from a front axle of the vehicle, and l r is a distance of the relative center of gravity position from a rear axle of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the method may include determining, responsive to the vehicle being mechanically coupled to a trailer, a lateral velocity of the vehicle v y,c using the following equation: where θ h is a hitch angle of the trailer relative to the vehicle.In another embodiment, a vehicle is provided. The vehicle includes a tow hitch for mechanically coupling the vehicle to the trailer, a camera, and a processing system. The processing system includes a memory having computer readable instructions and a processing device for executing the computer readable instructions, the computer readable instructions controlling the processing system to perform operations. The operations include receiving an image from the camera. The operations further include determining, using the image, a relative position and pose of the vehicle and the trailer relative to a lane marking of a roadway of a road occupied by the vehicle and the trailer and on which the vehicle and the trailer travel. The operations further include determining a relative center of gravity location of the vehicle and the trailer based at least in part on the relative position and pose of the vehicle and the trailer. The operations further include controlling the vehicle using an advanced driver assistance system based on a model of the vehicle and the trailer, the model utilizing the relative center of gravity position of the vehicle and the trailer.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include the advanced driver assistance system being an automated lane change system for causing the vehicle to make a lane change.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include the advanced driver assistance system being a front collision warning system to generate a warning to an operator of the vehicle that alerts of a possible front collision.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include the advanced driver assistance system being a imminent collision braking system to actuate the brakes of the vehicle to reduce the speed of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include the advanced driver assistance system being an automated avoidance steering system to adjust the trajectory of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include that controlling the vehicle comprises performing a perception task using at least the image and the model of the vehicle that uses the relative location of the center of gravity of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include determining the relative center of gravity position of the vehicle using the following equation: where ẏ c is a relative position to the lane marking, ψ c is a relative heading of the vehicle to the lane marking, δ is a front wheel angle of the vehicle, v x is a longitudinal speed of the vehicle, L is a wheelbase of the vehicle determined by l f+ l r, l f is a distance of the relative center of gravity position from a front axle of the vehicle, and l r is a distance of the relative center of gravity position from a rear axle of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the vehicle may include determining a lateral velocity of the vehicle v y,c using the following equation: where θ h is a hitch angle of the trailer relative to the vehicle, in response to the vehicle being mechanically coupled to a trailer.In another embodiment, a computer program product is provided. The computer program product includes a set of one or more computer readable storage media and program instructions stored in common in the set of one or more storage media for causing a processor set to perform computer operations. The operations include receiving an image from a camera of a vehicle. The operations include determining, using the image, a relative position and pose of the vehicle relative to a lane marking of a lane of a road occupied by the vehicle and on which the vehicle is traveling. The operations include determining a relative center of gravity location of the vehicle based at least in part on the relative position and pose of the vehicle. The operations include controlling the vehicle using an advanced driver assistance system based on a model of the vehicle, the model utilizing the relative center of gravity position of the vehicle.In addition to one or more of the features described herein, or as an alternative, further embodiments of the computer program product may include determining the relative center of gravity position of the vehicle using the following equation: where ẏ c is a relative position to the lane marking, ψ c is a relative direction of travel of the vehicle to the lane marking, δ is a front wheel angle of the vehicle, v x is a longitudinal speed of the vehicle, L is a wheelbase of the vehicle determined by l f+ l r, l f is a relative center of gravity location distance to a front axle of the vehicle and l r is a relative center of gravity location distance to a rear axle of the vehicle, and wherein in response to the vehicle being mechanically coupled to a trailer, a lateral velocity of the vehicle v y,c is determined using the equation: where θ h is a hitch angle of the trailer relative to the vehicle.The above features and advantages, as well as other features and advantages of the invention, will be readily apparent from the following detailed description when taken in conjunction with the accompanying drawings.Brief Description of the DrawingsOther features, advantages and details appear, by way of example only, in the following detailed description, wherein the detailed description refers to the drawings in which: FIG. 1 illustrates a vehicle having a processing system and a camera, according to one or more embodiments; FIG. 2 illustrates the processing system of FIG. 1, according to one or more embodiments; FIG. 3 illustrates a block diagram of a system for camera-based vehicle centroid estimation for model-based vehicle control, according to one or more embodiments; FIG. 4 illustrates a schematic diagram of a system for camera-based estimation of vehicle center of gravity for model-based vehicle control, according to one or more embodiments; FIG. 5A illustrates a flowchart of a method for camera-based estimation of vehicle centroid for model-based vehicle control, according to one or more embodiments; FIG. 5B illustrates a flowchart of a method for camera-based estimation of vehicle centroid for model-based vehicle control, according to one or more embodiments; and FIG. 6 illustrates a block diagram of a vehicle centroid camera-based estimation processing system for model-based vehicle control, according to one or more embodiments.Detailed DescriptionThe following description is merely exemplary in nature and is not intended to limit the present invention, its application, or uses. It should be understood that throughout the drawings, corresponding reference numerals indicate like or corresponding parts and features. As used herein, the term module refers to processing circuitry that may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (shared, dedicated, or group) and memory that executes one or more software or firmware programs, a combinational logic circuit, and / or other suitable components that provide the described functionality.As used herein, the term "controller" (e.g., a charging controller, as further described herein) refers to a dedicated controller including a processor and a memory, a general controller including control modules configured to implement a control process using the dedicated controller, a network of multiple different controllers in communication with each other and each including processors and memory and configured to cooperatively implement the control process, and any similar configuration for implementing the control process.One or more embodiments described herein relate to the camera-based center of gravity (CG) estimation for model-based vehicle control.Vehicles may use advanced driver assistance systems (ADAS) to improve vehicle performance and improve ride comfort by automatizing, adjusting, or improving vehicle systems to provide better attention, decision making, and control.An example of an ADAS is an adaptive cruise control (ACC) system that automatically adjusts the speed of a vehicle to maintain a safe following distance from a preceding vehicle. Another example of an ADAS is an automated lane change (ALC) system that causes the vehicle to make a lane change. Another example of an ADAS is a front collision alert (FCA) system to generate a warning to an operator of the vehicle that alerts a possible front collision. Another example of an ADAS is a collision minimal braking (CIBA) system to apply vehicle brakes to reduce the speed of the vehicle. Another example of an ADAS is an automated evasion steering (AES) system to adjust the trajectory of the vehicle.ADASs often use data (referred to as "sensor data") from sensors (e.g., RADAR sensors, LiDAR sensors, proximity sensors, etc.), images from cameras, and / or the like, including combinations and / or multiples thereof, to perform perception tasks, make decisions, and control one or more aspects of the vehicle.Modern vehicle systems rely on advanced technologies to perform perception tasks such as detecting, classifying, and tracking objects. These capabilities are useful for systems that enable accurate and efficient navigation, including semi-autonomous or autonomous operation of a vehicle by understanding an environment of the vehicle in real-time. Challenges arise when the CG of a vehicle is unknown or moves from a known location to an unknown location. For example, a CG of a vehicle may be known based on manufacturing specifications. However, the CG of the vehicle may change due to changes in weight distribution such as a load added to the vehicle, a trailer connected to the vehicle, and / or the like, including combinations and / or multiples thereof.One or more embodiments described herein utilize vehicle cameras, such as those cameras associated with ADAS, to estimate the CG location of a vehicle for model-based vehicle control of the vehicle. According to one or more embodiments, a technique for estimating or determining a center of rotation and CG location of a vehicle using perception data (e.g., images from a camera) is provided.According to one or more embodiments, a mathematical formulation is provided to model vehicle motion from detected lane markings (e.g., from images of a camera) to calculate the accuracy of the CG location. According to one or more embodiments, a technique is provided for systematically calculating contextual conditions suitable for learning the CG pose for the vehicle and updating the CG parameter to correct the suboptimal CG pose.FIG. 1 shows a vehicle 100 including a processing system 102 and a camera 104, according to one or more embodiments. The vehicle 100 may be a car, truck, van, bus, motorcycle, boat, or other type of automobile. According to one embodiment, the vehicle 100 is a hybrid electric vehicle, such as a partially or fully electric energy operated plug-in hybrid electric vehicle (PHEV). According to another embodiment, the vehicle 100 is an electric vehicle operated with electrical energy. A battery (not shown) is used to provide electrical energy to components of the vehicle 100, such as an electric motor (not shown), electrical components (not shown), and / or the like, including combinations and / or multiples thereof. According to one or more embodiments, the vehicle 100 is an autonomous or semi-autonomous vehicle. An autonomous vehicle is a vehicle that has autonomous driving capabilities. A semi-autonomous vehicle is a vehicle that has certain autonomous features (e.g., autonomous parking, lane keeping, etc.) but does not have full autonomous control.The processing system 102 is located within the vehicle and is responsible for managing and processing data (e.g., images) collected by the camera 104. The camera 104 is strategicly positioned on the vehicle 100 to capture images of the environment of the vehicle, such as a lane in which the vehicle 100 is travelling. The arrows between the camera 104 and the processing system 102 indicate the flow of data (e.g., images) from the camera 104 to the processing system 102, emphasizing the interaction between these components. This configuration enables the vehicle 100 to perform perception tasks that can be used for autonomous driving, for example, using the data (e.g., images) collected by the camera 104.Further features of the processing system 102 and the camera 104 will now be described with reference to FIG. 2.FIG. 2 specifically illustrates the processing system of FIG. 1, according to one or more embodiments. According to one or more embodiments, the processing system 102 includes a processing device 202, a memory 204, a CG engine 210, and a perception task engine 212. It should be appreciated that the processing system 102 may be any device suitable for camera-based estimation of vehicle center of gravity for model-based vehicle control. For example, the processing system 102 may be a device implemented in or otherwise associated with the vehicle 100, such as an electronic control unit (also referred to as an electronic control module). As another example, the processing system 102 may be a smartphone, a tablet computer, a laptop computer, a desktop computer, a handheld computing device, and / or the like, including combinations and / or multiples thereof. As yet another example, the processing system 102 may be the processing system 600 of FIG. 6 and / or may include one or more components of the processing system 600 of FIG. 6.The processing device 202 is responsible for executing instructions and managing the overall operation of the processing system 102. Processing device 202 may be any suitable processing circuitry for executing instructions and processing data. The processing device 202 may be, for example, a microcontroller, a microprocessor, an application specific integrated circuit (ASIC), or any other type of processing unit capable of handling the computational requirements of the processing system 102. The processing device 202 is an example of one or more of the processing devices 621 of FIG. 6, as described in more detail herein.The memory 204 stores data (e.g., images 211), computer readable instructions, and algorithms useful for operation of the processing system 102. This may include processing real-time data, analysis of historical data, and storage of firmware or software programs. The memory 204 is any suitable device for storing data, such as the images 211, and / or instructions. The memory 204 may be, for example, a combination of volatile memory (e.g., random access memory) and nonvolatile memory (e.g., read only memory, flash memory). Memory 204 is an example of one or more of system memory 622, random access memory 623, and / or read only memory 624 of FIG. 6, as described in more detail herein.The processing system 102 receives images 211 (from the camera 104) of objects, such as a target object, in an environment in which the vehicle 100 is operating. According to one or more embodiments, the images 211 may be images of a lane in which the vehicle 100 is traveling, including any lane markings (e.g., lane lines, turn indicators, etc.) of the lane. For example, the images 211 may be useful to perform perception tasks, which in turn are used to control the vehicle using an ADAS 214.The CG engine 210 is responsible for determining a CG of the vehicle 100. To do so, the CG engine 210 uses the images 211 to measure a relative position and pose of the vehicle with respect to lane markings, and then calculates the relative CG pose of the vehicle based on the relative position and pose of the vehicle with respect to the lane markings. Features and functions of the CG engine 210 will be further described with reference to FIGS. 3-5B.The perception task engine 212 processes the images 211 to perform various perception tasks such as object detection, classification, and tracking. It uses the images 211 to enable real-time awareness of the environment of the vehicle 100, including any target objects. The perception task engine 212 is useful for applications such as autonomous driving in which accurate and timely perception is utilized for efficient and effective navigation. By effectively utilizing advanced algorithms and processing techniques, the perception task engine 212 can interpret complex data sets such as the images 211, allowing the vehicle 100 (or an operator of the vehicle 100) to make informed decisions. According to one or more embodiments, the perception task engine 212 enables the vehicle 100 to navigate through its environment autonomously or semi-autonomously with reduced manual intervention requirements.According to one or more embodiments, the perception task engine 212 may be used in combination with an autonomous driving system, such as the ADAS 214, to control the capabilities of autonomous navigation of the vehicle 100, allowing the vehicle 100 to navigate with respect to detected objects. According to one or more embodiments, the autonomous driving system processes information received from the perception task engine 212 and / or images 211 received from the camera 104 to determine the exact location and orientation of the vehicle 100. The ADAS 214 then generates control signals to steer, accelerate, or brake the vehicle 100 as desired to securely and efficiently navigate the vehicle 100 within its environment. The ADAS 214 ensures that the vehicle 100 can perform complex maneuvers autonomously, reducing the need for manual interventions.FIG. 3 illustrates a block diagram of a camera-based vehicle centroid estimation system 300 for model-based vehicle control, according to one or more embodiments.The system 300 may be implemented using a vehicle 100 equipped with a camera 104 and a processing system 102. The vehicle 100 is mechanically coupled to a trailer 301. Together, the vehicle 100 and the trailer 301 have a CG layer 302 based on the size and weight of the vehicle 100 and the size and weight of the trailer 301.The vehicle 100 measures the relative position and pose of the vehicle 100 and a trailer 301 at block 310, calculates the relative CG pose for the vehicle 100 and the trailer 301 at block 312, and updates the motion estimate for control purposes at block 314.More specifically, at block 310, the processing system 102 measures the relative position and pose of the vehicle 100 and the trailer 301 with respect to lane markings of a lane 306 in which the vehicle 100 is traveling. The camera 104 captures images that are processed to determine the position and location of the vehicle 100 within the lane 306.The processing system 102 first determines a speed of the vehicle 100 using the images 211. Specifically, the processing system 102 determines a camera-based speed v y,c from lane markings given by the relative position to lane marking ẏ c, the relative heading of the vehicle to lane marking ψ c and the longitudinal speed v x, which may be derived from a front camera module (e.g., FCM 422 in FIG. 4 (e.g., camera 104)), a wheel speed sensor (e.g., WSS 424 in FIG. 4 ), a steering angle sensor (e.g., SAS 426 in FIG. 4 ), and / or an inertial measurement unit (IMU) (not shown) using the following equation:According to one or more embodiments, in low speed scenarios (e.g., less than about 8 km / h (5 miles per hour)), the kinematic-based speed may be accurately estimated according to the following equation: where δ is the front wheel angle of the vehicle 100 (from SAS 426), L is the wheelbase of the vehicle 100 determined by l f+ l r l f is a distance of the CG layer 302 from the front axle of the vehicle 100, and l r is a distance of the CG layer 302 from the rear axle of the vehicle 100. The same equation may be applied to a vehicle having a tail-in system (tail-in system) (e.g., the vehicle 100 and the trailer 301) in the form of a reduced model. If vision-based hitch angle information is available (e.g., detected by camera 104 or another suitable camera), a similar approach may be applied to estimate the lateral speed of vehicle 100 according to the following equation: where θ h is the hitch angle of trailer 301 relative to vehicle 100.Next, in block 312, the processing system 102 calculates the relative CG location (e.g., the CG location 302) of the vehicle 100 and the trailer 301. This calculation uses the measured position and pose data of block 310 to determine the CG location 302 that is important for stability and control. For example, the CG location 302 for the vehicle (and also for the reduced model for the trailer 301) may be determined using the following equation:The CG engine 210 is designed and activated when v x δ is greater than an error value ∈ 0, according to the following equation: where g k is an adaptive gain filter.At block 314, the processing system 102 updates the motion estimate for the vehicle 100 and the trailer 301 based on the CG pose 302 determined at block 312. This update ensures that the perception task engine 212 and / or the ADAS 214 have accurate data to make real-time adjustments to control the vehicle 100.At block 316, the processing system 102 updates a support load calculation and robust lateral controls for the vehicle 100 and the trailer 301. To do this, the load distribution and the lateral stability controls are adjusted to take into account the center of gravity location 302.In block 318, the processing system 102 notifies the driver of the vehicle 100 of any load shift. This notification provides the driver with information about changes in load distribution that may affect the vehicle's driving behavior.In block 320, the processing system 102 updates the ADAS controller with the new centroid location. This update allows the driver assistance systems to use the most up-to-date data for vehicle control, which improves safety and performance.FIG. 4 illustrates a schematic diagram of a vehicle centroid estimation system 400 for model-based vehicle control, according to one or more embodiments.The system 400 may be implemented using the processing system 102 of FIGS. 1 and 2, which receives inputs from various sensors (e.g., the camera 104) and performs calculations to estimate the center of gravity of the vehicle 100. The system 400 comprises a load position learning block 402, which comprises enabling criteria 404 and a learning and adaptation filter 412.The load position learning block 402 processes inputs from the FCM 422, WSS 424, and SAS 426. These inputs include lane markings, wheel speeds, and steering angle used to determine the dynamics of the vehicle 100.The enabling criteria 404 judges conditions for the process of learning the load position. This includes performing an excitation check 406, an error variance check 408, and a velocity check 410 to ensure that the data is suitable for further processing. For example, the excitation test 406 compares a prediction from the model to a measured value to identify a difference to obtain meaningful information to update the learning and adaptation filter 412. The error variance check 408 determines how much the error (e.g., the error in prediction and measurement) changes in a time window (e.g., whether the error becomes larger or smaller), where the error is given by the following equation set forth above:The speed check 410 determines the speed of the vehicle 100.The learning and adaptation filter 412 refines the estimation process. The vision-based speed estimate 414 calculates the lateral speed of the vehicle 100 as described herein. The prediction model 416 predicts the CG location 302 of the vehicle 100. The prediction error calculation block 418 evaluates the accuracy of the predictions. The statistical filter 420 processes the prediction errors to improve the reliability of the centroid estimation.The calibration block 428 sets an initial value for the CG location (e.g., as specified by the manufacturer) specific to the vehicle 100. This initial value for the CG location is adjusted to determine the CG location 302 based on changes in the loads and configurations (e.g., attached trailer) of the vehicle 100, according to one or more of the embodiments described herein.FIG. 5A illustrates a flowchart of a method 500 for camera-based estimation of vehicle centroid for model-based vehicle control, according to one or more embodiments. The method 500 may be implemented using any suitable system or apparatus. For example, the method 500 and its steps may be implemented using the processing system 102 of FIGS. 1 and 2, by the processing system 600 of FIG. 6, and / or the like, including combinations and / or multiples thereof. The method 500 will now be described with reference to at least portions of FIGS. 1-4, but is not so limited.The method 500 includes multiple steps to predict and adjust the dynamics and control of the vehicle 100.At block 502, the method 500 predicts the tow hitch angle of the trailer. This prediction uses data from the wheelbase and center of gravity of the trailer 520, as well as inputs from the FCM 422, map 524, and inertial measurement unit (IMU) 526.At block 504, the method 500 establishes predictive joint dynamics using a CG location estimate of block 505. This step includes using the predicted hitch angle to model the dynamics behavior of the trailer and vehicle system (e.g., the vehicle 100 with the trailer 301).At block 506, the method 500 predicts the lateral offset of the trailer at a look-ahead distance. This prediction helps to understand the position of the trailer relative to the roadway of the vehicle.At block 508, the method 500 adjusts the vehicle to correctly position the trailer. This adjustment ensures that the trailer remains aligned with the intended path of the vehicle.In block 510, the method 500 prevents the trailer from departing the intended path. This step includes implementing control actions to maintain the trailer within the designated lane.In block 512, the method 500 performs a cross control. This control maintains the stability and orientation of the vehicle and trailer while driving.In block 514, the method 500 issues a warning of leaving the trailer lane. This warning alerts the driver or autonomous system if the trailer begins to depart from the intended lane.Additional processes may also be included, and it should be understood that the processes depicted in FIG. 5A are illustrative only, and that other processes may be added or existing processes may be removed, modified, or rearranged. It should also be understood that the processes illustrated in FIG. 5A may be implemented as programmatic instructions stored on a non-transitory computer readable storage medium that, when executed by a processor (e.g., the processing device 202 of FIG. 2, the processor(s) 621 of FIG. 6, and / or the like, including combinations and / or multiples thereof) of a computer system (e.g., the processing system 102 of FIGS. 1 and 2, the processing system 600 of FIG. 6, and / or the like, including combinations and / or multiples thereof), cause the processor to execute the processes described herein.FIG. 5B shows a flowchart of a method 550 for camera-based estimation of vehicle centroid for model-based vehicle control, according to one or more embodiments. The method 550 may be implemented using any suitable system or apparatus. For example, the method 550 and its steps may be implemented using the processing system 102 of FIGS. 1 and 2, by the processing system 600 of FIG. 6, and / or the like, including combinations and / or multiples thereof. The method 500 will now be described with reference to at least portions of FIGS. 1-4, but is not so limited.At block 552, the method 550 begins with the processing system 102 receiving an image (e.g., one or more of the images 211 captured by the camera 104).In block 554, the CG engine 210 determines, using the image, a relative position and pose of the vehicle 100 relative to a lane marking of a roadway (e.g., the lane 306) of a road occupied by the vehicle 100 and on which the vehicle 100 is traveling.At block 556, the CG engine 210 determines a relative center-of-gravity position of the vehicle 100 based at least in part on the relative position and pose of the vehicle 100 determined at block 554.Finally, in block 558, the ADAS 214 controls the vehicle 100 based on a model of the vehicle 100 that utilizes the relative center of gravity position of the vehicle determined in block 556.According to one or more embodiments, controlling the vehicle 100 may include performing a perception task performed by the perception task engine 212 using the image from the camera 104 and the model of the vehicle 100 that uses the centroid location determined in block 556. More specifically, a perception task is performed using the aggregated radar data of the target object aggregated for the time period specific to the reflection point as defined by the dynamic aggregation duration. Perception tasks, as performed by the perception task engine 212, include, for example, processing the images (e.g., images captured by the camera 104) to detect, classify, and track objects in the environment of the vehicle 100. These tasks are useful for real-time awareness, which allows the vehicle 100 to make informed decisions and operate efficiently. For example, perception tasks in autonomous driving help identify obstacles, traffic signs, and other vehicles, enabling efficient navigation. The perception task engine 212 integrates data collected by the camera 104 and the model of the vehicle 100 that uses the center of gravity location determined in block 556 to increase the accuracy and reliability of these perception tasks.Additional processes may also be included, and it should be understood that the processes depicted in FIG. 5B are illustrative and that other processes may be added or existing processes may be removed, modified, or rearranged. It should also be understood that the processes illustrated in FIG. 5B may be implemented as programmatic instructions stored on a non-transitory computer readable storage medium that, when executed by a processor (e.g., the processing device 202 of FIG. 2, the processor(s) 621 of FIG. 6, and / or the like, including combinations and / or multiples thereof) of a computer system (e.g., the processing system 102 of FIGS. 1 and 2, the processing system 600 of FIG. 6, and / or the like, including combinations and / or multiples thereof), cause the processor to perform the processes described herein.One or more embodiments of camera-based vehicle centroid estimation for model-based vehicle control provide the following significant technical improvements and advantages.One or more embodiments provide improved vehicle stability and control. For example, by accurately estimating the center of gravity position of the vehicle, the system may improve the stability and control of the vehicle. This is particularly useful for vehicles towing trailers where the CG may shift and affect driving behavior. The accurate CG estimation enables better control algorithms, resulting in a safer and more stable driving experience.One or more embodiments provide improved advanced driver assistance systems. For example, one or more embodiments improve performance of various ADAS functions such as automatic lane change, front collision warning, imminent collision braking, and automatic avoidance steering. By utilizing the accurate CG location in the vehicle model, these systems can make more informed decisions, resulting in more effective and reliable assistance to the driver.One or more embodiments provide real-time load shift detection. For example, one or more embodiments may generate warnings indicative of a load shift based on the relative CG location. This is particularly useful for vehicles transporting different loads or towing trailers because it provides real-time feedback to the driver about changes in load distribution that may affect the vehicle's driving behavior.One or more embodiments provide robustness against model uncertainties. For example, one or more embodiments include a control robustness strategy to adjust the CG location of the vehicle and trailer. This ensures that the vehicle controller remains effective even in the presence of model uncertainties or changes in the load configuration of the vehicle.One or more embodiments provide systematic learning and adaptation. For example, one or more embodiments include an estimator excitation monitor (monitor) that systematically calculates the required contextual conditions to learn the CG pose. The estimator updates the parameters to correct suboptimal CG locations, ensuring continuous improvement and adaptation to changing conditions.One or more embodiments provide integration with perception tasks. For example, one or more embodiments use camera-based perception data to estimate CG location. This integration enables more accurate and reliable perception tasks such as object detection, classification, and tracking that are essential for autonomous and semi-autonomous driving.One or more embodiments provide flexibility for various types of vehicles. For example, one or more embodiments may be applied to various types of vehicles including cars, trucks, vans, buses, motor cycles, boats, and more. It is also suitable for various vehicle configurations such as those with trailers, making it a versatile solution for a wide range of applications.One or more embodiments provide improved reliability and performance. For example, by providing accurate CG location data and improving performance of ADAS features, one or more embodiments improve overall reliability and performance of the vehicle. This results in a more comfortable and safer driving experience for the driver or operator and passengers.Overall, the embodiments described herein provide a comprehensive solution for improving vehicle control and safety through accurate estimation and utilization of the vehicle's center of gravity.It should be appreciated that one or more embodiments described herein may be implemented in connection with any other type of computing environment known today or later developed. For example, FIG. 6 illustrates a block diagram of a processing system 600 for implementing the techniques described herein. According to one or more embodiments described herein, processing system 600 is an example of a cloud computing node of a cloud computing environment. In examples, processing system 600 includes one or more central processing units (also referred to as "processors" or "processing resources" or "processing devices") 621 a, 621 b, 621 c, etc. (collectively or generally referred to as processor(s) 621 and / or processing device(s) 621). In aspects of the present invention, each processor 621 may include a reduced instruction set computer (RISC) microprocessor. Processors 621 are coupled to system memory 622 and / or various other components via a system bus 633. System memory 622 may include one or more temporary and / or persistent storage devices such as random access memory (RAM) 623, read-only memory (ROM) 624, and / or the like, including combinations and / or multiples thereof. The system bus 633 may include a basic input / output system (BIOS) that controls certain basic functions of the processing system 600.Also shown are an input / output (I / O) adapter 627 and a network adapter 626 coupled to the system bus 633. I / O adapter 627 may be a small computer system interface (SCSI) adapter that communicates with hard disk 635 and / or storage device 636 or any other similar component. I / O adapter 627, hard disk 635, and storage device 636 are collectively referred to herein as mass storage 634. Operating system 640 for execution on processing system 600 may be stored in mass storage 634. Network adapter 626 connects system bus 633 to an external network 638, allowing processing system 600 to communicate with other such systems.A display (e.g., a display monitor) 639 is connected to the system bus 633 via the display adapter 632, which may include a graphics adapter to improve performance of graphics intensive applications and a video controller. In one aspect of the present invention, adapters 626, 627, and / or 632 may be connected to one or more I / O buses connected to system bus 633 via an intermediate bus bridge (not shown). Suitable I / O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols such as Peripheral Component Interconnect (PCI). Additional input / output devices are connected to the system bus 633 via a user interface adapter 628 and the display adapter 632. A keyboard 629, mouse 630 and speaker 631 may be connected to the system bus 633 via the user interface adapter 628, which may comprise, for example, a super I / O chip that integrates multiple device adapters into a single integrated circuit.In some aspects of the present invention, processing system 600 includes a graphics processing unit (GPU) 637. Graphics processing unit 637 is a specialized electronic circuit designed to manipulate and alter memory to speed up the generation of images in a frame buffer intended for output to a display. In general, the graphics processing unit 637 is very efficient in manipulating computer graphics and image processing and has a highly parallel structure that makes it more effective than general purpose CPUs for algorithms in which large blocks of data are processed in parallel.Thus, as configured herein, processing system 600 includes processing capability in the form of processors 621, storage capability including system memory 622 and mass storage 634, input means such as keyboard 625 and mouse 630, and output capability including speaker 631 and display 639. In some aspects of the present invention, a portion of system memory 622 and mass storage 634 collectively store operating system 640 to coordinate the functions of the various components shown in processing system 600.

Claims

A computer-implemented method, comprising: receiving an image from a camera (104) of a vehicle (100); determining, using the image, a relative position and pose of the vehicle (100) relative to a lane marking of a lane (306) of a road occupied by the vehicle (100) and on which the vehicle (100) is travelling; determining a relative center of gravity location of the vehicle (100) based at least in part on the relative position and pose of the vehicle (100); and controlling the vehicle (100) using an advanced driver assistance system based on a model of the vehicle (100), wherein the model utilizes the relative center of gravity location of the vehicle (100).The computer-implemented method of claim 1, wherein the vehicle (100) is mechanically coupled to a trailer (301), and wherein the relative center of gravity location of the vehicle (100) is based at least in part on the trailer (301).The computer-implemented method of claim 1, wherein the advanced driver assistance system is an automated lane change system for causing the vehicle (100) to make a lane change.The computer implemented method of claim 1, wherein the advanced driver assistance system is a front collision warning system to generate a warning to an operator of the vehicle (100) that alerts a possible front collision.The computer-implemented method of claim 1, further comprising generating a warning indicative of a load displacement based at least in part on the relative center of gravity position of the vehicle (100).The computer implemented method of claim 1, wherein the advanced driver assistance system is a imminent collision braking system to apply the brakes of the vehicle (100) to reduce the speed of the vehicle (100).The computer-implemented method of claim 1, wherein the advanced driver assistance system is an automated avoidance steering system to adjust a trajectory of the vehicle (100).The computer-implemented method of claim 1, wherein controlling the vehicle (100) comprises performing a perception task using at least the image and the model of the vehicle (100) that uses the relative center of gravity location of the vehicle (100).The computer-implemented method of claim 1, wherein the relative center of gravity position of the vehicle (100) is determined using the equation: l f = ( y%0020̇ c - ψ c v x ) L v x δ where ẏ c is a relative position to the lane marking, ψ c, is a relative heading direction of the vehicle (100) to the lane marking, δ is a front wheel angle of the vehicle (100), v x is a longitudinal speed of the vehicle (100), L is a wheelbase of the vehicle (100) determined by l f+ l r, l f is a distance of the relative center of gravity position to the front axle of the vehicle (100), and l r is a distance of the relative center of gravity position to the rear axle of the vehicle (100), and wherein in response to the vehicle (100) being mechanically coupled to a trailer (301), a lateral speed of the vehicle (100) v y,c is determined using the following equation: v y, c = v x δ l f L + v x θ h l t L t wherein θ h is the hitch angle of the trailer (301) relative to the vehicle (100).A vehicle (100) mechanically coupled to a trailer (301), the vehicle (100) comprising: a tow hitch for mechanically coupling the vehicle (100) to the trailer (301); a camera (104); and a processing system (600) comprising: a memory having computer readable instructions; and a processing device (621) for executing the computer readable instructions, the computer readable instructions controlling the processing system (600) to perform operations comprising: receiving an image from the camera (104); determining, using the image, a relative position and pose of the vehicle (100) and the trailer (301) relative to a lane marking of a lane (306) of a road occupied by the vehicle (100) and the trailer (301) and on which the vehicle (100) and the trailer (301) travel; determining a relative center of gravity position of the vehicle (100) and the trailer (301) based at least in part on the relative position and pose of the vehicle (100) and the trailer (301); and controlling the vehicle (100) using an advanced driver assistance system based on a model of the vehicle (100) and the trailer (301), wherein the model uses the relative center of gravity position of the vehicle (100) and the trailer.

Images