A chassis control method, a chassis control system, an electronic device, and a storage medium

Abstract

The application discloses a chassis control method, a chassis control system, an electronic device and a storage medium, and relates to the technical field of vehicles, and comprises the following steps: first, acquiring state data of a vehicle, collecting images of an external environment of the vehicle through a camera module of the vehicle, and collecting point cloud data of the external environment of the vehicle through a laser radar of the vehicle; then, acquiring environmental information outside the vehicle according to the images and the point cloud data; and finally, controlling a chassis of the vehicle according to the environmental information and the state data, so that the vehicle can adapt to the external environment of the vehicle, thereby avoiding the problem that, in the case that a driver is distracted, the driver cannot timely control the vehicle according to the external environment of the vehicle, and thus a danger occurs.

Description

Technical Field

[0001] This application belongs to the field of vehicle technology, specifically relating to a chassis control method, a chassis control system, electronic equipment, and a storage medium. Background Technology

[0002] In related technologies, images of the vehicle's external environment are captured by the vehicle's camera and displayed on a screen, allowing the driver to control the vehicle in a timely manner based on the external environment.

[0003] However, when drivers are distracted, they cannot control the vehicle in time according to the external environment, leading to danger. Summary of the Invention

[0004] This application aims to provide a chassis control method, chassis control system, electronic device and storage medium, which at least solves the problem in the related art that when the driver is distracted, the driver cannot control the vehicle in time according to the external environment, resulting in danger.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a chassis control method, the method comprising:

[0007] The system acquires vehicle status data by using the vehicle's camera module to collect images of the vehicle's external environment and by using the vehicle's lidar to collect point cloud data of the vehicle's external environment.

[0008] Based on the image and the point cloud data, obtain the environmental information outside the vehicle;

[0009] The chassis of the vehicle is controlled based on the environmental information and the status data.

[0010] Secondly, embodiments of this application also provide a chassis control system, including: a front-view integrated unit and a chassis control module;

[0011] The forward-looking integrated unit is connected to the chassis control module via a network line. The forward-looking integrated unit is used to collect images and point cloud data of the vehicle's external environment, as well as to acquire the vehicle's inertial measurement data.

[0012] The chassis control module is used to acquire sensor data of the vehicle through the vehicle's sensors, generate state data of the vehicle based on the inertial measurement data and the sensor data, acquire environmental information of the vehicle's external environment based on the image and the point cloud data, and control the vehicle's chassis based on the environmental information and the state data.

[0013] Thirdly, embodiments of this application also provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application also provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0015] In this embodiment, the vehicle's status data is first acquired, and images of the vehicle's external environment are collected through the vehicle's camera module, as well as point cloud data of the vehicle's external environment are collected through the vehicle's LiDAR. Then, based on the images and point cloud data, environmental information about the vehicle's external environment is obtained. Finally, based on the environmental information and status data, the vehicle's chassis is controlled to enable the vehicle to adapt to the external environment, thereby avoiding the problem that the driver cannot control the vehicle in time according to the external environment when the driver is distracted, which could lead to a dangerous situation. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of a chassis control method provided in an embodiment of this application;

[0017] Figure 2 This is a flowchart illustrating the specific steps of a chassis control method provided in an embodiment of this application;

[0018] Figure 3 This is a schematic diagram of a chassis control system provided in an embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0022] The chassis control method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0023] Figure 1 This is a flowchart illustrating the steps of a chassis control method provided in an embodiment of this application, as follows: Figure 1 As shown, the method may include:

[0024] Step 101: Obtain vehicle status data by acquiring images of the vehicle's external environment through the vehicle's camera module and point cloud data of the vehicle's external environment through the vehicle's lidar.

[0025] In some embodiments, vehicle status data includes tire torque, suspension damping force, vehicle speed, and chassis height.

[0026] In this embodiment of the application, vehicle status data is acquired by using the vehicle's camera module to collect images of the vehicle's external environment and by using the vehicle's lidar to collect point cloud data of the vehicle's external environment. Based on the images and point cloud data, environmental information about the vehicle's external environment is obtained, and the vehicle's status is adjusted based on the environmental information and status data.

[0027] Step 102: Obtain environmental information outside the vehicle based on the image and the point cloud data.

[0028] In some embodiments, the environmental information outside the vehicle includes road surface information, obstacle recognition results, road signs, traffic light information, etc.

[0029] In some embodiments, the types of environmental information include first environmental information obtained from an image, second environmental information obtained from point cloud data, and third environmental information obtained by deduplicating and fusing the first and second environmental information; wherein, the first environmental information includes first road surface information, first obstacle recognition result, road signs, traffic light information, etc.; and the second environmental information includes second road surface information, second obstacle recognition result, etc.

[0030] In some embodiments, optical flow tracking is performed on the first environmental information of consecutive frames and the second environmental information of consecutive frames to improve the robustness of dynamic scenes.

[0031] In some embodiments, first environmental information is obtained from the image, i.e., in a camera-based image segmentation algorithm, semantic segmentation and deep learning methods are combined to detect waterlogged roads or potholes.

[0032] In some embodiments, lens distortion of the camera module is eliminated through camera (i.e., camera module) calibration. The intrinsic parameter matrix K and distortion coefficients D are calculated using the Zhang Zhengyou calibration method, along with the image coordinates (u, v) and the coordinates (X) in the camera module coordinate system. C ,Y C Z C The mapping relationship is as follows:

[0033]

[0034] Where u is the u-axis coordinate of the pixel in the image, v is the v-axis coordinate of the pixel in the image, and X... C Let C be the X-axis coordinate of point C in the camera module coordinate system, and Y be the Y-axis coordinate of point C. C Let C be the Y-coordinate of point C in the camera module coordinate system, and Z be the Z-coordinate. C Let C be the coordinate of point C on the Z-axis of the camera module coordinate system; the camera module coordinate system is the real-world coordinate system with the vehicle as the origin.

[0035] In some embodiments, Contrast-Limited Adaptive Histogram Equalization (CLAHE) or the Retinex algorithm (an image enhancement algorithm) is used to enhance the illumination of pixels in backlit or low-contrast regions.

[0036] In some embodiments, a semantic segmentation model is used to process the image to obtain the segmentation result (i.e., the segmented binary image). Semantic segmentation models include U-Net, DeepLabv3+, BiSeNet, etc. Among them, U-Net is an encoder-decoder structure, which is suitable for small sample data, and skip connections preserve detailed features; DeepLabv3+ can combine atrous convolution (AtrousConv) and Atrous Spatial Pyramid Pooling (ASPP) modules to improve multi-scale feature extraction capabilities; BiSeNet is a lightweight model that balances real-time performance and accuracy (requires >15FPS for vehicle scenarios).

[0037] In some embodiments, the semantic segmentation model employs a weighted cross-entropy loss function, assigning higher weights to waterlogged roads and potholes. The weighted cross-entropy loss function is as follows:

[0038]

[0039] Where Λ represents the loss function, i is the number of classes, and c i This categorizes uneven areas, for example, c=1 represents potholes, c=2 represents speed bumps, c=3 represents smooth road surfaces, and c=4 represents flooded road surfaces; y i For real labels, p i To predict probabilities.

[0040] In some embodiments, a closing operation (dilation followed by erosion) is performed on the segmentation result to fill small holes and smooth the edges, i.e.:

[0041]

[0042] Where A is the segmented binary image, and B is the structuring element. This represents the expansion operation. This represents the erosion operation.

[0043] In some embodiments, false detection regions (such as reflective patches) are eliminated based on the area, aspect ratio, and circularity of the segmented image. For example, candidate regions with an area that is too small (<50 pixels) or too large (>10,000 pixels) are excluded, and segmented images with an aspect ratio <2.0 (close to square or circular) are retained. Segmented images with a circularity of 0.5 < 1.2 are retained, allowing for a certain degree of irregularity.

[0044] In some embodiments, first environmental information is obtained based on the segmentation results.

[0045] In some embodiments, obtaining second environmental information based on point cloud data requires fitting a road surface plane model using the Random Sample Consensus (RANSAC) algorithm to separate ground points from non-ground points (i.e., points above the ground) in the point cloud data. The road surface plane equation can be expressed as:

[0046] ax + by + cz + d = 0

[0047] Where (a,b,c) is the normal vector and d is the intercept. The parameters a,b,c,d are optimized by the least squares method.

[0048] Then, the Statistical Outlier Removal (SOR) algorithm is used to remove noisy points from the point cloud data. For each point in the cloud data, the mean distance μ and standard deviation σ of the neighborhood are calculated, and points exceeding μ±ασ are removed, where α is the threshold coefficient.

[0049] In some embodiments, the local elevation change rate (i.e., elevation gradient) is calculated for the set of ground points in the cloud data, and the ground point p i (x i ,y i ,z i In the neighborhood N i elevation gradient within for:

[0050]

[0051] Where, p i (x i ,y i ,z i Let x represent the coordinates of the i-th point in the set of ground points, where x... i Represents the x-coordinate of the i-th point, y-coordinate i The z-coordinate represents the y-axis coordinate of the i-th point. i N represents the z-axis coordinate (elevation) of the i-th point; i Let |N| represent the neighborhood of the i-th point. i | represents the number of points in the neighborhood; p j N represents i The j-th point in z; j For N i The z-coordinate (elevation) of the j-th point in the set; i is a positive integer less than n, j is a positive integer less than m, n and m are positive integers, and the set of ground points has a total of n points, N i There are m points in total.

[0052] exist In the case where θ is the i-th point, we determine that the i-th point is a point in the concave-convex region. grad As the gradient threshold, in In this case, If the value is positive, then the i-th point is a point in the convex region. If the value is negative, then the i-th point is a point in the concave region; the types of points in the concave and convex regions include points in the concave region and points in the convex region.

[0053] In some embodiments, the curvature of the ground point cloud is calculated based on principal component analysis (PCA), p j The eigenvalues ​​of the covariance matrix C satisfy λ1≥λ2≥λ3, p i(x i ,y i ,z i In the neighborhood N i The curvature κ within is:

[0054]

[0055] Where λ1=x j -x i λ2=y j -y i , λ3=z j -z i .

[0056] p i (x i ,y i ,z i In the neighborhood N i If the absolute value of the curvature κ within the region is greater than the curvature threshold, then the i-th point is determined to be a point in the concave-convex region, where, in p i (x i ,y i ,z i In the neighborhood N i If the absolute value of the curvature κ within the region is greater than the curvature threshold, then if κ is positive, the i-th point is a point in the convex region; if κ is negative, the i-th point is a point in the concave region.

[0057] In some embodiments, And p i (x i ,y i ,z i In the neighborhood N i If the absolute value of the curvature κ within the region is greater than the curvature threshold, the i-th point is determined to be a point in the concave-convex region.

[0058] In some embodiments, And p i (x i ,y i ,z i In the neighborhood N i When the absolute value of the curvature κ within the curvature is greater than the curvature threshold, If κ is positive, then the i-th point is a point in a convex region. If κ is negative, then the i-th point is a point in the concave region.

[0059] In some embodiments, if the number of points in the uneven area is within a preset range, the second road surface information is determined to be a bumpy road surface.

[0060] In some embodiments, θ gradThe curvature threshold can be freely set according to the accuracy of the lidar and the road surface material; the number of channels in which the lidar emits laser beams in the vertical direction includes 16 lines, 64 lines, etc. The accuracy of the 64-line lidar is higher than that of the 16-line lidar. Road surface materials include asphalt, cement, etc.

[0061] In some embodiments, when point cloud data is sufficient, such as when the point cloud coverage is 95%, a network such as PointNet++ (a method for processing point cloud data using a fully connected neural network) can be used to classify the types of concave and convex regions end-to-end. The types of concave and convex regions include potholes, speed bumps, etc.

[0062] In some embodiments, the cross-entropy loss function is:

[0063]

[0064] Where Λ represents the loss function, used to measure the difference between the model's prediction and the true label; C represents the total number of categories of the uneven region; and c represents the number of categories of the uneven region. For example, C=4 means that the total number of categories of the uneven region includes potholes, speed bumps, smooth roads, and puddles; c=1 represents potholes, c=2 represents speed bumps, c=3 represents smooth roads, and c=4 represents puddles. c For real labels, p c To predict probabilities.

[0065] The cross-entropy loss function is used to measure the difference between the prediction results of a classification model and the true labels. By minimizing the cross-entropy loss function, the classification model can continuously optimize its parameters and improve the accuracy of classification. For example, the parameters of the PointNet++ model can be optimized by using the cross-entropy loss function.

[0066] In some embodiments, by calculating p i (x i ,y i ,z i Let p be the average curvature of the curvature of all target points within a region centered at a preset radius, and let the average curvature be used as p. i (x i ,y i ,z i The curvature of the region can be analyzed by calculating the average curvature through a sliding window, which can suppress noise.

[0067] In some embodiments, a KD-Tree (a data structure for organizing points in k-dimensional space) is used to spatially partition the point cloud, and then outliers are removed.

[0068] In some embodiments, a neural network model (e.g., PointNet++) is used to process the set of non-ground points to obtain a second obstacle recognition result.

[0069] In this embodiment of the application, environmental information of the vehicle's external environment is obtained based on images and point cloud data, and then the vehicle's state is adjusted based on the environmental information and state data so that the vehicle can adapt to the external environment.

[0070] Step 103: Control the chassis of the vehicle based on the environmental information and the status data.

[0071] In this embodiment, the vehicle's state is adjusted based on environmental information and state data to adapt the vehicle to the external environment, thereby enabling the vehicle to adjust its state in the event of changes in the external environment and preventing danger from occurring.

[0072] In summary, in this embodiment, the vehicle's status data is first acquired, and images of the vehicle's external environment are collected through the vehicle's camera module, as well as point cloud data of the vehicle's external environment are collected through the vehicle's LiDAR. Then, based on the images and point cloud data, environmental information about the vehicle's external environment is obtained. Finally, based on the environmental information and status data, the vehicle's chassis is controlled to enable the vehicle to adapt to the external environment, thereby avoiding the problem of the driver being unable to control the vehicle in time according to the external environment when the driver is distracted, which could lead to a dangerous situation.

[0073] Figure 2 This is a flowchart illustrating specific steps of the chassis control method provided in this application embodiment, such as... Figure 2 As shown, the method may include:

[0074] Step 201: Obtain the inertial measurement data of the vehicle through the vehicle's inertial measurement unit.

[0075] In some embodiments, the inertial measurement unit (IMU) acquires inertial measurement data of the vehicle, including acceleration, attitude data, etc. The attitude data includes pitch angle, roll angle, yaw angle, etc., and the attitude data is represented by quaternions.

[0076] In this embodiment of the application, inertial measurement data of the vehicle is obtained through the vehicle's inertial measurement unit, and then sensor data of the vehicle is obtained through the vehicle's sensors. Based on the inertial measurement data and the sensor data, state data is generated.

[0077] Step 202: Obtain sensor data of the vehicle through the vehicle's sensors.

[0078] In some embodiments, the vehicle's sensors include an acceleration sensor, a height sensor, etc., and the sensor data includes acceleration, chassis height, etc., wherein the acceleration sensor is used to acquire the vehicle's acceleration, and the height sensor is used to acquire the vehicle's chassis height.

[0079] In this embodiment of the application, vehicle sensor data is acquired through the vehicle's sensors, and then state data is generated based on inertial measurement data and sensor data.

[0080] Step 203: Generate the state data based on the inertial measurement data and the sensor data.

[0081] In some embodiments, in the event of an accelerometer sensor failure, the acceleration in the inertial measurement data is determined as the vehicle's acceleration; in the event of an inertial measurement unit failure, the acceleration in the sensor data is determined as the vehicle's acceleration.

[0082] In this embodiment of the application, state data is generated based on inertial measurement data and sensor data to obtain the state data of the vehicle.

[0083] Step 204: Collect images of the vehicle's external environment using the vehicle's camera module, and collect point cloud data of the vehicle's external environment using the vehicle's lidar.

[0084] The implementation method of this step is similar to that in step 101 above, and will not be repeated here.

[0085] Step 205: Obtain environmental information about the outside of the vehicle based on the image and the point cloud data.

[0086] The implementation method of this step is similar to that of step 102 above, and will not be repeated here.

[0087] Step 206: Control the chassis of the vehicle based on the environmental information and the status data.

[0088] The implementation method of this step is similar to that in step 103 above, and will not be repeated here.

[0089] Optionally, in some embodiments, step 205 includes the following sub-steps:

[0090] Sub-step 2051: Obtain road surface information outside the vehicle based on the image and the point cloud data.

[0091] In this embodiment of the application, road surface information outside the vehicle is obtained based on images and point cloud data, and then the vehicle chassis is controlled based on the road surface information.

[0092] Optionally, in some embodiments, the type of road surface information includes first road surface information of the vehicle exterior obtained from the image, second road surface information of the vehicle exterior obtained from the point cloud data, and third road surface information fused by deduplicating the first road surface information and the second road surface information.

[0093] In this embodiment of the application, since the types of road surface information include first road surface information of the vehicle exterior obtained from images, second road surface information of the vehicle exterior obtained from point cloud data, and third road surface information fused by deduplicating the first road surface information and the second road surface information, the diversity of road surface information data sources is improved, and the type of road surface information can be selected according to the actual situation.

[0094] Optionally, in some embodiments, sub-step 2051 includes the following steps:

[0095] Step 2051a: Based on the image, obtain the first road surface information outside the vehicle.

[0096] In this embodiment of the application, first road surface information outside the vehicle is obtained based on the image, and then second road surface information outside the vehicle is obtained based on point cloud data. After deduplication of the first road surface information and the second road surface information, they are fused to generate third road surface information outside the vehicle.

[0097] Step 2051b: Obtain the second road surface information outside the vehicle based on the point cloud data.

[0098] In this embodiment of the application, second road surface information outside the vehicle is obtained based on point cloud data, and then the first road surface information and the second road surface information are deduplicated and fused to generate third road surface information outside the vehicle.

[0099] Step 2051c: After deduplication of the first road surface information and the second road surface information, merge them to generate the third road surface information outside the vehicle.

[0100] For example, if the first road surface information includes data A, data B, and data C, and the second road surface information includes data A, data B, and data D, then data A and data B in the second road surface information are removed (i.e., the first and second road surface information are deduplicated). Then, data A, data B, and data C in the first road surface information and data D in the second road surface information are merged to obtain the third road surface information, which includes data A, data B, data C, and data D. In other words, the deduplicated first road surface information is merged with the second road surface information to achieve the deduplication and fusion of the first and second road surface information to generate the third road surface information outside the vehicle.

[0101] In this embodiment, the first road surface information and the second road surface information are deduplicated and then fused to generate a third road surface information outside the vehicle. The vehicle's state is then adjusted based on the third road surface information and the vehicle's state data.

[0102] Optionally, in some embodiments, the state data includes the vehicle's tire torque, suspension system damping force, and vehicle speed, and step 206 includes the following sub-steps:

[0103] Sub-step 2061: When the road surface information is a bumpy road surface, and the tire torque is less than the torque threshold, and the damping force is greater than the damping force threshold, and the vehicle speed is greater than the first vehicle speed threshold, the tire torque is increased to a preset torque, the damping force is decreased to a preset damping force, and the vehicle speed is decreased to a first preset vehicle speed.

[0104] Wherein, the bumpy road surface is a road surface with the number of uneven areas within a preset range; the preset torque is greater than or equal to the torque threshold, the preset damping force is less than or equal to the damping force threshold, and the first preset vehicle speed is less than or equal to the first vehicle speed threshold.

[0105] In some embodiments, a bumpy road surface is a road surface with potholes and / or bumps.

[0106] In some embodiments, the vehicle's suspension system is a mechanical component that connects the vehicle body and the wheels. Its function is to support the vehicle body, mitigate impacts, dampen vibrations, and maintain vehicle stability. When the vehicle is traveling at high speed or cornering, increasing the damping force of the suspension system (i.e., increasing damping) can suppress body roll and pitching, and improve handling stability. When the vehicle is traveling on bumpy roads, decreasing the damping force of the suspension system (i.e., decreasing damping) can better absorb vibrations and keep the vehicle body stable.

[0107] In some embodiments, the vehicle speed is calculated based on acceleration and the initial vehicle speed.

[0108] In this embodiment, when the road surface information is a bumpy road surface, and the tire torque is less than the torque threshold, the damping force is greater than the damping force threshold, and the vehicle speed is greater than the first vehicle speed threshold, it is explained that the current tire torque, the current damping force, and the current vehicle speed result in poor comfort when the vehicle passes through the bumpy road surface. Since the preset torque is greater than or equal to the torque threshold, the preset damping force is less than or equal to the damping force threshold, and the first preset vehicle speed is less than or equal to the first vehicle speed threshold, the tire torque is increased to the preset torque, the damping force is decreased to the preset damping force, and the vehicle speed is decreased to the first preset vehicle speed. That is, the tire torque is increased, the damping force is decreased, and the vehicle speed is decreased, so that the vehicle can pass through the bumpy road surface smoothly, thereby improving the comfort of riding in the vehicle.

[0109] Optionally, in some embodiments, the status data includes the vehicle's chassis height and speed, and step 206 includes the following sub-steps:

[0110] Sub-step 2062: When the road surface information is a waterlogged road surface, the chassis height is less than the height threshold, and the vehicle speed is greater than the second vehicle speed threshold, the chassis height is raised to a preset height, and the vehicle speed is reduced to the second preset vehicle speed.

[0111] Wherein, the waterlogged road surface is a road surface with a waterlogged area within a preset area range; the preset height is greater than or equal to the height threshold, and the second preset vehicle speed is less than or equal to the second vehicle speed threshold.

[0112] In some embodiments, a waterlogged road surface is a road surface where obvious water has accumulated, forming a film of water or a pool of water, such as a pothole.

[0113] In some embodiments, chassis height refers to the height of the chassis above the road surface below it.

[0114] In this embodiment, when the road surface information is a flooded road surface, the chassis height is less than the height threshold, and the vehicle speed is greater than the second vehicle speed threshold, it is explained that the current chassis height and the current vehicle speed pose risks such as chassis immersion and slippage on the flooded road surface. Since the preset height is greater than or equal to the height threshold and the second preset vehicle speed is less than or equal to the second vehicle speed threshold, by raising the chassis height to the preset height and reducing the vehicle speed to the second preset vehicle speed, that is, by raising the chassis and reducing the vehicle speed, the vehicle can safely pass through the flooded road surface.

[0115] Optionally, in some embodiments, the state data includes the vehicle speed, and step 206 includes the following sub-steps:

[0116] Sub-step 2063: If the road surface information is an icy road surface and the vehicle speed is greater than the third vehicle speed threshold, reduce the vehicle speed to the third preset vehicle speed.

[0117] Wherein, the icy road surface is a road surface with an icy area greater than an area threshold; the third preset vehicle speed is less than or equal to the third vehicle speed threshold.

[0118] In this embodiment of the application, when the road surface information is an icy road surface and the vehicle speed is greater than the third vehicle speed threshold, it is explained that the current vehicle speed poses a risk of skidding on the icy road surface. Since the third preset vehicle speed is less than or equal to the third vehicle speed threshold, the vehicle can safely pass through the icy road surface by reducing the vehicle speed to the third preset vehicle speed.

[0119] Optionally, in some embodiments, the state data includes the vehicle speed, and step 205 includes the following sub-steps:

[0120] Sub-step 2052: Obtain obstacle recognition results outside the vehicle based on the image and the point cloud data.

[0121] In this embodiment of the application, obstacle recognition results of the vehicle's exterior are obtained based on images and point cloud data, and then the vehicle's chassis is controlled based on the obstacle recognition results of the vehicle's exterior.

[0122] Optionally, in some embodiments, the type of obstacle recognition result includes a first obstacle recognition result of the vehicle exterior obtained from the image, a second obstacle recognition result of the vehicle exterior obtained from the point cloud data, and a third obstacle recognition result obtained by fusing the first obstacle recognition result and the second obstacle recognition result after deduplication.

[0123] In this embodiment of the application, since the types of obstacle recognition results include the first obstacle recognition result obtained from the image of the vehicle exterior, the second obstacle recognition result obtained from the point cloud data of the vehicle exterior, and the third obstacle recognition result obtained by deduplicating the first obstacle recognition result and the second obstacle recognition result, the diversity of the data source of the obstacle recognition result is improved, and the type of obstacle recognition result can be selected according to the actual situation.

[0124] Optionally, in some embodiments, sub-step 2052 includes the following steps:

[0125] Step 2052a: Based on the image, obtain the first obstacle recognition result outside the vehicle.

[0126] In this embodiment of the application, a first obstacle recognition result of the vehicle exterior is obtained based on the image, and then a second obstacle recognition result of the vehicle exterior is obtained based on the point cloud data. The first obstacle recognition result and the second obstacle recognition result are deduplicated and then fused to generate a third obstacle recognition result of the vehicle exterior.

[0127] Step 2052b: Based on the point cloud data, obtain the second obstacle recognition result outside the vehicle.

[0128] In this embodiment of the application, a second obstacle recognition result is obtained based on point cloud data, and then the first obstacle recognition result and the second obstacle recognition result are deduplicated and fused to generate a third obstacle recognition result for the exterior of the vehicle.

[0129] Step 2052c: After deduplication of the first obstacle recognition result and the second obstacle recognition result, they are merged to generate a third obstacle recognition result for the exterior of the vehicle.

[0130] For example, if the first obstacle recognition result includes obstacles E, F, and G in front of the vehicle, and the second road obstacle recognition result includes obstacles E, F, and H in front of the vehicle, then obstacles E and F in front of the vehicle in the second obstacle recognition result are removed (i.e., the first obstacle recognition result and the second obstacle recognition result are deduplicated). Then, obstacles E, F, and G in front of the vehicle in the first obstacle recognition result and obstacle H in front of the vehicle in the second obstacle recognition result are merged to obtain the third obstacle recognition result. The third obstacle recognition result includes obstacles E, F, G, and H in front of the vehicle. In other words, the deduplicated first obstacle recognition result is merged with the second obstacle recognition result to achieve the deduplicated fusion of the first obstacle recognition result and the second obstacle recognition result, generating the third obstacle recognition result outside the vehicle.

[0131] In this embodiment of the application, the first obstacle recognition result and the second obstacle recognition result are deduplicated and then fused to generate a third obstacle recognition result outside the vehicle. Then, if the obstacle recognition result indicates that there is an obstacle in front of the vehicle and the vehicle speed is greater than the fourth vehicle speed threshold, the vehicle speed is reduced to the fourth preset vehicle speed.

[0132] 206 includes the following sub-steps:

[0133] Sub-step 2064: If the obstacle recognition result indicates that there is an obstacle in front of the vehicle and the vehicle speed is greater than the fourth vehicle speed threshold, reduce the vehicle speed to the fourth preset vehicle speed; the fourth preset vehicle speed is less than or equal to the fourth vehicle speed threshold.

[0134] In some embodiments, the types of obstacles include vehicles, pedestrians, buildings, road debris, road fences, etc.

[0135] For example, when a vehicle is driving on an urban road, if the vehicle in front suddenly brakes, the obstacle recognition result of the vehicle will indicate that there is an obstacle in front of the vehicle and the vehicle speed is greater than the fourth speed threshold. By reducing the vehicle speed to the fourth preset speed, the risk of collision between the vehicle and the vehicle in front can be reduced.

[0136] In this embodiment of the application, if the obstacle recognition result indicates that there is an obstacle in front of the vehicle and the vehicle speed is greater than the fourth vehicle speed threshold, it is explained that the current vehicle speed poses a risk of collision with the obstacle. Since the fourth preset vehicle speed is less than or equal to the fourth vehicle speed threshold, the risk of collision with the obstacle is reduced by decreasing the vehicle speed to the fourth preset vehicle speed.

[0137] In some embodiments, the first obstacle recognition result and the second obstacle recognition result are redundant data. For example, in the event of a lidar malfunction, the first obstacle recognition result is determined as environmental information outside the vehicle.

[0138] In some embodiments, the first obstacle recognition result includes the first location information of the obstacle and the distance information between the obstacle and the vehicle. The first location information of the obstacle is obtained by processing the image through a target detection model (e.g., YOLOv5); the distance information between the obstacle and the vehicle is obtained by processing the image through a monocular depth estimation model (e.g., MiDaS model) or a stereo vision model.

[0139] In some embodiments, the two-dimensional bounding box of the image is projected onto the vehicle coordinate system, and the vehicle displacement is calculated by combining it with the wheel speed to generate a three-dimensional bounding box estimate. This involves acquiring the vehicle's attitude information, including pitch, yaw, and roll angles, through the vehicle's IMU. Simultaneously, using the calibration extrinsic parameters (including rotation matrix and translation vector) between the camera and the vehicle coordinate system, the two-dimensional bounding box detected by the camera is transformed from the image coordinate system to the vehicle coordinate system. Using the vehicle speed information provided by the wheel speed sensor and the frame interval (Δt), the vehicle's displacement between two adjacent frames is calculated. By combining the vehicle's displacement information with the projection result of the two-dimensional bounding box, the position of the obstacle in the vehicle coordinate system can be estimated more accurately, thereby generating a three-dimensional bounding box estimate, which determines the spatial position, size, and orientation of the obstacle in three-dimensional space.

[0140] In some embodiments, the equation of motion for the vehicle is:

[0141]

[0142] Where v is the vehicle speed provided by the wheel speed sensor, Δt is the frame interval time, i.e., the time difference between two adjacent image frames acquired by the camera module; α x That is, the vehicle's acceleration, and Δx is the vehicle's displacement.

[0143] When both the vehicle and the obstacle are in motion, by performing motion correction on the obstacle's position, the actual position of the obstacle in the vehicle's coordinate system can be calculated more accurately, thereby improving the accuracy of obstacle detection and tracking. This helps the vehicle better understand the dynamics of obstacles in the surrounding environment, thus making more reasonable decisions.

[0144] In some embodiments, step 205 further includes the following sub-steps:

[0145] Sub-step 2057: Based on the image, obtain traffic light information outside the vehicle.

[0146] In this embodiment of the application, traffic light information outside the vehicle is obtained from the image, and then the vehicle's state is adjusted based on the traffic light information and the vehicle's state information.

[0147] Step 206 also includes the following steps:

[0148] Sub-step 2065: When the traffic light information is red or yellow and the vehicle speed is greater than the fifth speed threshold, reduce the vehicle speed to the fifth preset speed; the fifth preset speed is less than or equal to the fifth speed threshold.

[0149] In this embodiment of the application, when the traffic light information is red or yellow and the vehicle speed is greater than the fifth speed threshold, it indicates that the current vehicle speed poses a risk of running a red light. Since the fifth preset speed is less than or equal to the fifth speed threshold, the vehicle speed is reduced to the fifth preset speed to avoid running a red light.

[0150] Sub-step 2066: When the traffic light is green, the obstacle recognition result is that there is no obstacle in front of the vehicle, and the vehicle speed is less than the sixth speed threshold, increase the vehicle speed to the sixth preset speed; the sixth preset speed is greater than or equal to the sixth speed threshold.

[0151] In this embodiment of the application, when the traffic light is green, the obstacle recognition result is that there is no obstacle in front of the vehicle, and the vehicle speed is less than the sixth speed threshold, it indicates that the current vehicle speed is delaying the vehicle from moving forward at the intersection. Since the sixth preset speed is greater than or equal to the sixth speed threshold, the vehicle speed is increased to the sixth preset speed to reduce the waiting time of the vehicle at the intersection.

[0152] In some embodiments, the first obstacle identification result further includes the first location information of the obstacle, and the second obstacle identification result further includes the second location information of the obstacle. The first location information and the second location information are compared. If the distance deviation between the first location information and the second location information is continuously greater than the deviation threshold (e.g., 1 meter), the lidar is confirmed to be faulty, and the first location information is determined as the actual location information of the obstacle.

[0153] In summary, in this embodiment, the vehicle's status data is first acquired, and images of the vehicle's external environment are collected through the vehicle's camera module, as well as point cloud data of the vehicle's external environment are collected through the vehicle's LiDAR. Then, based on the images and point cloud data, environmental information about the vehicle's external environment is obtained. Based on the environmental information and status data, the vehicle's status is adjusted so that the vehicle adapts to the external environment, thereby avoiding the problem of the driver being unable to control the vehicle in time according to the external environment when the driver is distracted, which could lead to a dangerous situation.

[0154] Reference Figure 3This application embodiment also provides a chassis control system, including: a forward-view integrated unit 10 and a chassis control module 20; the forward-view integrated unit 10 and the chassis control module 20 are connected via a network line, the forward-view integrated unit 10 is used to collect images and point cloud data of the external environment of the vehicle, and to acquire inertial measurement data of the vehicle; the chassis control module 20 is used to acquire sensor data of the vehicle through the vehicle's sensors, and to generate state data of the vehicle based on the inertial measurement data and the sensor data, and to acquire environmental information of the external environment of the vehicle based on the images and the point cloud data, and to control the chassis of the vehicle based on the environmental information and the state data.

[0155] The specific implementation process of the chassis control system is similar to that described above, and will not be repeated here.

[0156] In some embodiments, the chassis control system further includes an acceleration sensor 30, a height sensor 40, a first high-side driver (HSD) switch 55, a first low-side driver (LSD) switch 56, a second high-side driver switch 66, a second low-side driver switch 67, a control driver chip 68, a damping control module 50, and an air spring module 60; the damping control module 50 includes a left front wheel damping control unit 51, a right front wheel damping control unit 52, a left rear wheel damping control unit 53, and a right rear wheel damping control unit 54; the air spring module 60 includes a temperature and pressure sensor 61, an air valve 62, an exhaust valve 63, a switching valve 64, and a motor 65.

[0157] The chassis control module 20 is connected to the acceleration sensor 30, the height sensor 40, the first high-side drive switch 55, the first low-side drive switch 56, the second high-side drive switch 66, the second low-side drive switch 67, and the control drive chip 68, respectively. The damping control module 50 is connected to the first high-side drive switch 55 and the first low-side drive switch 56, respectively. The air spring module 60 is connected to the second high-side drive switch 66, the second low-side drive switch 67, and the control drive chip 68, respectively. The control drive chip 68 is connected to the motor 65.

[0158] The chassis control module 20 controls the opening and closing of the damping control module 50 through the first high-side drive switch 55 and the first low-side drive switch 56. The chassis control module 20 controls the opening and closing of the air spring module 60 through the second high-side drive switch 66 and the second low-side drive switch 67. The chassis control module 20 controls the operation of the motor 65 through the control drive chip 68.

[0159] The left front wheel damping control unit 51 is used to adjust the damping force of the shock absorber of the left front wheel; the right front wheel damping control unit 52 is used to adjust the damping force of the shock absorber of the right front wheel; the left rear wheel damping control unit 53 is used to adjust the damping force of the shock absorber of the left rear wheel; and the right rear wheel damping control unit 54 is used to adjust the damping force of the shock absorber of the right rear wheel.

[0160] The temperature and pressure sensor 61 monitors the pressure and temperature information of the air spring in real time and transmits this data to the control unit in the air spring module 60. Based on this data and information from other sensors such as the vehicle height sensor, the control unit determines whether the vehicle height needs adjustment and whether inflation or deflation is required. When inflation is needed, the control unit instructs the motor to start, driving the air compressor to compress air and store it in the air tank. During inflation, the switching valve 64 switches to the corresponding inflation mode according to the control unit's instructions, opening the air passage between the air tank and the air valve 62 while simultaneously closing the deflation valve 63. The air valve 62 opens the corresponding channel according to the control unit's instructions, filling the air spring with compressed air from the air tank, thus raising the vehicle height. When the vehicle height reaches the set value, the control unit instructs the air valve 62 to close, stopping inflation. During deflation, the switching valve 64 switches to the deflation mode, closing the air passage between the air tank and the air valve 62 while simultaneously opening the deflation valve 63. The air valve 62 opens the corresponding deflation channel according to the control unit's instructions, expelling the compressed air from the air spring, thus lowering the vehicle height.

[0161] Throughout the process, the temperature and pressure sensor 61 continuously monitors the pressure and temperature changes of the air spring and feeds this information back to the control unit. Based on the feedback information, the control unit continuously adjusts the operating states of the motor 65, air valve 62, exhaust valve 63, and switching valve 64 to ensure the stable operation of the air spring system and keep the vehicle height within the set range.

[0162] In some embodiments, the damping control module is a continuous damping control (CDC) module, the first controller 11 is a multi-functional camera controller, and the chassis control module 20 is an intelligent chassis controller (ICC).

[0163] In some embodiments, the front-view all-in-one unit 10 is connected to the chassis control module 20 via an Ethernet network line.

[0164] In some embodiments, the chassis control module 20 can be a controller, and the type of controller includes a microcontroller unit (MCU), a single-chip microcomputer, or other types of chips.

[0165] In some embodiments, the first controller 11 may be a microcontroller, a single-chip microcomputer, or other types of chips.

[0166] In some embodiments, the forward-looking integrated unit 10 includes a first controller 11, a camera module 12, a lidar 13, and an inertial measurement unit 14. The first controller 11 is connected to the camera module 12, the lidar 13, and the inertial measurement unit 14. The camera module 12 is used to acquire images of the vehicle's external environment, the lidar 13 is used to acquire point cloud data of the vehicle's external environment, and the inertial measurement unit 14 is used to acquire inertial measurement data of the vehicle. The first controller 11 is used to transmit the images of the vehicle's external environment, the point cloud data of the vehicle's external environment, and the inertial measurement data of the vehicle to the chassis control module 20 via a network.

[0167] In some embodiments, the camera module 12 includes a camera, an image sensor, etc.

[0168] In some embodiments, the front-view integrated camera 10 uses the PTP (IEEE 802.1AS) protocol to synchronize the clocks of the camera module 12, the lidar 13, and the inertial measurement unit 14.

[0169] In some embodiments, the first controller 11 sends global trigger pulse signals to the camera module 12, the lidar 13, and the inertial measurement unit 14, respectively, so that the camera module 12, the lidar 13, and the inertial measurement unit 14 synchronously acquire data. The frequency of the global trigger pulse signal is the lowest frequency among the frequencies of the camera module 12, the lidar 13, and the inertial measurement unit 14. For example, if the lidar frequency is the lowest among the frequencies of the camera module 12, the lidar 13, and the inertial measurement unit 14, then the frequency of the global trigger pulse signal is set to the lidar frequency (e.g., the lidar frequency is 100 milliseconds). The alignment error is less than the error threshold, for example, the error threshold is 100 microseconds.

[0170] In some embodiments, when the clocks of the camera module 12, LiDAR 13, and inertial measurement unit 14 are out of sync, asynchronous data is aligned using an interpolation algorithm, for example:

[0171] If the attitude data collected by the inertial measurement unit 14 is not synchronized with the image collected by the camera module 12 and the point cloud data collected by the lidar 13 in time, then the adjacent timestamp t is found in the IMU data queue. k and t k+1 , satisfying t k ≤t target <t k+1 ;t target For the target time, t k For time k, t k+1At time k+1, the difference weight α is:

[0172]

[0173] The target attitude data (i.e., in quaternion form) is calculated by performing spherical linear interpolation (SLERP): q(t) target )=(1-α)×q(t k )+α×q(t k+1 ).

[0174] Where q(t) target ) represents the attitude data at the target time, q(t) k Let q(t) be the attitude data at time k. k+1 ) represents the attitude data at time k+1.

[0175] In some embodiments, the code that calculates the attitude data of the target time by performing spherical linear interpolation (SLERP) includes the following fields: "import numpy as np", "def quaternion_slerp(q1,q2,alpha):", "SLERP:param q1:[w,x,y,z]:param q2:[w,x,y,z]:param alpha:[0,1]:return:", "q1=q1 / np.linalg.norm(q1)", "q2=q2 / np.linalg.norm(q2)", "dot=np.dot(q1,q2)", "if dot<0.0:", "q2=-q2", "dot=-dot", "dot=np.clip(dot,-1.0,1.0)", "theta=np.arccos(dot)", "sin_theta=np.sin(theta)", "if sin_theta<1e-6:", "return(1.0-alpha)*q1+alpha*q2", "else:", "k0=np.sin((1-alpha)*theta) / sin_theta", "k1=np.sin(alpha*theta) / sin_theta", "return k0*q1+k1*q2";

[0176] Here, "import numpy as np" imports the NumPy library and aliases it as np. NumPy is a powerful Python scientific computing library that provides support for array operations, etc. "def quaternion_slerp(q1,q2,alpha):" defines a function named quaternion_slerp with three parameters: q1 and q2 are quaternions represented as [w,x,y,z], and alpha is an interpolation parameter in the range [0,1]. "SLERP:param q1:[w,x,y,z]:param q2:[w,x,y,z]:param alpha:[0,1]:return:" is the function's docstring, which describes the function's functionality, parameters, and return value. This function implements spherical linear interpolation (SLERP) for quaternions, explaining the meanings of parameters q1, q2, and alpha, as well as the return value. `q1 = q1 / np.linalg.norm(q1)`: Normalizes the quaternion q1. `q1 = q1 / np.linalg.norm(q1)` calculates the modulus (Euclidean norm) of q1, then divides each element of q1 by this modulus, making the normalized quaternion modulus 1. `q2 = q2 / np.linalg.norm(q2)`: Normalizes the quaternion q2 in the same way, using the same principle as the operation on q1.

[0177] "dot = np.dot(q1, q2)" calculates the dot product of normalized q1 and q2. The dot product reflects the angle between the two quaternions. "if dot < 0.0:" checks if the dot product dot is less than 0. If it is less than 0, it means the angle between the two quaternions on the sphere is greater than 90 degrees. "q2 = -q2" inverts q2 if the dot product is less than 0. This is because quaternions q and -q represent the same rotation, and inverting them makes the angle between the two quaternions the original supplementary angle, ensuring a shorter arc path during subsequent interpolation. "dot = -dot" similarly inverts the dot product when it is less than 0, so that the angle is between 0 and 180 degrees when calculating the angle. "dot = np.clip(dot, -1.0, 1.0)" clips the dot product to the range [-1.0, 1.0]. This is because the input value must be within this range when calculating the arccosine to avoid numerical calculation errors; "theta = np.arccos(dot)" calculates the angle theta between two quaternions, obtained through the arccosine function, with the unit being radians;

[0178] "sin_theta=np.sin(theta)" calculates the sine of the angle theta; "if sin_theta<1e-6:" checks if the sine is less than 1e-6. If sin_theta is almost 0, it means the angle between the two quaternions is very small, almost overlapping; "return(1.0-alpha)*q1+alpha*q2" means that when the angle between the two quaternions is small, linear interpolation is used directly to obtain the resulting quaternion, thus avoiding numerical instability caused by dividing by a near-zero sin_theta; "else:" means that if the sine value is not less than 1e-6, that is, the angle between the two quaternions is large, then the conventional calculation of spherical linear interpolation is performed; "k0=np.sin((1-alpha)*theta) / sin_theta" calculates the interpolation coefficient k0, which corresponds to the weight of quaternion q1 in the interpolation result; "k1=np.sin(alpha*theta) / sin_theta" calculates the interpolation coefficient k1, which corresponds to the weight of quaternion q2 in the interpolation result; "return "k0*q1+k1*q2" means that the interpolation coefficients k0 and k1 are multiplied by the corresponding quaternions q1 and q2 respectively, and then added together to obtain the interpolated quaternion result.

[0179] In some embodiments, the power supply, temperature, and laser emitter status of the lidar 13 are confirmed by a heartbeat signal (such as a CAN message signal with a period of 1Hz). If the data stream interruption of the lidar 13 exceeds a timeout threshold (e.g., 200 milliseconds), a primary fault is identified and displayed, for example, by displaying a primary fault identifier.

[0180] In some embodiments, the chassis control system further includes a braking system, a drive system, etc., and the chassis control module is also connected to the braking module and the drive module respectively. The chassis control module is used to control the braking module to decelerate the vehicle and to control the drive module to accelerate the vehicle.

[0181] For example, when there is a vehicle in front of the vehicle or a vehicle changes lanes and cuts in, the forward-looking integrated camera quickly transmits images and point cloud data of the vehicle's external environment, as well as the vehicle's inertial measurement data, to the chassis control module. If the obstacle recognition result indicates that there is an obstacle in front of the vehicle and the vehicle speed is greater than the fourth vehicle speed threshold, the chassis control module controls the vehicle's braking system to decelerate, maintain a safe distance from the vehicle in front, and avoid a rear-end collision.

[0182] Compared to systems where the front-view integrated unit and chassis system operate independently, or where they interact via a Controller Area Network (CAN) bus, this application's embodiments utilize a large amount of road condition data collected by the front-view integrated unit, combined with the vehicle's current driving status, to proactively adjust various chassis systems, further enhancing the vehicle's driving safety and stability, and enabling the vehicle to react quickly to complex road conditions.

[0183] This embodiment of the application constructs a high-speed data transmission line, employing an Ethernet bus to achieve fast and stable data transmission between the forward-view integrated unit 10 and the chassis control module 20. The forward-view integrated unit 10 can collect real-time road condition information ahead of the vehicle, including obstacle size, road conditions, road signs, and weather conditions. This data is transmitted to the chassis central control system via the high-speed transmission line with millisecond-level latency, ensuring the timeliness and accuracy of the information, improving the chassis's response speed to road conditions, and allowing for timely adjustments to the vehicle's status. Under different road conditions, each subsystem of the chassis can accurately adjust the vehicle's operating status based on the data from the forward-view integrated unit 10.

[0184] In this embodiment, the chassis control module 20 is equipped with advanced intelligent algorithms that can adjust the vehicle's state based on real-time road condition data transmitted from the forward-looking integrated camera, combined with data from the acceleration and height sensors. For example, when water or ice is detected on the road ahead, the system automatically reduces the torque output of the tires and adjusts the stiffness and damping of the suspension system to improve the vehicle's grip and driving stability. When driving at high speed and on a good road ahead, the system automatically optimizes the engine's fuel injection strategy and the transmission's shifting logic to achieve energy saving, emission reduction, and efficient driving.

[0185] In this embodiment, the chassis control system adopts a modular design, and the interfaces between the various components are standardized and universal. This enables the forward-looking integrated machine 10 and related chassis components to be quickly and conveniently replaced or upgraded during vehicle production, maintenance and upgrade processes, reducing maintenance costs and improving production efficiency. At the same time, it also reserves sufficient expansion space for the integration of new technologies in the future.

[0186] In this embodiment, through the collaborative work of the forward-looking integrated unit and the chassis control module, the vehicle can predict road conditions in advance and reduce unnecessary acceleration, deceleration and braking operations. This not only reduces fuel consumption but also reduces exhaust emissions, which is in line with the current development trend of energy conservation and environmental protection, provides strong technical support for green travel, and brings a better driving experience.

[0187] In this embodiment, multi-sensor, multi-source data acquisition and fusion are employed. A combination of LiDAR, camera modules, and accelerometers is used to achieve centimeter-level precision positioning and road condition perception, and real-time road condition feature extraction. Multi-sensor data is fused through timestamp alignment and Kalman filtering to eliminate noise and errors from individual sensors and improve data reliability.

[0188] In this embodiment, the vehicle's external environment is captured by a forward-looking integrated camera, along with image and point cloud data, and inertial measurement data. The chassis control module then acquires sensor data from the vehicle's sensors. Based on the inertial measurement data and sensor data, the vehicle's state data is generated. Additionally, the vehicle's external environment information is obtained based on the image and point cloud data. The chassis is then controlled based on the environmental information and state data to adapt the vehicle to its external environment. This prevents the driver from being unable to promptly adjust the vehicle's controls when distracted, thus avoiding potential hazards.

[0189] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described chassis control method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0190] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0191] Figure 4 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0192] The electronic device 400 includes, but is not limited to, components such as: radio frequency unit 401, network module 402, audio output unit 403, input unit 404, sensor 405, display unit 406, user input unit 407, interface unit 408, memory 409, and processor 410.

[0193] Those skilled in the art will understand that the electronic device 400 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 410 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0194] The processor 410 is used to acquire vehicle status data, collect images of the vehicle's external environment through the vehicle's camera module, and collect point cloud data of the vehicle's external environment through the vehicle's lidar.

[0195] Based on the image and the point cloud data, obtain the environmental information outside the vehicle;

[0196] The chassis of the vehicle is controlled based on the environmental information and the status data.

[0197] In this embodiment, the vehicle's status data is first acquired, and images of the vehicle's external environment are collected through the vehicle's camera module, as well as point cloud data of the vehicle's external environment are collected through the vehicle's LiDAR. Then, based on the images and point cloud data, environmental information about the vehicle's external environment is obtained. Finally, based on the environmental information and status data, the vehicle's chassis is controlled to enable the vehicle to adapt to the external environment, thereby avoiding the problem that the driver cannot control the vehicle in time according to the external environment when the driver is distracted, which could lead to a dangerous situation.

[0198] Optionally, the processor 410 is also configured to acquire road surface information outside the vehicle based on the image and the point cloud data.

[0199] Optionally, the status data includes the vehicle's tire torque, suspension system damping force, and vehicle speed. The processor 410 is further configured to, when the road surface information is a bumpy road surface, and the tire torque is less than a torque threshold, the damping force is greater than a damping force threshold, and the vehicle speed is greater than a first vehicle speed threshold, increase the tire torque to a preset torque, decrease the damping force to a preset damping force, and decrease the vehicle speed to a first preset vehicle speed; wherein, the bumpy road surface is a road surface with the number of uneven areas within a preset range; the preset torque is greater than or equal to the torque threshold, the preset damping force is less than or equal to the damping force threshold, and the first preset vehicle speed is less than or equal to the first vehicle speed threshold.

[0200] Optionally, the status data includes the vehicle's chassis height and speed. The processor 410 is further configured to, when the road surface information is a flooded road surface, and the chassis height is less than a height threshold, and the vehicle speed is greater than a second speed threshold, raise the chassis height to a preset height and reduce the vehicle speed to a second preset speed; wherein the flooded road surface is a road surface with a flooded area within a preset area range; the preset height is greater than or equal to the height threshold, and the second preset speed is less than or equal to the second speed threshold.

[0201] Optionally, the status data includes the vehicle speed. The processor 410 is further configured to reduce the vehicle speed to a third preset speed when the road surface information is an icy road surface and the vehicle speed is greater than a third speed threshold; wherein the icy road surface is a road surface with an icy area greater than an area threshold; and the third preset speed is less than or equal to the third speed threshold.

[0202] Optionally, the types of road surface information include first road surface information of the vehicle exterior obtained from the image, second road surface information of the vehicle exterior obtained from the point cloud data, and third road surface information fused by deduplicating the first road surface information and the second road surface information.

[0203] Optionally, the state data includes the vehicle speed. The processor 410 is further configured to obtain obstacle recognition results outside the vehicle based on the image and the point cloud data; if the obstacle recognition result indicates that there is an obstacle in front of the vehicle and the vehicle speed is greater than a fourth vehicle speed threshold, the vehicle speed is reduced to a fourth preset vehicle speed; the fourth preset vehicle speed is less than or equal to the fourth vehicle speed threshold.

[0204] In this embodiment, the vehicle's status data is first acquired, and images of the vehicle's external environment are collected through the vehicle's camera module, as well as point cloud data of the vehicle's external environment are collected through the vehicle's LiDAR. Then, based on the images and point cloud data, environmental information about the vehicle's external environment is obtained. Finally, based on the environmental information and status data, the vehicle's chassis is controlled to enable the vehicle to adapt to the external environment, thereby avoiding the problem that the driver cannot control the vehicle in time according to the external environment when the driver is distracted, which could lead to a dangerous situation.

[0205] It should be understood that, in this embodiment, the input unit 404 may include a graphics processing unit (GPU) 4041 and a microphone 4042. The GPU 4041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 406 may include a display panel 4061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 407 includes at least one of a touch panel 4071 and other input devices 4072. The touch panel 4071 is also called a touch screen. The touch panel 4071 may include a touch detection device and a touch controller. Other input devices 4072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0206] The memory 409 can be used to store software programs and various data. The memory 409 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 409 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 409 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0207] Processor 410 may include one or more processing units; optionally, processor 410 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 410.

[0208] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described chassis control method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0209] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0210] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above chassis control method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0211] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0212] In this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0213] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0214] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A chassis control method, characterized in that, The method includes: The system acquires vehicle status data by using the vehicle's camera module to collect images of the vehicle's external environment and by using the vehicle's lidar to collect point cloud data of the vehicle's external environment. Based on the image and the point cloud data, obtain the environmental information outside the vehicle; The chassis of the vehicle is controlled based on the environmental information and the status data. The step of obtaining environmental information outside the vehicle based on the image and the point cloud data includes: Based on the image and the point cloud data, obtain the road surface information outside the vehicle; The status data includes the vehicle's tire torque, suspension system damping force, and vehicle speed. Controlling the vehicle's chassis based on the environmental information and the status data includes: If the road surface information is a bumpy road surface, and the tire torque is less than the torque threshold, the damping force is greater than the damping force threshold, and the vehicle speed is greater than the first vehicle speed threshold, then the tire torque is increased to a preset torque, the damping force is decreased to a preset damping force, and the vehicle speed is decreased to a first preset vehicle speed. Wherein, the bumpy road surface is a road surface with the number of uneven areas within a preset range; the preset torque is greater than or equal to the torque threshold, the preset damping force is less than or equal to the damping force threshold, and the first preset vehicle speed is less than or equal to the first vehicle speed threshold.

2. The method according to claim 1, characterized in that, The status data includes the vehicle's chassis height and speed. Controlling the vehicle's chassis based on the environmental information and the status data includes: If the road surface information is a waterlogged road surface, the chassis height is less than a height threshold, and the vehicle speed is greater than a second vehicle speed threshold, the chassis height will be raised to a preset height, and the vehicle speed will be reduced to a second preset vehicle speed. Wherein, the waterlogged road surface is a road surface with a waterlogged area within a preset area range; the preset height is greater than or equal to the height threshold, and the second preset vehicle speed is less than or equal to the second vehicle speed threshold.

3. The method according to claim 1, characterized in that, The status data includes the vehicle speed, and controlling the vehicle chassis based on the environmental information and the status data includes: If the road surface information indicates an icy road surface and the vehicle speed is greater than a third vehicle speed threshold, the vehicle speed will be reduced to a third preset vehicle speed. Wherein, the icy road surface is a road surface with an icy area greater than an area threshold; the third preset vehicle speed is less than or equal to the third vehicle speed threshold.

4. The method according to claim 1, characterized in that, The types of road surface information include first road surface information of the vehicle exterior obtained from the image, second road surface information of the vehicle exterior obtained from the point cloud data, and third road surface information fused by deduplicating the first road surface information and the second road surface information.

5. The method according to claim 1, characterized in that, The status data includes the vehicle speed, and the step of obtaining the external environmental information of the vehicle based on the image and the point cloud data includes: Based on the image and the point cloud data, obtain the obstacle recognition result outside the vehicle; The step of controlling the vehicle chassis based on the environmental information and the status data includes: If the obstacle recognition result indicates that there is an obstacle in front of the vehicle and the vehicle speed is greater than the fourth vehicle speed threshold, the vehicle speed is reduced to the fourth preset vehicle speed; the fourth preset vehicle speed is less than or equal to the fourth vehicle speed threshold.

6. A chassis control system, characterized in that, include: Forward-facing integrated camera and chassis control module; The forward-looking integrated unit is connected to the chassis control module via a network line. The forward-looking integrated unit is used to collect images and point cloud data of the vehicle's external environment, as well as to acquire the vehicle's inertial measurement data. The chassis control module is used to acquire sensor data of the vehicle through the vehicle's sensors, generate state data of the vehicle based on the inertial measurement data and the sensor data, acquire environmental information of the vehicle's external environment based on the image and the point cloud data, and control the chassis of the vehicle based on the environmental information and the state data. The chassis control module is also used for: The step of obtaining environmental information outside the vehicle based on the image and the point cloud data includes: Based on the image and the point cloud data, obtain the road surface information outside the vehicle; The status data includes the vehicle's tire torque, suspension system damping force, and vehicle speed. Controlling the vehicle's chassis based on the environmental information and the status data includes: If the road surface information is a bumpy road surface, and the tire torque is less than the torque threshold, the damping force is greater than the damping force threshold, and the vehicle speed is greater than the first vehicle speed threshold, then the tire torque is increased to a preset torque, the damping force is decreased to a preset damping force, and the vehicle speed is decreased to a first preset vehicle speed. Wherein, the bumpy road surface is a road surface with the number of uneven areas within a preset range; the preset torque is greater than or equal to the torque threshold, the preset damping force is less than or equal to the damping force threshold, and the first preset vehicle speed is less than or equal to the first vehicle speed threshold.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the chassis control method as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the chassis control method as described in any one of claims 1 to 5.

Images