A chassis intelligent early warning system and method based on binocular vision

Through an intelligent early warning system based on binocular vision, combined with multi-scale transformation model and multi-scale attention head, the chassis passivity and fire risk are evaluated, and the insufficient detection and fire hazards of the chassis early warning system in the existing technology in complex environments is solved. , achieving efficient and accurate early warning and security guarantee.

CN118736531BActive Publication Date: 2024-11-19YUQIWEI (CHONGQING) INTELLIGENT TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411217170.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-19
Estimated Expiration
2044-09-02

AI Technical Summary

Technical Problem

The existing chassis early warning system has insufficient detection capabilities under complex road conditions, making it difficult to provide timely and effective early warnings, especially when driving at high speed or in narrow spaces, it is easy to cause collisions or scratches, and ignores the hidden dangers of chassis fires, leading to safety risks.

Method used

Using an intelligent early warning system based on binocular vision, images are collected through binocular cameras, depth maps are generated and pre-trained chassis early warning model is input for risk assessment, and chassis passivity and fire are carried out in combination with multi-scale transformation models and multi-scale attention heads. Assessment of risks and trigger visual and auditory warning prompts through early warning control modules.

Benefits of technology

Significantly improve the real-time monitoring and early warning capabilities of the chassis system, provide accurate risk warning, enhance the adaptability and reliability of the system, reduce damage to automobile parts caused by road conditions, avoid congestion of later vehicles, and ensure roads smooth.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118736531B_ABST
    Figure CN118736531B_ABST
Patent Text Reader

Abstract

The present invention discloses a chassis intelligent early warning system and method based on binocular vision, and relates to the field of automobile assisted driving technology. The system includes an image acquisition module, which is used to collect original images in front and on the side of the vehicle's driving direction through a binocular camera; an image processing module, which is used to generate a depth map based on the original image; an intelligent analysis module, which is used to input the depth map into a pre-trained chassis early warning model to perform chassis passability analysis and generate a risk assessment result of chassis passability; an early warning control module, which is used to compare the risk assessment result with a preset threshold value to determine whether to trigger an early warning signal; an interactive module, including a touch display screen and an early warning broadcast unit. The present invention performs non-contact detection through a binocular camera and does not rely on a single sensor; by constructing a chassis early warning model, combining a multi-scale transformation model with a multi-scale attention head, the real-time monitoring and early warning capabilities of the chassis system are significantly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of automobile assisted driving, and in particular to a chassis intelligent early warning system and method based on binocular vision. Background Art

[0002] With the rapid development of automobile intelligence, the safety monitoring and early warning functions of vehicle chassis are increasingly valued. When a vehicle is driving on a complex road, the passability of the chassis directly affects the driving safety and the service life of the vehicle, especially on a bumpy road or a road with protrusions. It is difficult for the driver to visually judge the distance between the chassis and the ground, which may cause the vehicle chassis to collide or scratch with the ground or road obstacles, causing chassis damage and vehicle entrapment, which will not only cause great economic losses, but also make the vehicle unable to pass later, and even cause traffic accidents in serious cases.

[0003] Traditional chassis warning systems mainly rely on ultrasonic or infrared sensors. When faced with special roads such as flooded sections, road collapses and uneven roads, the detection capabilities are insufficient and it is difficult to provide effective warnings in a timely manner, which increases the risk of vehicles driving under complex road conditions; when driving at high speeds or in narrow spaces, it is easy to cause the chassis to collide or scratch; it is difficult to fully cover low obstacles around the vehicle, and drivers tend to ignore these dangers when reversing or driving at low speeds, increasing the risk of accidents. Therefore, there are limitations in the detection range, accuracy, and ability to cope with complex environments. In addition, existing warning systems usually ignore the fire hazards in the chassis area, resulting in the inability to provide timely warnings when a fire occurs, endangering the safety of the vehicle and passengers.

[0004] Therefore, there is an urgent need for a technical solution that can improve monitoring accuracy, timeliness and adaptability to better protect the safety of the vehicle chassis and provide effective decision support for the driver. Summary of the invention

[0005] The purpose of the present invention is to provide a chassis intelligent warning system and method based on binocular vision to solve the limitations of existing chassis warning technology, provide an all-round and real-time chassis warning function, and provide a more reliable technical guarantee for the safe driving of vehicles.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A chassis intelligent warning system based on binocular vision, the system comprising:

[0008] An image acquisition module is used to acquire original images in front and on the sides of the vehicle through a binocular camera;

[0009] An image processing module, used for generating a depth map according to the original image, and transmitting the depth map to an intelligent analysis module;

[0010] An intelligent analysis module, used to receive the depth map, input a pre-trained chassis warning model to perform chassis passability analysis, and generate a chassis passability risk assessment result;

[0011] An early warning control module is used to compare the risk assessment result with a preset threshold value to determine whether to trigger an early warning signal;

[0012] The interactive module includes a touch display screen and a warning broadcast unit. The touch display screen is used to manually control the opening and closing of the system. The warning broadcast unit is used to provide visual and / or auditory warning prompts to the driver when the warning signal is triggered.

[0013] As a preferred solution of the present invention, the intelligent analysis module further includes:

[0014] A temperature detection unit, used to receive temperature data from a chassis temperature sensor and transmit the temperature data to the chassis early warning model to generate a fire risk assessment result;

[0015] The chassis warning model is a multi-task learning model that can simultaneously process the chassis passability and chassis fire risk assessment results. The chassis passability risk assessment results are used to describe the vehicle chassis passability risk under current road conditions, including obstacle height, road slope, collision risk, scratch risk and wading risk.

[0016] As a preferred solution of the present invention, the image acquisition module includes a front-view camera and a side-view camera. The front-view camera is installed on the front of the car and adopts a front-view binocular camera including a first lens and a second lens; the side-view camera is installed on the side of the vehicle and adopts a side-view binocular camera including a third lens and a fourth lens.

[0017] As a preferred solution of the present invention, the image processing module includes a camera calibration unit, a binocular correction unit, a binocular matching unit and a depth information unit;

[0018] The camera calibration unit is used to calibrate the binocular camera to obtain internal and external parameters and a homography matrix of the binocular camera, wherein the internal parameters include focal length, distortion coefficient and distortion factor, and the external parameters include displacement and rotation angle;

[0019] The binocular correction unit is used to correct the original image according to the calibration result, including eliminating distortion and row alignment, so that the imaging origin coordinates of each view are consistent, the optical axes of the two lenses in the binocular camera are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned;

[0020] The binocular matching unit is used to reduce the matching of corresponding points from a two-dimensional search space to a one-dimensional search space by using epipolar constraints, and to match corresponding pixel points on the original image collected by the binocular camera in the same scene to obtain a disparity map;

[0021] The depth information unit is used to calculate and generate the depth map in front and on the side of the vehicle based on the binocular vision algorithm. The formula is:

[0022] ;

[0023] In the formula, Z represents the depth map, f is the camera focal length, B is the center distance between the two lenses, X R , X T are the imaging points of the object P , P’ Relative to the camera optical center O R and O T The imaging offset.

[0024] As a preferred solution of the present invention, the construction process of the chassis warning model is as follows:

[0025] Data collection: Use binocular cameras to collect image data from the front and sides of the vehicle under complex road conditions, and use temperature sensors to record temperature data of the chassis area. The complex road conditions include obstacles, wading sections, ramps, and potholes.

[0026] Collection optimization: Preliminary processing of collected data through multi-scale transformation models, dynamic adjustment of the time and space granularity of data collection, and optimization of data quality and relevance;

[0027] Data preprocessing: Apply Gaussian filter to remove noise from image data and apply moving average filter to temperature data; use multi-scale convolutional neural network based on deep learning to extract obstacle features and road features from image data, extract thermal distribution features from temperature data, and use annotation tools to classify the obstacle features, road features and thermal distribution features;

[0028] Feature extraction: Use multi-scale attention heads to extract motion features from images and temperature data at different time granularities; analyze motion features through the information increment mode adaptive module, calculate the information increment between time steps, and generate continuity features for describing chassis motion trends;

[0029] Model training: The motion features and continuity features are input into a multi-task learning model that can simultaneously handle chassis passability and fire risk assessment for training. The model parameters are optimized by weighted summation of the loss functions of different tasks.

[0030] Model optimization: Use grid search or Bayesian optimization methods to adjust the model's hyperparameters, use pruning technology to remove redundant neurons, reduce the model's computational complexity, and use model compression technology to reduce the model's storage requirements and improve inference speed;

[0031] Through online learning methods, new data is continuously collected, and the model is retrained and updated regularly.

[0032] As a preferred solution of the present invention, the multi-scale transformation model is used to divide the collected data into different time and space scales, and perform preliminary processing through the following steps:

[0033] Performing segmentation processing on the collected original image data on a time scale to generate multiple image data subsets of different time granularities;

[0034] Performing a multi-scale convolution operation on the image data subset at a spatial scale to extract features at each spatial scale;

[0035] By fusing features at different temporal and spatial scales, a multi-scale feature representation is formed as the input of the multi-scale attention mechanism.

[0036] As a preferred solution of the present invention, the multi-scale attention head is implemented by the following steps:

[0037] Calculate the query matrix at each time granularity Q , key matrix K Sum Matrix V , the formula is as follows:

[0038] ; ; ;

[0039] In the formula, W Q , W K , W V is the linear transformation matrix, X is the input feature data;

[0040] Compute the query matrix at different time scales Q With key matrix K The dot product of , and the attention weight is calculated by the softmax function, the formula is:

[0041] ;

[0042] In the formula, Used to calculate the weighted sum of input features; T is the transpose operator; Represents the key matrix K Dimensions;

[0043] For the attention weights at different time scales, a padding mask matrix is ​​used to mask irrelevant or unrelated time steps, so that only the most relevant features are focused on at each time scale; the mask matrix The generation method is:

[0044] ;

[0045] In the formula, t i and t j are the time steps on different time scales respectively;

[0046] The attention results at each time scale are weightedly summed to generate multi-scale motion features, and the multi-scale motion features are passed as input to the multi-task learning model.

[0047] As a preferred solution of the present invention, the information increment mode adaptive module is implemented by the following steps:

[0048] The observation matrix of each time step is calculated based on the sequence mask matrix to describe the observable motion features at the current time step. The formula is expressed as:

[0049] ;

[0050] In the formula, is the time step The observation matrix, is the sequence mask matrix, is the time step The original characteristics of

[0051] By calculating the current time step With the previous time step The feature difference between them is used to obtain the information increment , the formula is:

[0052] ;

[0053] By comparing the information increments between different step lengths, the feature weights of each time step are dynamically adjusted;

[0054] The adjusted feature weights are applied to generate the final continuous features and passed to the multi-task learning model for subsequent training.

[0055] A warning method of a chassis intelligent warning system based on binocular vision comprises the following steps:

[0056] The original images in front and on the sides of the vehicle’s travel direction are collected by using a binocular camera;

[0057] Generate a depth map according to the original image, input it into a pre-trained chassis warning model to perform chassis passability analysis, and generate a chassis passability risk assessment result;

[0058] The risk assessment result is compared with a preset threshold to determine whether a warning signal is triggered; if a warning signal needs to be triggered, a visual and / or auditory warning prompt is provided to the driver.

[0059] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the binocular camera performs non-contact detection through the camera, does not produce any physical impact on the object being measured, and does not rely on a single sensor; by constructing a chassis warning model, combining a multi-scale transformation model with a multi-scale attention head, the real-time monitoring and warning capabilities of the chassis system are significantly improved; the chassis warning model can accurately capture the dynamic changes of the vehicle chassis under complex road conditions and provide accurate risk warnings; the multi-scale transformation model enables the system to process information at different scales, effectively enhancing the recognition and processing capabilities of multi-dimensional data; the multi-scale attention head further optimizes the aggregation of data and the extraction of key information, and improves the global perception ability of the model; the information increment mode adaptive module adaptively adjusts and updates the warning strategy to ensure that the system can maintain efficient and accurate warning performance under different environments and operating conditions. The present invention uses machines instead of manpower to analyze road conditions, effectively reducing the error of empirical judgment, not only improving the warning accuracy and response speed of the chassis system, but also enhancing the adaptability and reliability of the system, and can also reduce the significant losses caused by damage to automobile parts due to road conditions, and can avoid congestion for subsequent vehicles, which can ensure the smoothness of the road to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0061] Figure 1 It is a schematic diagram of the modular structure of the system of the present invention;

[0062] Figure 2 Schematic diagram of binocular imaging principle in an embodiment of the present invention;

[0063] Figure 3 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0065] The existing chassis sensor system is easily disturbed by external factors in complex environments, causing false alarms or missed alarms, resulting in inaccurate measurement results; some early warning systems have delays in signal processing and alarm triggering, and cannot issue early warnings in time when the vehicle encounters an emergency, resulting in the chassis reacting only after the risk occurs, lacking sufficient lead time; the chassis heights of different models, the types and adjustments of vehicle suspension systems are different, resulting in a unified early warning threshold setting that is difficult to take into account the needs of all models and road conditions, and the system is difficult to adjust individually. The present invention uses machines instead of manpower to analyze road conditions, effectively reducing the errors of empirical judgment, and can increase the service life of the main components of the vehicle.

[0066] like Figure 1 As shown, an embodiment of the present invention provides a chassis intelligent warning system based on binocular vision, including an image acquisition module, an image processing module, an intelligent analysis module, a warning control module and an interaction module.

[0067] (1) Image acquisition module

[0068] Used to collect original images in front and on the sides of the vehicle's travel direction through a binocular camera;

[0069] The image acquisition module includes a front-view camera and a side-view camera. The front-view camera is installed on the front of the car and adopts a front-view binocular camera including a first lens and a second lens; the side-view camera is installed on the side of the vehicle and adopts a side-view binocular camera including a third lens and a fourth lens.

[0070] A binocular camera is installed in front of the car. By calculating the parallax of the two images, the distance to the scene in front (the range captured by the image) can be directly measured without judging what type of obstacle is in front. Therefore, for any type of obstacle, necessary warnings or braking can be carried out according to the change of distance information. The principle of binocular camera is similar to that of human eyes. The human eye can perceive the distance of objects because there is a difference in the images presented by the two eyes to the same object, which is also called "parallax". The farther the object is, the smaller the parallax; conversely, the larger the parallax.

[0071] (2) Image processing module

[0072] Used to generate a depth map based on the original image and pass the depth map to the intelligent analysis module; including a camera calibration unit, a binocular correction unit, a binocular matching unit and a depth information unit;

[0073] The camera calibration unit is used to calibrate the binocular camera to obtain the internal and external parameters and homography matrix of the binocular camera. The internal parameters include focal length, distortion coefficient and distortion factor, and the external parameters include displacement and rotation angle.

[0074] The essence of camera calibration is the process of determining the internal and external parameters of the camera, and the geometric parameters of the camera are the parameter matrix between the image and the actual object.

[0075] The binocular correction unit is used to correct the original image according to the calibration results, including eliminating distortion and row alignment, so that the coordinates of the imaging origin of each view are consistent, the optical axes of the two lenses in the binocular camera are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned;

[0076] Binocular correction is to eliminate distortion and align the left and right views based on the monocular internal parameter data (focal length, imaging origin, distortion coefficient) and the binocular relative position relationship (rotation matrix and translation vector) obtained after camera calibration, so that the imaging origin coordinates of the left and right views are consistent, the optical axes of the two cameras are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned. In this way, any point on one image and its corresponding point on the other image must have the same row number, and the corresponding point can be matched by performing a one-dimensional search in the row.

[0077] The binocular matching unit is used to reduce the matching of corresponding points from a two-dimensional search space to a one-dimensional search space by using epipolar constraints, and to match corresponding pixel points on the original image collected by the binocular camera in the same scene to obtain a disparity map;

[0078] The premise of camera calibration is to locate the same point on the two camera images, that is, to match the points of the left and right images, which involves binocular calibration. If the corresponding point is matched in another two-dimensional image space based on the feature of a point in one image, this process will be very time-consuming. In order to reduce the amount of matching search calculations, limit constraints can be used to reduce the matching of corresponding points from a two-dimensional search space to a one-dimensional search space.

[0079] The depth information unit is used to calculate and generate the depth map in front and on the side of the vehicle based on the binocular vision algorithm, such as Figure 2 As shown:

[0080] O R , O T are the optical centers of the left and right lenses of the camera, p , p’ are the imaging points of the object on the two camera sensors, f is the camera focal length, B is the center distance between the two lenses, Z represents the generated depth map, X R , X T are the imaging points of the object P , P’ Relative to the camera optical center O R and O T Imaging offset;

[0081] dis for p arrive p’ The distance is:

[0082] ;

[0083] According to the principle of similar triangles:

[0084] ;

[0085] We can get:

[0086] ;

[0087] In the formula, f is the camera focal length, B is the center distance between the two lenses, which can be obtained through calibration; therefore, as long as X R - X T The value of can be used to find the depth map Z .

[0088] (3) Intelligent analysis module

[0089] Used to receive the depth map, input the pre-trained chassis warning model to perform chassis passability analysis, and generate chassis passability risk assessment results;

[0090] The intelligent analysis module also includes a temperature detection unit for receiving temperature data from a chassis temperature sensor and transmitting the temperature data to a chassis early warning model to generate a fire risk assessment result;

[0091] The chassis warning model is a multi-task learning model that can simultaneously process the risk assessment results of chassis passability and chassis fire. The risk assessment results of chassis passability are used to describe the passability risk of the vehicle chassis under the current road conditions, including obstacle height, road slope, collision risk, scratch risk and wading risk.

[0092] In one embodiment, the construction process of the chassis warning model is as follows:

[0093] a. Data collection: Use binocular cameras to collect image data from the front and sides of the vehicle under complex road conditions, and use temperature sensors to record temperature data of the chassis area. Complex road conditions include but are not limited to obstacle environments, wading sections, ramps, and potholes;

[0094] b. Collection optimization: Preliminary processing of collected data through multi-scale transformation models, dynamic adjustment of the time and space granularity of data collection, and optimization of data quality and relevance;

[0095] The multi-scale transformation model is used to segment the collected data into different time and space scales and perform preliminary processing through the following steps:

[0096] Performing segmentation processing on the collected original image data on a time scale to generate multiple image data subsets of different time granularities;

[0097] Perform multi-scale convolution operations on the image data subsets at different spatial scales to extract features at each spatial scale;

[0098] By fusing features at different temporal and spatial scales, a multi-scale feature representation is formed as the input of the multi-scale attention mechanism.

[0099] c. Data preprocessing: Apply Gaussian filter to remove noise from image data and apply moving average filter to temperature data; use multi-scale convolutional neural network based on deep learning to extract obstacle features and road features from image data, extract thermal distribution features (such as temperature gradient, hot spot area) through temperature data, and use annotation tools to classify obstacle features, road features and thermal distribution features (such as obstacle type, road slope, lighting conditions, road surface conditions, etc.);

[0100] Examples of obstacle features:

[0101] Obstacle types: such as rocks, bumps, potholes, speed bumps, traffic cones, etc.;

[0102] Obstacle size: including height, width, length and other size information;

[0103] Obstacle material: such as metal, concrete, plastic, wood, etc. These materials may affect the reflective properties of the obstacle and the risk assessment;

[0104] Obstacle position: front, rear, left, and right positions relative to the vehicle, and distance from the center line of the road;

[0105] Obstacle mobility: stationary or moving obstacles, direction and speed of movement;

[0106] Examples of road features:

[0107] Road slope: including uphill, downhill and their angles (expressed in degrees or percentage);

[0108] Road curvature: the degree of curvature of the road, including straight roads, curves and curvature radius;

[0109] Road surface type: such as asphalt road, concrete road, gravel road, dirt road, flooded road, etc.;

[0110] Road conditions: such as dry, slippery, snowy, icy, etc. These conditions will affect the vehicle's passability;

[0111] Road width: including road width information such as one-way streets and two-way streets.

[0112] d. Feature extraction: Use multi-scale attention heads to extract motion features from images and temperature data at different time granularities; analyze the motion features through the information increment mode adaptive module, calculate the information increment between time steps, and generate continuity features for describing the chassis motion trend;

[0113] The multi-scale attention head is implemented by the following steps:

[0114] Calculate the query matrix at each time granularity Q , key matrix K Sum Matrix V , the formula is as follows:

[0115] ; ; ;

[0116] In the formula, W Q ,W K , W V is the linear transformation matrix, X is the input feature data;

[0117] Compute the query matrix at different time scales Q With key matrix K The dot product of , and the attention weight is calculated by the softmax function, the formula is:

[0118] ;

[0119] In the formula, Used to calculate the weighted sum of input features; T is the transpose operator; Represents the key matrix K Dimensions;

[0120] For the attention weights at different time scales, a padding mask matrix is ​​used to mask irrelevant or unrelated time steps, so that only the most relevant features are focused on at each time scale; the mask matrix The generation method is:

[0121] ;

[0122] In the formula, t i and t j are the time steps on different time scales respectively;

[0123] Calculate the information increment of each time step, and dynamically adjust the feature weight of each time step by comparing the information increment between different steps;

[0124] The adjusted feature weights are applied to generate the final continuous features and passed to the multi-task learning model for subsequent training.

[0125] The information increment mode adaptive module is implemented through the following steps:

[0126] The observation matrix of each time step is calculated based on the sequence mask matrix to describe the observable motion features at the current time step. The formula is expressed as:

[0127] ;

[0128] In the formula, is the time step The observation matrix, is the sequence mask matrix, is the time step The original characteristics of

[0129] By calculating the current time step With the previous time step The feature difference between them is used to obtain the information increment , the formula is:

[0130] ;

[0131] By comparing the information increments between different step lengths, the feature weights of each time step are dynamically adjusted;

[0132] The adjusted feature weights are applied to generate the final continuous features and passed to the multi-task learning model for subsequent training.

[0133] e. Model training: Input motion features and continuity features into a multi-task learning model that can simultaneously handle chassis passability and fire risk assessment for training. Optimize model parameters by weighted summation of loss functions of different tasks.

[0134] f. Model optimization: Use grid search or Bayesian optimization methods to adjust the model's hyperparameters (such as learning rate, batch size, convolution kernel size, etc.) to improve the model's performance. Use pruning technology to remove redundant neurons and reduce the model's computational complexity. Use model compression technology to reduce the model's storage requirements and improve inference speed.

[0135] Through online learning methods, new data is continuously collected, and the model is regularly retrained and updated to ensure that it maintains high accuracy in ever-changing road conditions and environments.

[0136] (4) Early warning control module

[0137] Used to compare the risk assessment results with the preset thresholds to determine whether to trigger an early warning signal;

[0138] (5) Interaction module

[0139] Including touch display screen and early warning broadcast unit;

[0140] The touch screen is used to manually control the system on and off. In the parking state, the driver can operate the vehicle system through the central control screen to turn on the chassis passability warning function and operate the camera illumination range on the central control screen to determine the road area for chassis passability warning.

[0141] The warning broadcast unit is used to provide visual and / or auditory warning prompts to the driver when the warning signal is triggered. For example, it can be sounded once, and if it is not possible, it will sound a long time until the warning system is turned off.

[0142] like Figure 3As shown in FIG. 1 , another embodiment of the present invention is provided, which provides a warning method of a chassis intelligent warning system based on binocular vision, comprising the following steps:

[0143] The original images in front and on the sides of the vehicle’s travel direction are collected by using a binocular camera;

[0144] Generate a depth map according to the original image, input it into a pre-trained chassis warning model to perform chassis passability analysis, and generate a chassis passability risk assessment result;

[0145] The risk assessment result is compared with a preset threshold to determine whether a warning signal is triggered; if a warning signal needs to be triggered, a visual and / or auditory warning prompt is provided to the driver.

[0146] In summary, the present invention uses a binocular camera for non-contact detection, which will not have any physical impact on the object being measured and does not rely on a single sensor; by constructing a chassis warning model, combining a multi-scale transformation model with a multi-scale attention head, the real-time monitoring and warning capabilities of the chassis system are significantly improved; the chassis warning model can accurately capture the dynamic changes of the vehicle chassis under complex road conditions and provide accurate risk warnings; the multi-scale transformation model enables the system to process information at different scales, effectively enhancing the recognition and processing capabilities of multi-dimensional data; the multi-scale attention head further optimizes the aggregation of data and the extraction of key information, and improves the global perception ability of the model; the information increment mode adaptive module adaptively adjusts and updates the warning strategy to ensure that the system can maintain efficient and accurate warning performance under different environments and operating conditions. The present invention uses machines instead of manpower to analyze road conditions, effectively reducing the error of empirical judgment, not only improving the warning accuracy and response speed of the chassis system, but also enhancing the adaptability and reliability of the system, and can also reduce the significant losses caused by damage to automobile parts due to road conditions, and can avoid congestion of subsequent vehicles, which can ensure the smoothness of the road to a certain extent.

[0147] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0148] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0149] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A chassis intelligent warning system based on binocular vision, characterized in that: The system comprises: An image acquisition module is used to acquire original images in front and on the sides of the vehicle through a binocular camera; An image processing module, used for generating a depth map according to the original image, and transmitting the depth map to an intelligent analysis module; An intelligent analysis module, used to receive the depth map, input a pre-trained chassis warning model to perform chassis passability analysis, and generate a chassis passability risk assessment result; An early warning control module is used to compare the risk assessment result with a preset threshold value to determine whether to trigger an early warning signal; An interactive module, comprising a touch screen and a warning broadcast unit, wherein the touch screen is used to manually control the opening and closing of the system, and the warning broadcast unit is used to provide a visual and / or auditory warning prompt to the driver when a warning signal is triggered; The intelligent analysis module also includes: A temperature detection unit, used to receive temperature data from a chassis temperature sensor and transmit the temperature data to the chassis early warning model to generate a fire risk assessment result; The chassis warning model is a multi-task learning model that can simultaneously process the chassis passability and chassis fire risk assessment results. The chassis passability risk assessment results are used to describe the vehicle chassis passability risk under the current road conditions, including obstacle height, road slope, collision risk, scratch risk, and wading risk. The construction process of the chassis early warning model is as follows: Data collection: Use binocular cameras to collect image data from the front and sides of the vehicle under complex road conditions, and use temperature sensors to record temperature data of the chassis area. The complex road conditions include obstacles, wading sections, ramps, and potholes. Collection optimization: Preliminary processing of collected data through multi-scale transformation models, dynamic adjustment of the time and space granularity of data collection, and optimization of data quality and relevance; Data preprocessing: Apply Gaussian filter to remove noise from image data and apply moving average filter to temperature data; use multi-scale convolutional neural network based on deep learning to extract obstacle features and road features from image data, extract thermal distribution features from temperature data, and use annotation tools to classify the obstacle features, road features and thermal distribution features; Feature extraction: Use multi-scale attention heads to extract motion features from images and temperature data at different time granularities; analyze motion features through the information increment mode adaptive module, calculate the information increment between time steps, and generate continuity features for describing chassis motion trends; Model training: The motion features and continuity features are input into a multi-task learning model that can simultaneously handle chassis passability and fire risk assessment for training. The model parameters are optimized by weighted summation of the loss functions of different tasks. Model optimization: Use grid search or Bayesian optimization methods to adjust the model's hyperparameters, use pruning technology to remove redundant neurons, reduce the model's computational complexity, and use model compression technology to reduce the model's storage requirements and improve inference speed; Through online learning methods, new data is continuously collected, and the model is retrained and updated regularly.

2. The chassis intelligent warning system based on binocular vision according to claim 1 is characterized in that: The image acquisition module includes a front-view camera and a side-view camera. The front-view camera is installed on the front of the car and adopts a front-view binocular camera including a first lens and a second lens; the side-view camera is installed on the side of the vehicle and adopts a side-view binocular camera including a third lens and a fourth lens.

3. The chassis intelligent warning system based on binocular vision according to claim 1 is characterized in that: The image processing module includes a camera calibration unit, a binocular correction unit, a binocular matching unit and a depth information unit; The camera calibration unit is used to calibrate the binocular camera to obtain internal and external parameters and a homography matrix of the binocular camera, wherein the internal parameters include focal length, distortion coefficient and distortion factor, and the external parameters include displacement and rotation angle; The binocular correction unit is used to correct the original image according to the calibration result, including eliminating distortion and row alignment, so that the imaging origin coordinates of each view are consistent, the optical axes of the two lenses in the binocular camera are parallel, the left and right imaging planes are coplanar, and the epipolar lines are aligned; The binocular matching unit is used to reduce the matching of corresponding points from a two-dimensional search space to a one-dimensional search space by using epipolar constraints, and to match corresponding pixel points on the original image collected by the binocular camera in the same scene to obtain a disparity map; The depth information unit is used to calculate and generate the depth map in front and on the side of the vehicle based on the binocular vision algorithm. The formula is: ; In the formula, Z represents the depth map, f is the camera focal length, B is the center distance between the two lenses, X R , X T are the imaging points of the object P , P’ Relative to the camera optical center O R and O T The imaging offset.

4. The chassis intelligent warning system based on binocular vision according to claim 1 is characterized in that: The multi-scale transformation model is used to divide the collected data into different time and space scales, and perform preliminary processing through the following steps: Performing segmentation processing on the collected original image data on a time scale to generate multiple image data subsets of different time granularities; Performing a multi-scale convolution operation on the image data subset at a spatial scale to extract features at each spatial scale; By fusing features at different temporal and spatial scales, a multi-scale feature representation is formed as the input of the multi-scale attention mechanism.

5. The chassis intelligent warning system based on binocular vision according to claim 1 is characterized in that: The multi-scale attention head is implemented by the following steps: Compute the query matrix at each time scale Q , key matrix K Sum Matrix V , the formula is as follows: ; ; ; In the formula, W Q , W K , W V is the linear transformation matrix, X is the input feature data; Compute the query matrix at different time scales Q With key matrix K The dot product of , and the attention weight is calculated by the softmax function, the formula is: ; In the formula, Used to calculate the weighted sum of input features; T is the transpose operator; Represents the key matrix K Dimensions; For the attention weights at different time scales, a padding mask matrix is ​​used to mask irrelevant or unrelated time steps, so that only the most relevant features are focused on at each time scale; the mask matrix The generation method is: ; In the formula, t i and t j are the time steps on different time scales respectively; The attention results at each time scale are weightedly summed to generate multi-scale motion features, and the multi-scale motion features are passed as input to the multi-task learning model.

6. The chassis intelligent warning system based on binocular vision according to claim 1 is characterized in that: The information increment mode adaptive module is implemented by the following steps: The observation matrix of each time step is calculated based on the sequence mask matrix to describe the motion features observable at the current time step. The formula is expressed as: ; In the formula, is the time step The observation matrix, is the sequence mask matrix, is the time step The original characteristics of By calculating the current time step With the previous time step The feature difference between them is used to obtain the information increment , the formula is: ; By comparing the information increments between different step lengths, the feature weights of each time step are dynamically adjusted; The adjusted feature weights are applied to generate the final continuous features and passed to the multi-task learning model for subsequent training.

7. The warning method of a chassis intelligent warning system based on binocular vision according to any one of claims 1 to 6, characterized in that: The following steps are involved: The original images in front and on the sides of the vehicle’s travel direction are collected by using a binocular camera; Generate a depth map according to the original image, input it into a pre-trained chassis warning model to perform chassis passability analysis, and generate a chassis passability risk assessment result; Compare the risk assessment result with a preset threshold to determine whether a warning signal is triggered; If a warning signal needs to be triggered, a visual and / or audible warning prompt is provided to the driver.