Vehicle-mounted environment perception and early warning integrated system
By integrating multi-source data acquisition, data fusion and multimodal neural network processing, and combining visual, auditory and tactile warnings, the problems of information sharing difficulties and inaccurate hazard assessments in existing vehicle-mounted environmental perception systems are solved, achieving high-precision, real-time environmental perception and warning, and improving driving safety and experience.
Patent Information
- Application Number
- CN202511033814.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-09-16
AI Technical Summary
Existing vehicle-mounted environmental perception systems have difficulties in information sharing and integration, inaccurate risk assessment, and a single and inflexible warning method, resulting in low safety and efficiency of the system in complex environments.
It adopts a system that integrates multi-source data collection, data fusion, hazard assessment and early warning. It collects multi-source heterogeneous data through radar, cameras and other equipment, uses multimodal neural networks and machine learning models to process data and assess hazards, and combines multiple early warning methods such as vision, hearing and touch.
It achieves high precision, real-time and reliability in environmental perception and warning, improves driving safety and the driver's driving experience, reduces misjudgments and missed judgments, and provides personalized warning information.
Smart Images
Figure CN120645951A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of environmental perception, and in particular to an integrated vehicle-mounted environmental perception and early warning system. Background Art
[0002] With the development of intelligent transportation systems and the continuous advancement of automotive automation technology, vehicle-mounted environmental perception and warning systems have gradually become an important part of improving driving safety.
[0003] However, existing systems typically use different types of sensors for environmental perception, and the data generated by each sensor is often stored on different platforms, making information sharing and integration difficult. The limitations of a single sensor prevent the system from fully perceiving the surrounding situation in complex environments, making it prone to information omissions, which in turn affects the timeliness and effectiveness of safety warnings.
[0004] Furthermore, in terms of hazard assessment, some systems rely solely on fixed rule-based approaches. These rules, formulated based on limited experience and scenario assumptions, are difficult to adapt to complex and changing traffic environments and are prone to misjudgments or missed judgments. While some machine learning-based systems have some adaptive capabilities, they suffer from limitations in training data and insufficient model generalization. Consequently, the accuracy of hazard assessments decreases significantly in new scenarios or extreme conditions.
[0005] Furthermore, existing systems lack flexibility and rely solely on audio or simple visual prompts, failing to provide personalized warning information based on hazard levels and driving scenarios. This results in low driver attention and response efficiency. Furthermore, independent functional modules lack effective collaboration and information sharing, resulting in low overall system performance and efficiency, making it difficult to meet the high-precision, real-time, and reliable environmental perception and warning requirements of intelligent driving.
[0006] To this end, technicians in this field have proposed an integrated vehicle-mounted environmental perception and warning system, which aims to comprehensively utilize multi-source data, integrate environmental perception, data fusion, hazard assessment and warning functions, improve the accuracy and efficiency of environmental perception and warning, and thus significantly improve the driver's safety experience. Summary of the Invention
[0007] In order to solve the above technical problems, the present invention provides an integrated vehicle-mounted environment perception and warning system to solve the problems raised in the background technology.
[0008] An integrated vehicle-mounted environmental perception and warning system, comprising:
[0009] A data acquisition module is used to collect data about the vehicle's surroundings through acquisition equipment, such as radar and cameras, to obtain multi-source heterogeneous data.
[0010] A data processing module is used to extract features from the multi-source heterogeneous data after preprocessing to obtain key feature parameters, and to perform data fusion on the key feature parameters to obtain fused data;
[0011] An environmental state perception module is used to determine the current environmental state of the vehicle based on the fused data and a dynamic model of the vehicle's surrounding environment to obtain state perception information;
[0012] A risk assessment module is used to obtain a risk assessment result based on the fused data and state perception information through a risk assessment model, and to obtain an adjusted risk assessment result in combination with a risk assessment rule base;
[0013] The early warning module is used to issue an early warning signal according to the adjusted risk assessment result when a preset early warning threshold is exceeded, and send the early warning signal to the terminal corresponding to the current vehicle.
[0014] Preferably, the data processing module is further used for:
[0015] Collecting the surrounding environment of the vehicle based on the acquisition device to obtain multi-source heterogeneous data, wherein the multi-source heterogeneous data includes image data and point cloud data;
[0016] The multi-source heterogeneous data is pre-processed and then feature extracted to obtain key feature parameters, which include image feature parameters f img and point cloud feature parameters f pc ;
[0017] Use the following formula to convert the image feature parameter f img and point cloud feature parameters f pc As input, it is fused through a multimodal neural network to obtain fused data:
[0018] f fusion =F(f img ,f pc )
[0019] Among them, F is the mapping function of the multimodal neural network, f fusion To fuse data.
[0020] Preferably, the data processing module is further used for:
[0021] Based on the fused data f fusion , output the target category and location information through the target detection model;
[0022] Generate candidate regions through the region proposal network RPN in the target detection model, perform feature extraction and classification regression on the candidate regions, and output the target category probability P(c) and position coordinates (x, y, w, h), where c represents the target category, (x, y) is the center coordinate of the target box, and w and h are the width and height of the target box respectively;
[0023] According to the category probability P(c) and the position coordinates (x, y, w, h), the target is continuously tracked and identified through feature matching.
[0024] Preferably, the environmental state perception module is further used to:
[0025] Based on the fused data f fusion ,Through 3D reconstruction technology, a road model is constructed;
[0026] Based on the fused data f fusion , combined with the category probability P(c) and the location coordinates (x, y, w, h), when the category probability P(c) is determined to be a traffic participant, a traffic participant model is constructed:
[0027] Based on the road model and traffic participant model, the current vehicle environment state is inferred. The distance d between the vehicle center position and the lane boundary is calculated using the road model. The following formula is used to determine whether the current vehicle is traveling within the lane:
[0028]
[0029] Where D is the lane status determination result of the current vehicle, D = 1 means the current vehicle is in the lane, D = 0 means the current vehicle is outside the lane, d θ is the preset lane distance threshold;
[0030] The distance m between the vehicle and surrounding traffic participants is calculated using the traffic participant model. The following formula is used to determine whether there are any obstacles in close proximity to the vehicle:
[0031]
[0032] Among them, M is the distance between the current vehicle and the traffic participants. M = 1 means that there is a close obstacle to the current vehicle, and M = 0 means that there is no close obstacle to the current vehicle. θ is the preset safety distance threshold;
[0033] The lane state determination result D and the distance condition determination result M are integrated and expressed as a vector: S = (s1, s2, ..., s n ), where s iRepresents different environmental state judgment results, n is the total number of environmental state judgment items, and S is the state perception information of the current vehicle environment.
[0034] Preferably, the risk assessment module is further used to:
[0035] According to the fusion data f fusion and state perception information S, extract the evaluation feature vector X=(x1,x2,...,x n ), where x i represents the i-th feature, n is the number of features;
[0036] The evaluation feature vector X is used as input through the multi-layer perceptron model, and the following formula is used to obtain the category with the highest probability as the risk assessment result:
[0037]
[0038] Among them, y k represents the output of the kth neuron, The results of the risk assessment;
[0039] Based on the Boolean condition C={C1,C2,...,C z}, where each Boolean condition corresponds to a rule, for Boolean condition C i Expressed as:
[0040]
[0041] Combined with the risk assessment rule base, the risk assessment results Make adjustments to obtain the adjusted risk assessment results:
[0042]
[0043] Among them, y final is the adjusted risk assessment result.
[0044] Preferably, the early warning module is further used to:
[0045] The adjusted risk assessment result y final Compare with the preset warning threshold T and use the following formula to determine whether the current situation issues a warning signal:
[0046]
[0047] Among them, Q represents the state of the warning signal, Q=1 means that the warning signal is issued, and Q=0 means that the warning signal is not issued.
[0048] Compared with the prior art, the present invention has the following beneficial effects:
[0049] 1. By integrating environmental perception, data fusion, hazard assessment, and early warning functions into one system, the present invention achieves close collaboration and information sharing among various modules, thereby improving the overall performance and efficiency of the system. By perceiving the environment in real time, accurately assessing hazards, and issuing timely early warnings, it helps drivers promptly identify potential hazards and take measures to avoid accidents, significantly improving driving safety.
[0050] 2. The present invention improves the accuracy and reliability of the system's environmental perception and hazard assessment by processing multi-source data and combining the machine learning model with the hazard assessment rule base in the hazard assessment module, reduces misjudgments and missed judgments, provides drivers with comprehensive environmental information and timely warnings, reduces the driver's driving burden, makes the driving process easier and more comfortable, and enhances the driving experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 This is a block diagram of the vehicle-mounted environmental perception and warning integrated system of the present invention. DETAILED DESCRIPTION
[0052] The following embodiments of the present invention are described in further detail with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.
[0053] As attached Figure 1 As shown:
[0054] Embodiment: The present invention provides an integrated vehicle-mounted environment perception and warning system, comprising:
[0055] The data acquisition module is used to collect information about the vehicle's surroundings through acquisition equipment, obtaining multi-source heterogeneous data; the acquisition equipment includes radars and cameras; through real-time collection of information about the vehicle's surroundings by multiple sensors, the system can fully understand the vehicle's environmental conditions and provide a basis for decision-making and early warning; among them, the millimeter-wave radar detects the distance, speed and angle of targets around the vehicle in real time; the lidar obtains three-dimensional information of the target and constructs a high-precision point cloud map; various cameras realize all-round visual perception and identify road targets; ultrasonic sensors are responsible for close-range obstacle detection, providing support for low-speed driving and parking.
[0056] The data processing module is used to extract features from multi-source heterogeneous data after preprocessing to obtain key feature parameters, and to fuse the key feature parameters to obtain fused data;
[0057] The vehicle's surrounding environment is collected using acquisition equipment to obtain multi-source heterogeneous data, including image data and point cloud data;
[0058] After preprocessing, the multi-source heterogeneous data is subjected to feature extraction to obtain key feature parameters, which include image feature parameters f img and point cloud feature parameters f pc ;
[0059] Use the following formula to convert the image feature parameter f img and point cloud feature parameters f pc As input, it is fused through a multimodal neural network to obtain fused data:
[0060] f fusion =F(f img ,f pc )
[0061] Among them, F is the mapping function of the multimodal neural network, f fusion To fuse data.
[0062] And, based on the fusion data f fusion , output the target category and location information through the target detection model;
[0063] The candidate regions are generated by the region proposal network (RPN) in the target detection model. Feature extraction and classification regression are performed on the candidate regions to output the target category probability P(c) and position coordinates (x, y, w, h), where c represents the target category, (x, y) is the center coordinate of the target box, and w and h are the width and height of the target box respectively.
[0064] According to the category probability P(c) and position coordinates (x, y, w, h), the target is continuously tracked and identified through feature matching.
[0065] The data from different sensors are fused and processed to eliminate redundancy and contradiction between the data, improve the accuracy and reliability of the data, thereby obtaining a more comprehensive and accurate representation of environmental information, reducing the error and uncertainty of a single sensor, and enhancing the robustness of the system's environmental perception.
[0066] The environmental state perception module is used to determine the current environmental state of the vehicle based on the fused data and the dynamic model of the vehicle's surrounding environment to obtain state perception information;
[0067] Based on the fusion data f fusion ,Through 3D reconstruction technology, a road model is constructed;
[0068] Based on the fusion data f fusion , combined with the category probability P(c) and the location coordinates (x, y, w, h), when the category probability P(c) is determined to be a traffic participant, the traffic participant model is constructed:
[0069] Based on the road model and traffic participant model, the vehicle's current environmental state is inferred. The distance d between the vehicle's center position and the lane boundary is calculated using the road model. The following formula is used to determine whether the vehicle is currently traveling within the lane:
[0070]
[0071] Where D is the lane status determination result of the current vehicle, D = 1 means the current vehicle is in the lane, D = 0 means the current vehicle is outside the lane, d θ is the preset lane distance threshold;
[0072] The distance m between the vehicle and surrounding traffic participants is calculated using the traffic participant model. The following formula is used to determine whether there are any obstacles in close proximity to the vehicle:
[0073]
[0074] Among them, M is the distance between the current vehicle and the traffic participants. M = 1 means that there is a close obstacle to the current vehicle, and M = 0 means that there is no close obstacle to the current vehicle. θ is the preset safety distance threshold;
[0075] The lane state determination result D and the distance state determination result M are integrated and expressed as a vector: S = (s1, s2, ..., s n ), where s i Represents different environmental state judgment results, n is the total number of environmental state judgment items, and S is the state perception information of the current vehicle environment.
[0076] The hazard assessment module, based on the fusion data f fusion and state perception information S, extract the evaluation feature vector X=(x1,x2,...,x n ), where x i represents the i-th feature, n is the number of features;
[0077] Through the multi-layer perceptron model, the evaluation feature vector X is used as input, and the following formula is used to obtain the category with the highest probability as the risk assessment result:
[0078]
[0079] Among them, y k represents the output of the kth neuron, The results of the risk assessment;
[0080] Based on the Boolean condition C={C1,C2,...,C z}, where each Boolean condition corresponds to a rule, for Boolean condition C i Expressed as:
[0081]
[0082] Combined with the risk assessment rule base to evaluate the risk assessment results Make adjustments to obtain the adjusted risk assessment results:
[0083]
[0084] Among them, y final This is the adjusted hazard assessment result. Based on the fused data and state perception information, a machine learning-based model, combined with a hazard assessment rule base, assesses the danger level of the current environment. This hazard assessment result accurately identifies potential dangers facing the vehicle, providing decision-making support for the early warning module and helping the driver understand dangerous situations in advance and take appropriate measures.
[0085] The early warning module will adjust the risk assessment results y final Compare with the preset warning threshold T and use the following formula to determine whether the current situation issues a warning signal:
[0086]
[0087] Where Q represents the status of the warning signal: Q = 1 indicates a warning signal is issued, and Q = 0 indicates no warning signal is issued. Based on the risk assessment results, when the preset warning threshold is exceeded, a warning signal is issued to alert the driver to potential dangers. By promptly notifying the driver of dangerous situations, the driver has ample time to react, reducing the possibility of traffic accidents and ensuring driving safety.
[0088] The warning methods use multiple means such as vision, hearing and touch. Among them, visual warning displays warning information such as red warning icons and text prompts through devices such as instrument panel display screen and head-up display (HUD); auditory warning emits sounds of different frequencies and rhythms through speakers to remind the driver of potential dangers; tactile warning uses seat vibration, steering wheel vibration and other methods to attract the driver's attention in a more intuitive way.
[0089] Furthermore, when the danger level reaches a critical level, a warning signal is sent to the terminal corresponding to the current vehicle, and the system automatically takes proactive safety measures, including automatic emergency braking, electronic stability program intervention, and automatic steering assistance, to avoid or mitigate collisions. The implementation of proactive safety measures requires real-time communication and coordination with the vehicle's electronic control system to ensure accuracy and timeliness.
[0090] As can be seen from the above, by integrating environmental perception, data fusion, hazard assessment, and early warning functions into a single system, close collaboration and information sharing between modules are achieved, improving the overall performance and efficiency of the system. Compared with traditional decentralized systems, it has a higher degree of integration and coordination. Furthermore, by combining a machine learning-based model with a hazard assessment rule base, it fully utilizes the self-learning capabilities of machine learning and the prior knowledge of the rule base, enabling more accurate hazard assessments. Compared with methods that rely solely on machine learning or a rule base, it has greater adaptability and accuracy.
[0091] It is important to note that the construction and arrangement of the present application shown in a plurality of different exemplary embodiments are merely illustrative. Although only a few embodiments are described in detail in this disclosure, it will be readily understood by those who consult this disclosure that many modifications are possible without departing substantially from the novel teachings and advantages of the subject matter described in this application. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the exemplary embodiments without departing from the scope of the present invention. Therefore, the present invention is not limited to specific embodiments, but extends to a variety of modifications that still fall within the scope of the appended claims.
[0092] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment (i.e., those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention) may not be described.
[0093] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. An integrated vehicle environment perception and warning system, characterized in that: include: The data acquisition module is used to collect the surrounding environment of the vehicle through acquisition equipment and obtain multi-source heterogeneous data; The acquisition equipment includes radar and camera; A data processing module is used to extract features from the multi-source heterogeneous data after preprocessing to obtain key feature parameters, and to perform data fusion on the key feature parameters to obtain fused data; An environmental state perception module is used to determine the current environmental state of the vehicle based on the fused data and a dynamic model of the vehicle's surrounding environment to obtain state perception information; A risk assessment module is used to obtain a risk assessment result based on the fused data and state perception information through a risk assessment model, and to obtain an adjusted risk assessment result in combination with a risk assessment rule base; The early warning module is used to issue an early warning signal according to the adjusted risk assessment result when a preset early warning threshold is exceeded, and send the early warning signal to the terminal corresponding to the current vehicle.
2. The integrated vehicle environment perception and warning system according to claim 1, characterized in that: The data processing module is further configured to: Collecting the surrounding environment of the vehicle based on the acquisition device to obtain multi-source heterogeneous data, wherein the multi-source heterogeneous data includes image data and point cloud data; The multi-source heterogeneous data is pre-processed and then feature extracted to obtain key feature parameters, which include image feature parameters f img and point cloud feature parameters f pc ; Use the following formula to convert the image feature parameter f img and point cloud feature parameters f pc As input, it is fused through a multimodal neural network to obtain fused data: f fusion =F(f img ,f pc ) Among them, F is the mapping function of the multimodal neural network, f fusion To fuse data.
3. The integrated vehicle environment perception and warning system according to claim 1, characterized in that: The data processing module is further configured to: Based on the fused data f fusion , output the target category and location information through the target detection model; Generate candidate regions through the region proposal network RPN in the target detection model, perform feature extraction and classification regression on the candidate regions, and output the target category probability P(c) and position coordinates (x, y, w, h), where c represents the target category, (x, y) is the center coordinate of the target box, and w and h are the width and height of the target box respectively; According to the category probability P(c) and the position coordinates (x, y, w, h), the target is continuously tracked and identified through feature matching.
4. The integrated vehicle environment perception and warning system according to claim 1, characterized in that: The environmental state perception module is also used for: Based on the fused data f fusion ,Through 3D reconstruction technology, a road model is constructed; Based on the fused data f fusion , combined with the category probability P(c) and the location coordinates (x, y, w, h), when the category probability P(c) is determined to be a traffic participant, a traffic participant model is constructed: Based on the road model and traffic participant model, the current vehicle environment state is inferred. The distance d between the vehicle center position and the lane boundary is calculated using the road model. The following formula is used to determine whether the current vehicle is traveling within the lane: Where D is the lane status determination result of the current vehicle, D = 1 means the current vehicle is in the lane, D = 0 means the current vehicle is outside the lane, d θ is the preset lane distance threshold; The distance m between the vehicle and surrounding traffic participants is calculated using the traffic participant model. The following formula is used to determine whether there are any obstacles in close proximity to the vehicle: Among them, M is the distance between the current vehicle and the traffic participants. M = 1 means that there is a close obstacle to the current vehicle, and M = 0 means that there is no close obstacle to the current vehicle. θ is the preset safety distance threshold; The lane state determination result D and the distance condition determination result M are integrated and expressed as a vector: S = (s1, s2, ..., s n ), where s i Represents different environmental state judgment results, n is the total number of environmental state judgment items, and S is the state perception information of the current vehicle environment.
5. The integrated vehicle environment perception and warning system according to claim 1, characterized in that: The risk assessment module is further configured to: According to the fusion data f fusion and state perception information S, extract the evaluation feature vector X=(x1,x2,...,x n ), where x i represents the i-th feature, n is the number of features; The evaluation feature vector X is used as input through the multi-layer perceptron model, and the following formula is used to obtain the category with the highest probability as the risk assessment result: Among them, y k represents the output of the kth neuron, The results of the risk assessment; Based on the Boolean condition C={C1,C2,...,C z }, where each Boolean condition corresponds to a rule, for Boolean condition C i Expressed as: Combined with the risk assessment rule base, the risk assessment results Make adjustments to obtain the adjusted risk assessment results: Among them, y final is the adjusted risk assessment result.
6. The integrated vehicle environment perception and warning system according to claim 1, characterized in that: The early warning module is also used for: The adjusted risk assessment result y final Compare with the preset warning threshold T and use the following formula to determine whether the current situation issues a warning signal: Among them, Q represents the state of the warning signal, Q=1 means that the warning signal is issued, and Q=0 means that the warning signal is not issued.