An automobile on-board multi-source data fusion processing method and system

Abstract

The application relates to the technical field of vehicle-mounted data processing, and provides a vehicle-mounted multi-source data fusion processing method and system, which comprises the following steps: according to logical association, identifying contradictory items existing in an environment understanding graph structure, and quantifying the degrees of the contradictory items to obtain contradiction degrees; obtaining the degree of conformity between a preliminary environment interpretation result and a preset rule; according to reliability scores, contradiction degrees and the degree of conformity, judging the root sources causing the contradictory items; applying weight reduction processing to the root sources to reduce the reliability scores corresponding to the root sources, and updating the confidence labels of nodes or logical associations of the environment understanding graph structure based on the reduced reliability scores to obtain the weight-reduced environment understanding graph structure, so as to complete vehicle-mounted multi-source data fusion processing; the nodes of the environment understanding graph structure are environment element nodes, which are used for representing the categories and spatial attributes of corresponding environment elements. The application has the effect of improving the decision-making capability of an intelligent driving vehicle.

Description

Technical Field

[0001] This invention relates to the technical field of vehicle data processing, and specifically to a method and system for fusion processing of multi-source vehicle data. Background Technology

[0002] In complex and ever-changing environments, intelligent driving vehicles need to integrate information from multiple sources, such as radar, cameras, and vehicle-to-everything (V2X) communication, to accurately perceive the surrounding world and make driving decisions. However, these information sources each have their own characteristics, and in adverse weather conditions or specific scenarios, the information they provide may be inconsistent or even contradictory. This poses a significant challenge to the vehicle's judgment and can potentially affect driving safety and efficiency. Especially in situations with very poor visibility, such as dense fog, and temporary traffic changes like those in construction zones, ensuring that the vehicle system can still clearly "see" the road and make reliable judgments is a critical issue.

[0003] For example, a heavy truck equipped with an advanced driver assistance system (ADAS) is traveling on a highway when it encounters sudden dense fog, severely impairing the visibility of its onboard cameras. At this time, the vehicle-to-everything (V2X) communication unit receives road condition information about a highway construction zone approximately two kilometers ahead. However, this information only provides a general location and nature of the construction zone, lacking detailed geometric information about the temporary lane layout, specific obstacle types, or temporary traffic signs within the construction area. While millimeter-wave radar can detect dense point clouds of obstacles within a construction zone, such as construction barriers, temporary roadblocks, and parked construction vehicles, due to inherent limitations, it cannot distinguish whether these obstacles are traversable or impassable, nor can it identify the boundaries of the temporary lanes formed by these obstacles.

[0004] This contradiction becomes particularly pronounced when a vehicle enters the edge of a construction zone. In extremely blurry conditions, the camera might mistakenly identify the edge of temporary fencing or piled building materials at the construction zone's edge as a new, but not actually existing, "lane line." Simultaneously, millimeter-wave radar detects a large number of dense, irregularly shaped obstacles ahead. The system's internal path planning module struggles when faced with this conflicting information. A deep inconsistency arises between the macroscopic warnings from the vehicle-to-everything (V2X) network and the vague, potentially erroneous details provided by local sensors: the V2X says, "Construction ahead, be careful," but doesn't provide specific "how to get there"; the camera "sees" a potentially incorrect "road" in the fog; the radar "feels" a bunch of "walls," but doesn't know which walls are bypassable and which are insurmountable. The system fails to form a logically consistent, unified understanding of the "currently passable area."

[0005] In complex scenarios with severely conflicting multi-source information and a lack of clear understanding, the vehicle's decision-making system cannot generate a highly reliable environmental perception description. It cannot determine whether the vehicle is still in the correct temporary lane, nor can it predict the exact nature and location of obstacles ahead. Ultimately, unable to establish a logically consistent and highly reliable environmental understanding across different information sources, the system is forced to adopt the highest level of safety strategy: immediately issuing an emergency braking command and, while ensuring safety, reducing the vehicle speed to an extremely low level, simultaneously issuing a strong takeover request to the driver and warning of significant uncertainties ahead. While this conservative strategy avoids potential accidents, it leads to unnecessary stops or extremely low speeds on highways, severely impacting traffic efficiency and posing a potential rear-end collision risk to following vehicles. It also significantly reduces user trust in the autonomous driving system and the driving experience. The system lacks a mechanism capable of assessing the reliability of each information source based on the current situation, and performing deep reasoning and judgment on the logical consistency of the entire traffic scenario in complex scenarios with conflicting multi-source information understanding and severely degraded sensor performance, thereby maintaining the vehicle's traffic capacity as much as possible while ensuring safety.

[0006] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0007] This application discloses a method and system for fusion processing of multi-source data in automotive vehicles, which aims to solve the problem that in complex and ever-changing environments, multi-source sensor information may be inconsistent or even contradictory, causing the system to be unable to form a logically consistent understanding of the environment, thereby affecting driving safety and efficiency, and even causing unnecessary safety braking.

[0008] The technical solution of this application is as follows: In a first aspect, this application discloses a method for fusing and processing multi-source data from an automotive vehicle, the method comprising: Acquire sensor information about the environment surrounding the vehicle; the sensor information includes preliminary environmental interpretation results from different information sources and the reliability scores corresponding to each preliminary environmental interpretation result; the preliminary environmental interpretation results are the environmental element identification and attribute estimation results output by different information sources, which at least include the environmental element category and the corresponding geometric attribute or occupancy attribute; Based on the preliminary environmental interpretation results and the corresponding reliability scores, an environmental understanding diagram structure is constructed; the logical relationships between environmental elements are pre-defined in the environmental understanding diagram structure; Based on logical connections, identify contradictory items in the environmental understanding graph structure and quantify the degree of contradiction to obtain the degree of contradiction. Obtain the degree of conformity between the preliminary environmental interpretation results and the preset rules; Based on the reliability score, degree of contradiction, and degree of conformity, determine the root cause of the contradiction. The root source is weighted down to reduce its corresponding reliability score. The confidence flags of the nodes or logical connections in the environmental understanding graph structure are updated based on the reduced reliability score to obtain the weighted environmental understanding graph structure, thus completing the fusion processing of multi-source data from the vehicle. The nodes in the environmental understanding graph structure are environmental element nodes, used to represent the category and spatial attributes of the corresponding environmental elements.

[0009] This technical solution effectively integrates multi-source sensor information, constructs an environmental understanding graph structure, identifies contradictory items, and determines the root cause by combining reliability scores, degree of contradiction, and degree of conformity. It then reduces the weight of each information source when information conflicts occur, thereby dynamically adjusting the weight of each source to improve the accuracy and robustness of environmental understanding. This avoids erroneous decisions caused by inconsistent information, solves the problem in existing technologies where multi-source information conflicts prevent the system from forming a logically consistent environmental understanding, and significantly enhances the perception and decision-making capabilities of intelligent driving vehicles in complex environments.

[0010] Secondly, this application also discloses an in-vehicle multi-source data fusion processing system for performing in-vehicle multi-source data fusion processing, the system comprising: The sensor information acquisition module is used to acquire sensor information about the environment around the vehicle; the sensor information includes preliminary environmental interpretation results from different information sources and the reliability scores corresponding to each preliminary environmental interpretation result. The environment structure construction module is used to construct the environment understanding graph structure based on the preliminary environment interpretation results and the corresponding reliability score; the environment understanding graph structure pre-defines the logical relationships between environmental elements; The contradiction degree quantification module is used to identify contradictory items in the environmental understanding graph structure based on logical connections, and to quantify the degree of contradiction to obtain the contradiction degree. The conformity assessment module is used to obtain the degree of conformity between the preliminary environmental interpretation results and the preset rules; The root cause determination module is used to determine the root cause of the contradictory item based on the reliability score, degree of contradiction, and degree of conformity. The root source weighting module is used to apply weighting processing to the root source to reduce the reliability score corresponding to the root source, and update the confidence flags of the nodes or logical associations of the environment understanding graph structure based on the reduced reliability score to obtain the weighted environment understanding graph structure, so as to complete the vehicle-mounted multi-source data fusion processing.

[0011] Through this technical solution, this application can achieve the acquisition of multi-source sensor information, the construction of environmental understanding graph structure, the identification and quantification of contradictory items, the acquisition of conformity degree, the judgment of root cause, and the weight reduction processing of root cause through modular design, thereby forming a complete automotive in-vehicle multi-source data fusion processing system. This system can efficiently and accurately process complex and ever-changing sensor information, effectively solve information conflict problems, improve the environmental perception and decision-making capabilities of intelligent driving vehicles, and has good engineering feasibility and practicality.

[0012] Beneficial Effects: The vehicle-mounted multi-source data fusion processing method disclosed in this application acquires sensor information about the vehicle's surrounding environment, including preliminary environmental interpretation results and their reliability scores from different information sources, and constructs an environmental understanding graph structure based on this, pre-setting logical relationships between environmental elements. On this basis, the method can identify contradictory items in the environmental understanding graph structure according to logical relationships and quantify their degree of contradiction. Simultaneously, it obtains the degree of conformity between the preliminary environmental interpretation results and preset rules. Finally, based on the reliability score, degree of contradiction, and degree of conformity, the root cause of the contradictory item is determined, and a weight reduction processing is applied to the root cause, lowering its corresponding reliability score. This updates the confidence flags of nodes or logical relationships in the environmental understanding graph structure, resulting in a weighted environmental understanding graph structure, thus completing the vehicle-mounted multi-source data fusion processing.

[0013] Through the above technical solution, this application can effectively solve the problem in the prior art where, in complex and ever-changing environments, multi-source sensor information of intelligent driving vehicles may be inconsistent or even contradictory, leading to the system's inability to form a logically consistent environmental understanding, thus affecting driving safety and efficiency, and even causing unnecessary safety braking. Specifically, this method constructs an environmental understanding graph structure to uniformly represent the preliminary environmental interpretation results and their logical connections from different sources, enabling the system to perform deep reasoning on the relationships between environmental elements. When information contradictions occur, the system can not only identify the contradictory items but also quantify them, and combine the reliability scores of each information source with the degree of conformity with preset rules to accurately determine the root cause of the contradiction. Applying a weight reduction processing to the root cause source can dynamically adjust its weight in subsequent fusion, effectively suppressing the negative impact of unreliable information on the overall environmental understanding, thereby obtaining a more accurate, robust, and logically consistent weighted environmental understanding graph structure.

[0014] For example, in scenarios involving both dense fog and construction zones, the method of this application can identify discrepancies between lane lines misidentified by the camera in fog and dense obstacles detected by radar. By combining the camera's low reliability score in dense fog, radar echo characteristic analysis results, and traffic rules in the construction zone, the system determines that camera information is the root cause of the discrepancy. Subsequently, the system reduces the weight of camera information, allowing more reliable information such as radar and vehicle-to-everything (V2X) communication to dominate environmental understanding. This avoids emergency braking caused by camera misjudgments, enabling the vehicle to continue driving safely based on more reliable information, effectively improving traffic efficiency and user experience. Therefore, the method of this application can significantly improve the accuracy of environmental perception and the reliability of decision-making for intelligent driving vehicles in complex, multi-source information conflict scenarios, overcoming the system uncertainty and safety risks caused by information inconsistency in existing technologies. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for fusion processing of multi-source data in an automotive vehicle, as described in one embodiment of the present invention. Figure 2 This is a flowchart of a method for fusion processing of multi-source data in an automotive vehicle according to another embodiment of the present invention; Figure 3 This is a system block diagram of an automotive in-vehicle multi-source data fusion processing system according to another embodiment of the present invention; Explanation of reference numerals in the attached figures: 1. Vehicle-mounted multi-source data fusion processing system; 11. Sensor information acquisition module; 12. Environmental structure construction module; 13. Conflict degree quantification module; 14. Compliance degree acquisition module; 15. Root cause judgment module; 16. Root cause weight reduction module. Detailed Implementation

[0016] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0017] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0018] This application proposes a method for fusion processing of multi-source data from automotive in-vehicle systems, combining... Figure 1 As shown, it includes: S1, acquire sensor information of the environment around the vehicle; the sensor information includes preliminary environmental interpretation results from different information sources and the reliability scores corresponding to each preliminary environmental interpretation result; the preliminary environmental interpretation results are the environmental element identification and attribute estimation results output by different information sources, which at least include the environmental element category and the corresponding geometric attribute or occupancy attribute. S2, Based on the preliminary environmental interpretation results and the corresponding reliability scores, construct the environmental understanding graph structure; the logical relationships between environmental elements are preset in the environmental understanding graph structure; S3. Based on logical connections, identify contradictory items in the environmental understanding graph structure and quantify the degree of contradiction to obtain the degree of contradiction. S4, Obtain the degree of conformity between the preliminary environmental interpretation results and the preset rules; S5. Based on the reliability score, degree of contradiction, and degree of conformity, determine the root cause of the contradiction. S6 applies a weight reduction process to the root source to lower the reliability score corresponding to the root source, and updates the confidence flags of the nodes or logical associations of the environmental understanding graph structure based on the reduced reliability score to obtain the weighted environmental understanding graph structure, thereby completing the vehicle-mounted multi-source data fusion processing; the nodes of the environmental understanding graph structure are environmental element nodes, used to represent the category and spatial attributes of the corresponding environmental elements.

[0019] To facilitate understanding of the vehicle-mounted multi-source data fusion processing method of this application, the key terms involved in the text will be explained in a unified manner.

[0020] "Sensory information" refers to data acquired through vehicle-mounted cameras, millimeter-wave radar, lidar, ultrasonic sensors, vehicle-to-everything (V2X) communication units, or other sensing carriers, used to characterize the state of the environment surrounding a vehicle. Sensory information can originate from different sensing modalities, reflecting the state characteristics of the road environment, traffic participants, static facilities, and dynamic obstacles.

[0021] "Preliminary environmental interpretation results" refer to the environmental characterization results formed after various information sources preprocess, identify, detect, or estimate the raw sensor data. Preliminary environmental interpretation results include at least the category information of environmental elements and the corresponding geometric or occupancy attributes. Category information may include vehicles, pedestrians, lane lines, obstacles, traffic signs, curbs, etc.; geometric attributes may include location, size, shape, orientation, or outline range; occupancy attributes may include whether the target area is occupied, the boundaries of passable areas, and spatial coverage status.

[0022] A "reliability score" is a quantitative value assigned to each preliminary environmental interpretation result, characterizing the degree of trustworthiness of that result under current conditions and at the current moment. The reliability score can be obtained by comprehensively considering factors such as sensor performance status, environmental interference levels, data integrity, and historical stability.

[0023] An "environmental understanding graph structure" refers to a data structure used to uniformly represent environmental elements and their interrelationships. Nodes in the environmental understanding graph structure represent environmental elements and carry their category information, spatial attributes, and corresponding confidence information. Logical relationships are used to represent the constraints or interactions between different environmental elements, such as location inclusion relationships, occlusion relationships, passage constraints, traffic rule constraints, and motion consistency relationships.

[0024] "Contradictory items" refer to conflicting information in the environmental understanding map structure that violates pre-set logical connections, traffic rule constraints, physical constraints, or common-sense knowledge. Contradictory items can manifest as incompatible interpretations of the same environmental area from different information sources, or as illogical relationships between a certain environmental element and its surrounding road structure, passable area, or other traffic participants.

[0025] "Degree of contradiction" refers to the result of quantifying the severity of contradictory items, used to characterize the degree of impact of conflicting information on the consistency of the overall understanding of the environment.

[0026] "Predefined rules" refer to a set of predefined rules used to evaluate the reasonableness of preliminary environmental interpretation results. Predefined rules may include traffic rules, road topology constraints, physical accessibility constraints, vehicle kinematics constraints, target occupancy consistency constraints, and domain common sense rules.

[0027] "Compliance level" refers to the degree of consistency between the preliminary environmental interpretation results and the preset rules, and is used to characterize whether the interpretation results meet the reasonableness requirements defined by the preset rules.

[0028] "Root cause" refers to the original source of information that leads to the contradiction. The original source of information can be a specific sensor, an information processing module, or an external data input unit.

[0029] "Deweighting" refers to a process that reduces the reliability score of the initial environmental interpretation results for information sources identified as root causes, thereby weakening the influence of such information sources in the subsequent fusion process.

[0030] "Confidence markers" refer to confidence information attached to nodes or logical connections in the context understanding graph structure, used to characterize the reliability of the corresponding nodes or logical connections.

[0031] Based on the above terminology definitions, this application provides a method for fusion processing of multi-source data in automotive vehicles. This method does not simply superimpose multi-source sensing results, but rather constructs an environment understanding graph structure to perform correlation modeling, conflict identification, root cause attribution, and dynamic weighting on the preliminary environment interpretation results output from different information sources, thereby obtaining more consistent and reliable environment perception results.

[0032] During implementation, the first step is to acquire sensor information about the vehicle's surrounding environment. This sensor information can come from various sources, such as onboard cameras, millimeter-wave radar, lidar, and vehicle-to-everything (V2X) communication units. Each information source processes the raw data based on its respective perception mechanism and outputs a preliminary environmental interpretation result. For example, cameras can output lane lines, traffic signs, vehicle and pedestrian identification results, along with corresponding location and category information; millimeter-wave radar can output target distance, speed, and orientation information; lidar can output obstacle outlines, space occupancy status, and three-dimensional structure information; and the V2X communication unit can output information about construction ahead, traffic congestion, or road events. For each preliminary environmental interpretation result, a corresponding reliability score is then obtained. The reliability score can be set based on sensor performance, current weather conditions, lighting conditions, occupancy status, historical recognition stability, or the data source's credibility level, ensuring that different information sources have a distinguishable credibility basis from the initial stage.

[0033] After obtaining preliminary environmental interpretation results and their corresponding reliability scores, an environmental understanding graph structure is constructed based on these results. During construction, each environmental element is mapped to a node in the environmental understanding graph structure, and the spatial relationships, occupancy relationships, passage relationships, traffic rule relationships, and physical constraint relationships between environmental elements are mapped to logical associations. The environmental understanding graph structure does not merely record isolated identification results of environmental elements; rather, it incorporates environmental elements and their interrelationships into a unified expression framework, enabling consistent analysis of environmental perception results under overall constraints. For example, logical associations such as "vehicle is located within the lane," "obstacle occupies a non-passable area," and "traffic signs correspond to road rule constraints" can be established, with the reliability score serving as the initial confidence marker for the corresponding node or logical association. Through the construction of the environmental understanding graph structure, environmental interpretation results from different sources are transformed from dispersed representations into a unified, associative, and verifiable representation.

[0034] After the environmental understanding map structure is established, contradictory items are identified based on pre-defined logical relationships. Contradictory items characterize inconsistencies within the environmental understanding map structure. For example, one information source interprets an area as a passable lane, while another information source interprets the same area as having a physical obstacle; or the spatial location of an environmental element is clearly inconsistent with road topology, direction of movement, or traffic rules, all of which constitute contradictory items. The identified contradictory items are further quantified to obtain the corresponding degree of contradiction. The degree of contradiction can be calculated by combining factors such as the number of conflicting elements, the type of conflict relationship, the degree of geometric deviation, the scope of conflict, and the reliability score of the relevant information sources, reflecting the strength of the corresponding contradictory item's impact on the overall consistency of environmental understanding. By introducing the degree of contradiction, subsequent root cause attribution no longer relies solely on a binary judgment based on the existence of conflict, but further considers the actual degree of impact of the conflict.

[0035] While identifying and quantifying contradictions, the degree of conformity between each preliminary environmental interpretation and the preset rules is obtained. Preset rules can include road geometry rules, traffic regulations, vehicle motion constraints, and common-sense spatial logic rules. For example, lane lines should maintain continuity with the road topology, obstacles should not appear arbitrarily in the center of a clearly passable area, and vehicle trajectories should not violate basic kinematic continuity constraints. By calculating the degree of conformity between the preliminary environmental interpretation results and the preset rules, it can be determined whether the corresponding interpretation results are reasonable at the rule level. A high degree of conformity indicates that the interpretation result not only comes from a reliable information source but also aligns with the road environment, traffic rules, and common-sense physics; a low degree of conformity indicates that even if the interpretation result originates from a sensor output, there may be identification bias, positioning bias, or misinterpretation.

[0036] After obtaining the reliability score, degree of contradiction, and degree of conformity, the root cause of the contradiction is determined based on these three factors. The root cause determination process does not rely solely on the original reliability score of a single information source, but rather jointly analyzes the initial credibility of the information source, the severity of the contradiction, and the conformity of the corresponding explanation result with preset rules. When the initial environmental explanation result output by a certain information source significantly conflicts with other high-credibility results and also shows a low degree of conformity under preset rules, this information source can be determined as the root cause of the contradiction. For example, under low visibility conditions, if the lane line recognition result output by the camera is inconsistent with the road geometry and conflicts with the obstacle occupancy result provided by radar or lidar, and the lane line result has a low degree of conformity in rule verification, then the processing link corresponding to the camera can be determined as the root cause.

[0037] Once the root cause is identified, it is weighted down to reduce the reliability score of its corresponding preliminary environmental interpretation result. The purpose of this weighting down is to weaken the continued impact of unreliable information sources on the overall fusion result, ensuring that subsequent environmental understanding processes rely more on interpretation results that conform to pre-defined rules and have a lower degree of contradiction. Weighting down does not simply delete the data output by the root cause; rather, it reduces its reliability score, decreasing its weight in the fusion process, thereby preserving necessary information redundancy while suppressing the spread of erroneous information.

[0038] After weight reduction, the confidence markers of nodes or logical connections in the environment understanding graph structure are updated based on the reduced reliability scores, resulting in a weighted environment understanding graph structure. As the confidence markers of nodes and logical connections are updated, the credibility of each environmental element and its relationships in the environment understanding graph structure is adjusted accordingly. This allows for the re-formation of a more consistent environmental representation under the new weight distribution. Through the aforementioned dynamic weight reduction and graph structure update mechanism, the system can continuously suppress the influence of low-credibility information sources and strengthen the dominant role of high-credibility information sources when there are local conflicts in multi-source perception results. Ultimately, this completes the fusion processing of multi-source automotive data and outputs more accurate, stable, and rule-compliant environment perception results.

[0039] In some implementations, when a vehicle is in a low-visibility or high-reflection environment, the initial environmental interpretation results from some information sources may be biased. For example, a camera may incorrectly identify lane lines that do not conform to the road geometry, while the obstacle and occupancy results from radar and lidar may be consistent with the road traffic logic. In this case, the system identifies the contradiction of "lane lines exist, but the corresponding area is simultaneously occupied by obstacles" through the environmental understanding graph structure. Combining the degree of contradiction and the low degree of conformity of the lane line results to the road topology rules, the system determines the camera processing link as the root cause. Subsequently, the system performs downweighting on this source and updates the confidence flags of relevant nodes and logical associations, so that the environmental understanding results change from a state dependent on conflicting information to a state dependent on highly consistent information, thereby improving the reliability of the vehicle's understanding of the surrounding environment.

[0040] Optional, combined Figure 2 As shown, based on reliability score, degree of contradiction, and degree of conformity, the steps to determine the root cause of the contradiction include: A1. Acquire image data from the image sensing device, perform dehazing and edge enhancement on the image data, extract the corresponding visual continuity cues, and calculate the visual cue authenticity score based on the visual continuity cues as a reliability score component of the visual source. A2. Acquire point cloud data from the LiDAR and perform echo characteristic analysis on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds. The occupancy credibility score of the real obstacle point cloud and multipath reflection phantom point cloud is used as a reliability score component for the point cloud source. The occupancy credibility score characterizes the reliability of point cloud occupancy evidence; a higher score indicates higher credibility of the real obstacle occupancy evidence. Point cloud occupancy evidence is the geometric evidence used to update the occupancy probability when the point cloud data is mapped to a 3D raster map. Real obstacle occupancy evidence is the portion of the point cloud occupancy evidence formed by real obstacle point clouds. A3 maps the visual cue authenticity score, occupancy credibility score, and vehicle physical size and kinematic constraints onto a 3D grid map to obtain the occupancy probability and empty probability of each grid cell. A4. In a 3D grid map, the accessibility score of each grid cell is calculated by combining the occupancy probability, the openness probability, and the vehicle's physical size and kinematic constraints. Based on the accessibility score, a topology map of the accessible area in front of the vehicle is generated. A5. Based on the topology map of the passable area, calculate the conflict intensity between the passability hypothesis corresponding to the visual continuity cue and the blocking hypothesis corresponding to the point cloud occupancy, and perform a consistency check on the conflict intensity and the degree of contradiction to obtain the conflict intensity that passes the consistency check. A6. Based on the reliability score components of the visual source, the reliability score components of the point cloud source, the conflict intensity, the degree of contradiction, and the degree of conformity of the consistency check, the root source is determined to be either the visual source or the point cloud source.

[0041] Specifically, after acquiring the image data output by the image sensing device, dehazing and edge enhancement processing are first performed on the image data. Dehazing is used to reduce the impact of severe weather such as fog and haze on image quality and improve image clarity; edge enhancement is used to strengthen the structural boundary information in the image, enabling visual information with continuous geometric features, such as road boundaries, lane lines, and obstacle outlines, to be extracted more stably. Based on the processed image data, visual continuity cues reflecting the continuity of road boundaries, lane line extension, and obstacle outlines are extracted. The visual continuity cues are then combined with their stability in the temporal dimension, consistency in spatial geometry, and conformity to traffic rules to calculate a visual cue authenticity score. The visual cue authenticity score characterizes the credibility of the environmental interpretation results corresponding to the visual source. A higher score indicates that the visual continuity cues more accurately reflect the road environment state, and the visual cue authenticity score is used as a reliability score component of the visual source.

[0042] After acquiring the point cloud data output by the lidar, echo characteristic analysis is performed on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds. Multipath reflection phantom point clouds refer to point clouds where abnormal echoes formed after the laser beam is reflected by multiple reflective surfaces are misidentified as corresponding to actual obstacles during the calculation process; real obstacle point clouds are point clouds formed by the echoes generated by actual obstacles to the laser beam. During echo characteristic analysis, echo intensity, phase characteristics, Doppler frequency shift characteristics, or other echo response parameters can be used to distinguish different point cloud sources. Based on the identified real obstacle point clouds and multipath reflection phantom point clouds, an occupancy confidence score is calculated. The occupancy confidence score is used to characterize the reliability of point cloud occupancy evidence; the higher the score, the more reliable the obstacle occupancy judgment provided by the point cloud data. Among them, point cloud occupancy evidence refers to the geometric occupancy information used to update the occupancy probability of the corresponding grid cell after the point cloud data is mapped to a 3D raster map; real obstacle occupancy evidence refers to the part of the point cloud occupancy evidence formed by real obstacle point clouds. Occupation credibility score is used as a reliability score component for point cloud sources.

[0043] After obtaining the visual cue authenticity score and occupancy credibility score, these scores, along with the vehicle's physical dimensions and kinematic constraints under its current motion state, are uniformly mapped onto a 3D grid map. During the mapping process, the occupancy probability and empty probability are calculated for each grid cell in the 3D grid map, combining the indicative role of visual continuity cues on the continuity of spatial regions, the constraint role of point cloud occupancy evidence on the spatial occupancy state, and the limiting role of vehicle size and kinematic constraints on the reachable area. The occupancy probability characterizes the likelihood that the corresponding grid cell is occupied by an obstacle, while the empty probability characterizes the likelihood that the corresponding grid cell is an empty area.

[0044] Based on this, and considering the occupancy probability and openness probability of each grid cell, as well as the vehicle's physical dimensions and kinematic constraints, a accessibility score is calculated for each grid cell in the 3D grid map. The accessibility score characterizes the feasibility and safety of a vehicle passing through the corresponding grid cell. Based on the accessibility scores of each grid cell, a topology map of the accessible area in front of the vehicle is generated. This topology map characterizes the continuous, connectable potential driving areas in front of the vehicle that meet the vehicle's passage conditions, providing a spatial basis for subsequent conflict analysis between the accessibility and obstruction assumptions.

[0045] After generating a topology map of passable areas, the conflict intensity between the passability hypothesis indicated by visual continuity cues and the blocking hypothesis indicated by point cloud occupancy is calculated based on the passability map. The passability hypothesis refers to the continuous passability of a target area supported by visual continuity cues, while the blocking hypothesis refers to the existence of an obstacle blocking the target area supported by point cloud occupancy evidence. A conflict arises when visual continuity cues indicate that an area is passable, while point cloud occupancy evidence simultaneously indicates that the area is occupied. The conflict intensity is used to quantify the severity of this inconsistency in terms of spatial extent, degree of occupancy, and impact on passage. Furthermore, a consistency check is performed between the conflict intensity and the identified and quantified degree of contradiction in the environmental understanding map structure to determine whether the conflict intensity corresponds to the contradictory item in nature and degree, and to obtain the conflict intensity that passes the consistency check.

[0046] After obtaining the reliability score components of the visual source, the point cloud source, and the conflict intensity that passed the consistency check, the root cause of the conflict is determined by combining the degree of contradiction and the degree of conformity between the preliminary environmental interpretation results and the preset rules. This determination process is not based solely on the local score results of a single source, but rather on a comprehensive attribution that combines the authenticity of the visual continuity cues, the credibility of the point cloud occupancy evidence, the conflict intensity between the passability hypothesis and the blocking hypothesis, the severity of the conflict, and the conformity between the interpretation results and traffic rules and physical constraints. Thus, it is possible to identify whether the root cause of the conflict is the visual source, the point cloud source, or, under specific circumstances, a combination of both.

[0047] In some preferred embodiments, when a vehicle travels in a foggy road environment, the image data acquired by the image sensing device experiences a decrease in clarity due to the fog, leading to uncertainty in the lane line boundaries and obstacle outline boundaries corresponding to visual continuity cues, resulting in a lower visual cue authenticity score. Simultaneously, the lidar detects a cluster of point clouds in a certain area, but after echo characteristic analysis, it identifies this cluster of point clouds as having multipath reflection characteristics and classifies it as a multipath reflection phantom point cloud, causing the occupancy confidence score for that area to also be low. At this point, the environmental understanding map structure may contain contradictory statements: a certain area is interpreted as passable by the visual source, while being interpreted as occupied by the point cloud source.

[0048] To address this contradiction, the system first calculates the visual cue authenticity score and occupancy credibility score, and then maps the scores, along with the vehicle's physical dimensions and kinematic constraints, onto a 3D grid map. This yields the occupancy probability, openness probability, and corresponding passability score for each grid cell, generating a topology map of the passable area in front of the vehicle. Subsequently, based on the topology map, the system calculates the conflict strength between the passability hypothesis supported by the visual continuity cue and the blocking hypothesis supported by the point cloud occupancy, and performs a consistency check between the conflict strength and the degree of contradiction in the environmental understanding map structure. Finally, the system comprehensively judges the root cause by combining the visual cue authenticity score, the point cloud occupancy credibility score, the conflict strength, degree of contradiction, and degree of conformity that passed the consistency check. When both the visual source and the point cloud source exhibit low credibility and significantly conflict with each other in the same area, the contradiction can be determined to be caused by both sources. When the authenticity score or occupancy credibility score of one source is significantly lower than that of the other source, and its conformity with preset rules is also lower, that source can be identified as the primary root cause. Based on this judgment, the corresponding root causes are downweighted, and the relevant confidence markers in the environmental understanding graph structure are updated accordingly to improve the stability and accuracy of subsequent multi-source fusion results.

[0049] Optionally, the steps of performing dehazing and edge enhancement on the image data, extracting the corresponding visual continuity cues, and calculating the visual cue authenticity score based on the visual continuity cues, with the reliability score component of the visual source including: Image data is dehazed and edge-enhanced to extract initial visual continuity cues; Obtain candidate regions with brightness variations and geometric shapes from image data; Based on the geometric features, spatial distribution, and similarity to preset traffic signs or construction markings, the candidate regions are classified to obtain classification results. Based on the classification results, temporary pavement marking areas were identified, and the geometric parameters of the temporary pavement marking areas were extracted. Based on geometric parameters, the initial visual continuity cues are corrected to obtain the corrected visual continuity cues. Based on the modified visual continuity cues in terms of temporal continuity, geometric consistency, and consistency with traffic rule constraints, a visual cue authenticity score is calculated and output as a reliability score component of the visual source.

[0050] Specifically, image data undergoes dehazing and edge enhancement processing to eliminate blurring caused by weather conditions (such as fog and haze) and highlight edge information, thereby initially extracting visual continuity cues such as road boundaries and lane lines. However, these initial cues may contain noise or misjudgments caused by non-standard or temporary markings. To improve the accuracy of the cues, it is necessary to acquire candidate regions with brightness variations and geometric shapes in the image data. These candidate regions typically correspond to various markings or objects on the road surface.

[0051] Furthermore, these candidate regions are classified based on their geometric features (e.g., length, width, curvature), spatial distribution (e.g., relative position to other markings), and similarity to preset traffic signs or construction markings (e.g., color, shape, pattern). Classification distinguishes different types of visual elements such as regular lane lines, traffic signs, and construction zone markings. Among these, temporary pavement marking areas can be identified based on the classification results. Temporary pavement marking areas refer to those that are not part of regular road infrastructure but provide temporary guidance or warnings for vehicle traffic, such as temporary lane lines, traffic cones, and temporary signs in construction zones. After identifying these areas, their geometric parameters, such as their position, size, and orientation in the image, need to be extracted.

[0052] Therefore, based on the extracted geometric parameters, the initial visual continuity cues can be corrected. For example, if the initial cues conflict with or are inconsistent with temporary pavement markings, the initial cues can be adjusted or replaced based on the geometric parameters of the temporary markings, thus obtaining corrected visual continuity cues. Finally, based on the corrected visual continuity cues' temporal continuity (i.e., stability across consecutive frames), geometric consistency (i.e., conformity to the geometric laws of the physical world), and consistency with traffic rule constraints (i.e., compliance with road traffic regulations), a visual cue authenticity score is calculated and output. This score, as a reliability component of the visual source, can more accurately reflect the reliability of visual information.

[0053] Based on reliability scores, degree of inconsistency, and degree of conformity, the steps to determine the root cause of the inconsistency include: Obtain the vehicle's physical dimensions and kinematic constraints under its current motion state, and project the kinematic constraints onto a 3D grid map in front of the vehicle to define the vehicle's spatial envelope; Under the constraints of spatial envelope, visual continuity cues and point cloud data are acquired, and these cues and data are used as geometric evidence to indicate whether a grid cell is empty or occupied. Echo characteristic analysis is performed on point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds. A first occupancy evidence weight is assigned to real obstacle point clouds, and a second occupancy evidence weight is assigned to multipath reflection phantom point clouds. The first occupancy evidence weight is higher than the second occupancy evidence weight. Visual continuity cues, real obstacle point clouds, and multipath reflection phantom point clouds are mapped onto a 3D grid map, and the occupancy probability and empty probability of each grid cell are updated by weighting based on the first occupancy evidence weight and the second occupancy evidence weight. Based on the weighted updated occupancy and vacancy probabilities, under the constraints of vehicle accessibility and traffic rules, connectivity exploration is performed on the 3D raster map to generate a topology map of continuous, unobstructed passable areas in front of the vehicle; wherein, the connectivity exploration treats raster cells with occupancy probabilities exceeding a threshold as impassable cells. Based on the topology map of passable areas, the degree of conformity between visual continuity cues and passable areas and the degree of conformity between point cloud data and impassable areas are calculated respectively. Combined with the reliability score and the degree of contradiction, it is determined whether the root cause is caused by visual continuity cues or by point cloud data.

[0054] Specifically, the system acquires the vehicle's physical dimensions and kinematic constraints under its current motion state, and projects these constraints onto a 3D grid map in front of the vehicle to define its spatial envelope. The vehicle's physical dimensions refer to its basic geometric parameters such as length, width, height, and wheelbase, while the kinematic constraints under its current motion state include the vehicle's maximum steering angle, maximum acceleration, maximum deceleration, and instantaneous lateral and longitudinal speed limits. Projecting these constraints onto the 3D grid map in front of the vehicle dynamically defines the spatial range that the vehicle can physically reach or occupy in the current moment and in the short future—that is, the vehicle's spatial envelope. The purpose is to provide a physically feasible constraint boundary for subsequent environmental perception and decision-making, excluding environmental interpretations that do not conform to the vehicle's kinematic characteristics.

[0055] Under the constraints of spatial envelope, visual continuity cues and point cloud data are acquired and used as geometric evidence to indicate whether a grid cell is vacant or occupied. Visual continuity cues typically refer to visual information with continuous geometric features, such as road markings, lane boundaries, and curbs, acquired from image sensors. These cues usually indicate passable vacant areas. Point cloud data typically refers to 3D point clouds acquired from LiDAR, which can indicate the presence of obstacles, thus representing the occupancy status of the grid cell. Using this information as geometric evidence can provide direct or indirect indications of the occupancy or vacancy status of each cell in a 3D grid map.

[0056] Furthermore, echo characteristic analysis is performed on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds. A first occupancy evidence weight is assigned to real obstacle point clouds, and a second occupancy evidence weight is assigned to multipath reflection phantom point clouds. The first occupancy evidence weight is higher than the second occupancy evidence weight. Echo characteristic analysis can be understood as a deep analysis of the physical characteristics of the lidar echo signal, such as intensity, phase, and Doppler shift, to identify echoes generated by real physical obstacles and false echoes (i.e., phantom point clouds) generated by multipath reflection (e.g., a laser beam reflecting between multiple surfaces before returning to the sensor). Real obstacle point clouds, due to their directness, generally have higher reliability and are therefore assigned a higher first occupancy evidence weight. Multipath reflection phantom point clouds, due to their falseness, are assigned a lower second occupancy evidence weight to reduce their negative impact on environmental understanding.

[0057] Subsequently, visual continuity cues, real obstacle point clouds, and multipath reflection phantom point clouds are mapped onto a 3D raster map. The occupancy and empty probability of each raster cell are then updated with weighted averages based on a first occupancy evidence weight and a second occupancy evidence weight. Specifically, visual continuity cues are typically used to update the empty probability of raster cells, while real obstacle point clouds and multipath reflection phantom point clouds are used to update the occupancy probability. During the update process, evidence from different sources is weighted according to its assigned weights, ensuring that highly reliable evidence has a greater impact on the probability update, thus obtaining more accurate and robust estimates of raster cell occupancy and empty state.

[0058] Building upon this foundation, and based on the weighted updated occupancy and vacancy probabilities, a connectivity exploration algorithm is performed on the 3D raster map under constraints of vehicle accessibility and traffic rules. This generates a topology map of continuous, unobstructed, drivable areas ahead of vehicles. The connectivity exploration algorithm is a graph search algorithm that treats raster cells with occupancy probabilities exceeding a preset threshold as impassable and searches for continuous, unobstructed paths among the remaining drivable cells. Vehicle accessibility constraints ensure that the generated paths are physically reachable by vehicles, while traffic rule constraints ensure that the paths comply with road regulations (e.g., not entering pedestrian crossings or driving against traffic). This generates a topology map representing a safe, drivable area ahead of vehicles.

[0059] Finally, based on the topology map of traversable areas, the degree of conformity between visual continuity cues and traversable areas, and the degree of conformity between point cloud data and impassable areas, are calculated separately. Combined with reliability scores and the degree of inconsistency, it is determined whether the root cause is due to visual continuity cues or point cloud data. Specifically, if the path indicated by the visual continuity cues is highly consistent with the topology map of traversable areas, while the degree of conformity between the point cloud data (especially the portion containing many phantom points) and impassable areas is low, then the root cause of the inconsistency may be determined to be the point cloud data. Conversely, if the point cloud data is highly consistent with impassable areas, while the degree of conformity between visual continuity cues and traversable areas is low, then the root cause may be determined to be the visual continuity cues. This method of judgment, combining multiple aspects of information, can more accurately locate the root cause of the inconsistency.

[0060] Optionally, echo characteristic analysis is performed on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds, and a first occupancy evidence weight is assigned to the real obstacle point clouds, and a second occupancy evidence weight is assigned to the multipath reflection phantom point clouds; the step where the first occupancy evidence weight is higher than the second occupancy evidence weight includes: Obtain environmental parameters and statistical characteristics of radar echo signals; Detect whether the statistical characteristics of environmental parameters or radar echo signals have changed; When changes in environmental parameters or the statistical characteristics of radar echo signals are detected, point cloud clusters with geometric consistency and spatial stability are identified in the point cloud data. Echo characteristics of point cloud clusters are extracted and correlation analysis is performed between echo characteristics and environmental parameters to obtain correlation analysis results; Based on the correlation analysis results, echo discrimination parameters are generated; wherein, the echo discrimination parameters are a set of discrimination parameters used to distinguish between real obstacle echoes and multipath reflection echoes, including at least a phase consistency threshold, an echo intensity statistical threshold and / or a Doppler frequency shift distribution discrimination threshold, and are used as the discrimination basis for echo characteristic analysis to output the distinction results between real obstacle point clouds and multipath reflection phantom point clouds; Echo characteristic analysis is performed on point cloud data using echo discrimination parameters to output the distinction between real obstacle point clouds and phantom point clouds; Based on the differentiation results, a first occupancy evidence weight is assigned to the real obstacle point cloud, and a second occupancy evidence weight is assigned to the multipath reflection phantom point cloud; the first occupancy evidence weight is higher than the second occupancy evidence weight.

[0061] Specifically, before performing echo characteristic analysis, it is necessary to first obtain environmental parameters and the statistical characteristics of the radar echo signal. Environmental parameters can be understood as external conditions affecting radar signal propagation and reflection, such as temperature, humidity, rainfall, and road conditions. These parameters can be obtained through an onboard environmental sensor array. The statistical characteristics of the radar echo signal refer to the quantitative indicators obtained after processing the received radar signal, such as average intensity, standard deviation, phase consistency, and Doppler shift distribution. These characteristics reflect the inherent physical properties of the echo signal.

[0062] Furthermore, the system continuously monitors for changes in environmental parameters or the statistical characteristics of radar echo signals. This change detection aims to adapt to dynamically changing external environments and ensure the real-time performance and accuracy of echo discrimination parameters. For example, when environmental parameters (such as increased rainfall) or the statistical characteristics of radar echo signals (such as a significant decrease in average intensity) change significantly, it indicates that the current environment may be having a new impact on the radar signal, requiring a reassessment of the discrimination criteria.

[0063] When changes in environmental parameters or the statistical characteristics of radar echo signals are detected, the system identifies point cloud clusters with geometric consistency and spatial stability within the point cloud data. A point cloud cluster is a collection of spatially close point clouds with similar geometric features; these typically correspond to real physical obstacles or specific reflectors. Geometric consistency refers to the consistency of the spatial arrangement and shape characteristics of the point clouds within a cluster, while spatial stability refers to the ability of the cluster to maintain a relatively stable position and shape across consecutive frames. Identifying these point cloud clusters helps focus the analysis on meaningful regions.

[0064] Subsequently, the echo characteristics of the point cloud clusters are extracted and correlated with environmental parameters to obtain correlation analysis results. Echo characteristics include, but are not limited to, echo intensity, phase, and Doppler shift. By correlating these echo characteristics with current environmental parameters, it is possible to understand the patterns exhibited by the echo signals of different types of obstacles (real obstacles or multipath reflection phantoms) under specific environmental conditions. For example, in rainy weather, raindrops produce specific echo intensity and Doppler shift distributions, while the echoes of real obstacles may be attenuated but their geometric characteristics remain unchanged.

[0065] Based on the correlation analysis results, the system generates echo discrimination parameters. These parameters are a set used to distinguish between real obstacle echoes and multipath reflection echoes, and include at least a phase consistency threshold, an echo intensity statistics threshold, and / or a Doppler frequency shift distribution discrimination threshold. These thresholds are dynamically adjusted and optimized based on the correlation analysis results and serve as the basis for echo characteristic analysis. For example, multipath reflections typically lead to reduced phase consistency, abnormal echo intensity distribution, or multiple peaks in the Doppler frequency shift distribution. By setting appropriate thresholds, these characteristics can be effectively distinguished from the echo characteristics of real obstacles.

[0066] After generating echo discrimination parameters, echo characteristic analysis is performed on the point cloud data using these parameters to output the distinction between real obstacle point clouds and phantom point clouds. This step is the core classification process; each point cloud is compared with its echo characteristics and discrimination parameters to be classified as either a real obstacle point cloud or a multipath reflection phantom point cloud.

[0067] Finally, based on the differentiation results, a first occupancy evidence weight is assigned to the real obstacle point cloud, and a second occupancy evidence weight is assigned to the multipath reflection phantom point cloud. The first occupancy evidence weight is higher than the second. This means the system assigns a higher confidence level to point clouds identified as real obstacles and a lower confidence level to point clouds identified as phantoms. This differentiated weight allocation ensures that in subsequent occupancy probability updates, real obstacles have a greater impact on the environmental understanding map structure, while interference from phantom point clouds is effectively suppressed.

[0068] Optionally, the steps for obtaining environmental parameters and statistical characteristics of radar echo signals include: Acquire environmental parameters provided by the onboard environmental sensor array and obtain radar echo signals; Time series analysis of environmental parameters is performed to identify their changing trends and fluctuations. Statistical characteristics are calculated from radar echo signals to obtain statistical properties; time series analysis is performed on these statistical properties to identify the changing trends and fluctuation amplitudes of the radar echo signals; the statistical properties include average intensity, standard deviation, phase consistency, and Doppler frequency shift distribution. Based on the changing trends and fluctuation ranges of environmental parameters, the environmental parameters are calibrated by combining the vehicle's geographical location and preset meteorological data to obtain the calibrated environmental parameters; Based on the changing trend and fluctuation amplitude of the radar echo signal, outlier detection and filtering are performed on the statistical characteristics to suppress false fluctuations caused by sensor noise, transient interference or abnormal data transmission, and the filtered radar echo signal is obtained.

[0069] Specifically, acquiring environmental parameters provided by the vehicle's environmental sensor array refers to the real-time collection of environmental data surrounding the vehicle through various environmental sensors mounted on the vehicle, such as temperature sensors, humidity sensors, light sensors, and barometric pressure sensors. Simultaneously, acquiring radar echo signals refers to receiving and processing electromagnetic wave signals reflected from targets through the vehicle's radar system.

[0070] Time series analysis of environmental parameters aims to identify their normal trends and fluctuations by analyzing their patterns of change over time, such as trends, periodicity, and seasonality. This helps distinguish between normal environmental changes and anomalous disturbances.

[0071] Statistical analysis of radar echo signals involves mathematically processing the received radar echo signals to extract key statistical quantities. These statistical characteristics can be understood as describing the features of the radar echo signal in different dimensions. For example, the average intensity reflects the overall energy level of the echo, the standard deviation reflects the dispersion of the echo intensity, phase consistency characterizes the phase stability of the echo signal, and the Doppler shift distribution reveals the radial velocity information of the target relative to the radar. These statistical characteristics are crucial for distinguishing between real obstacle echoes and multipath reflection echoes.

[0072] Furthermore, time-series analysis of the statistical characteristics aims to identify the changing trends and fluctuations in the statistical characteristics of radar echo signals. Long-term monitoring and analysis of these statistical characteristics can establish normal behavioral patterns, making it easier to detect anomalies.

[0073] Based on the changing trends and fluctuations of environmental parameters, and combined with the vehicle's geographical location and preset meteorological data, the environmental parameters are calibrated to improve their accuracy and reliability. For example, the measured values ​​of environmental parameters may deviate under different geographical locations and meteorological conditions; calibration can eliminate or reduce these deviations.

[0074] In practical applications, based on the changing trends and fluctuation amplitudes of radar echo signals, outlier detection and filtering are performed on the statistical characteristics to identify and eliminate spurious fluctuations caused by sensor noise, transient interference, or abnormal data transmission. Outlier detection can identify data points that deviate from the normal range, while filtering can smooth the data and suppress high-frequency noise, thereby obtaining more stable and realistic statistical characteristics of radar echo signals.

[0075] Optionally, the step of calculating the statistical characteristics of the radar echo signal to obtain the statistical characteristics may include the following operations: By performing joint time-domain and frequency-domain analysis on radar echo signals, echo components with different center frequencies and bandwidths were identified. Waveform features are extracted for each echo component to obtain instantaneous amplitude, instantaneous phase, and instantaneous frequency; Based on the instantaneous amplitude, instantaneous phase, and instantaneous frequency, the average intensity, standard deviation, phase consistency, and Doppler frequency shift distribution of the corresponding echo components are calculated and output as statistical characteristics.

[0076] The joint time-domain and frequency-domain analysis of radar echo signals aims to comprehensively analyze the characteristics of the echo signals from both time and frequency dimensions. This joint analysis effectively identifies echo components with different center frequencies and bandwidths generated by different targets or reflection paths. For example, methods such as Short-Time Fourier Transform (STFT), wavelet transform, or Hilbert-Huang transform can be used to convert the radar echo signal from a one-dimensional time-domain signal into a two-dimensional time-frequency representation, thus clearly showing the frequency components and energy distribution of the signal at different times.

[0077] Furthermore, after identifying each echo component, waveform feature extraction is required for each component. Waveform feature extraction refers to obtaining the inherent parameters that characterize the signal properties from the signal waveform of the echo component. Specifically, instantaneous amplitude, instantaneous phase, and instantaneous frequency can be extracted. Instantaneous amplitude reflects the signal intensity at a certain moment, instantaneous phase describes the phase change of the signal, and instantaneous frequency reveals how the signal frequency changes over time. These instantaneous features are key to understanding the physical characteristics of the echo signal.

[0078] Therefore, based on the extracted instantaneous amplitude, instantaneous phase, and instantaneous frequency, the average intensity, standard deviation, phase consistency, and Doppler shift distribution of the corresponding echo components can be calculated and output as statistical characteristics of the radar echo signal. The average intensity reflects the overall energy level of the echo signal; the standard deviation measures the fluctuation of the signal intensity; phase consistency is used to assess the stability or coherence of the signal phase, which is crucial for distinguishing between real target echoes and multipath reflection echoes; and the Doppler shift distribution reveals the radial velocity information of the target relative to the radar and its distribution characteristics. These statistical characteristics provide a quantitative basis for subsequent echo discrimination and source determination.

[0079] Optionally, the steps for calculating the statistical characteristics of the radar echo signal and obtaining the statistical characteristics include: By performing joint time-domain and frequency-domain analysis on radar echo signals, echo components with different center frequencies and bandwidths were identified. Multiple echo components with similar time-frequency characteristics and spatial consistency are clustered into echo clusters; echo clusters are used to characterize the same obstacle or the same reflector. The average intensity, standard deviation, phase consistency, and Doppler frequency shift distribution of the echo cluster are output as statistical characteristics of the echo cluster by assigning weights based on the signal-to-noise ratio, phase consistency, and time continuity of the echo components. The statistical characteristics of echo clusters are used as the statistical characteristics of radar echo signals.

[0080] Specifically, after performing joint time-domain and frequency-domain analysis on radar echo signals, multiple echo components with different center frequencies and bandwidths can be identified. These echo components may originate from different reflection sources, or from the same reflection source but generated due to multipath effects or complex object geometry. To more accurately characterize actual physical obstacles, this application clusters multiple echo components with similar time-frequency characteristics and spatial consistency into echo clusters. "Similar time-frequency characteristics" refers to the similarity in the time and frequency domain performance of these echo components (e.g., center frequency, bandwidth, duration), indicating that they may originate from the same physical process or object; "spatial consistency" refers to the proximity of these echo components in spatial location, further supporting the hypothesis that they originate from the same obstacle or reflector. Through this clustering, an echo cluster can more comprehensively and robustly characterize a single obstacle or reflector.

[0081] Furthermore, in calculating the statistical characteristics of echo clusters, this application assigns weights based on the signal-to-noise ratio (SNR), phase consistency, and temporal continuity of the echo components. The SNR measures the signal quality of the echo components; components with high SNRs are generally more reliable. Phase consistency reflects the smoothness of phase changes within the echo component; high phase consistency usually indicates a more stable reflection source. Temporal continuity represents the stability of the echo components across consecutive time frames; persistent echo components are generally more reliable. By assigning weights to each echo component, the contribution of high-quality, high-stability echo components to the overall statistical characteristics of the echo cluster can be highlighted, while the influence of low-quality or unstable components can be suppressed. Therefore, the average intensity, standard deviation, phase consistency, and Doppler shift distribution of the output echo cluster can more accurately reflect the true physical characteristics of the obstacles or reflectors they represent. Finally, using the statistical characteristics of these echo clusters as the statistical characteristics of the radar echo signal provides a more reliable basis for distinguishing between real obstacle point clouds and multipath reflection phantom point clouds.

[0082] Optionally, the steps of performing joint time-domain and frequency-domain analysis on the radar echo signal to identify echo components with different center frequencies and bandwidths include: Perform a short-time Fourier transform on the radar echo signal to obtain the time-frequency distribution map; Identify different energy accumulation centers on the time-frequency distribution map to obtain candidate regions for the extended range corresponding to different energy accumulation centers; Boundary extraction and shape analysis are performed on the candidate regions to distinguish between echo component regions generated by a single obstacle and echo component regions generated by the superposition of multiple obstacles, and the echo component region discrimination results are obtained. Adaptive time-frequency window adjustment is performed on the echo component region generated by the superposition of multiple obstacles to separate the independent echo components inside; Based on the echo component region differentiation results and independent echo components, echo components with different center frequencies and bandwidths are identified and output.

[0083] Specifically, performing a short-time Fourier transform on the radar echo signal aims to convert the radar echo signal from the time domain to the time-frequency domain, thereby obtaining a time-frequency distribution map. This time-frequency distribution map can intuitively show the distribution of signal energy at different times and frequencies, providing a foundation for subsequent echo component identification.

[0084] Furthermore, by identifying different energy concentration centers on the time-frequency distribution map, the main components in the radar echo signal can be preliminarily located. These energy concentration centers typically correspond to echo signals generated by obstacles, and their extended range defines candidate regions for potential echo components.

[0085] Subsequently, boundary extraction and shape analysis are performed on the candidate regions. Boundary extraction aims to accurately delineate the outline of each candidate region, while shape analysis determines its origin by evaluating the geometric characteristics of these regions, such as area, aspect ratio, or compactness. A single obstacle typically produces echo component regions with regular shapes and concentrated energy, while multiple obstacles stacked together may result in echo component regions with irregular shapes and complex energy distributions. Thus, echo component region differentiation results can be obtained, classifying candidate regions into single-obstacle or multi-obstacle stacked types.

[0086] For regions identified as echo component areas generated by multiple obstacle superpositions, adaptive time-frequency window adjustment is required because these regions may contain multiple independent radar echo components. This adjustment process dynamically adjusts the size and shape of the time-frequency analysis window function based on the local characteristics of the region, such as energy distribution or frequency change rate, thereby more effectively separating overlapping independent echo components.

[0087] Ultimately, based on the results of echo component region differentiation and the independent echo components separated from the multi-obstacle superposition region, each radar echo component with a specific center frequency and bandwidth can be accurately identified and output. The center frequency represents the main frequency component of the echo, while the bandwidth reflects the frequency range or duration of the echo.

[0088] This application also discloses a vehicle-mounted multi-source data fusion processing system for performing vehicle-mounted multi-source data fusion processing. The vehicle-mounted multi-source data fusion processing system 1 includes: The sensor information acquisition module 11 is used to acquire sensor information of the environment around the vehicle; the sensor information includes preliminary environmental interpretation results from different information sources and the reliability score corresponding to each preliminary environmental interpretation result. The environment structure construction module 12 is used to construct an environment understanding graph structure based on the preliminary environment interpretation results and the corresponding reliability score; the environment understanding graph structure pre-defines the logical relationships between environmental elements; The contradiction degree quantification module 13 is used to identify contradictory items in the environmental understanding graph structure based on logical connections, and to quantify the degree of contradiction of the contradictory items to obtain the degree of contradiction. The conformity assessment module 14 is used to acquire the conformity between the preliminary environmental interpretation results and the preset rules; The root cause determination module 15 is used to determine the root cause of the contradictory item based on the reliability score, degree of contradiction, and degree of conformity. The root source weight reduction module 16 is used to apply weight reduction processing to the root source to reduce the reliability score corresponding to the root source, and update the confidence flags of the nodes or logical associations of the environment understanding graph structure based on the reduced reliability score to obtain the weighted environment understanding graph structure, so as to complete the vehicle-mounted multi-source data fusion processing.

[0089] Specifically, the sensor information acquisition module can be configured to integrate interface circuits and a data preprocessing unit for various vehicle-mounted sensors (such as cameras, millimeter-wave radar, and lidar). This module receives raw sensor data, converts it into preliminary environmental interpretation results, and calculates corresponding reliability scores. In a preferred embodiment, the sensor information acquisition module may include one or more dedicated processors programmed to perform data acquisition, format conversion, preliminary feature extraction, and reliability assessment algorithms. For example, the module may employ a hardware accelerator to process high-bandwidth image or point cloud data to ensure real-time performance.

[0090] The environment structure construction module can be implemented as a software component running on the vehicle's central processing unit or domain controller. This module is responsible for receiving preliminary environmental interpretation results and reliability scores from the sensor information acquisition module, and dynamically constructing or updating the environment understanding graph structure according to preset logical association rules for environmental elements. Its implementation can be based on graph database technology or in-memory graph data structures to efficiently store and manage environmental element nodes and their logical associations.

[0091] The contradiction quantification module and the compliance degree acquisition module can be integrated as part of the environmental understanding graph structure processing subsystem. The contradiction quantification module is designed to traverse the environmental understanding graph structure, identify contradictory items that violate predefined logical relationships, and calculate the degree of contradiction based on a predefined quantification model (e.g., based on the number of conflicting elements, geometric distance, or semantic conflict type). The compliance degree acquisition module is configured to compare the initial environmental interpretation results with predefined rules stored in the system (e.g., high-precision map data, traffic regulation library, vehicle kinematics model) to calculate their compliance degree. These two modules can run in parallel to improve processing efficiency.

[0092] The root cause determination module can be implemented as a decision reasoning engine. This module receives the degree of contradiction from the contradiction quantification module, the degree of conformity from the conformity acquisition module, and reliability scores from each information source as input. By applying pre-defined decision logic, Bayesian networks, fuzzy reasoning, or machine learning models, this module can intelligently analyze these inputs to determine the specific root cause of the contradiction. For example, this module can perform reasoning based on a rule base or identify sources of anomalous information using a trained classifier.

[0093] The root source weighting module is responsible for applying weight reduction processing to the corresponding root sources based on the output of the root source judgment module. This can be achieved by adjusting the weight parameters of the source in subsequent fusion algorithms or by directly reducing its output reliability score. Based on the reduced reliability score, this module updates the confidence flags of affected nodes or logical connections in the environment understanding graph structure, thereby obtaining the weighted environment understanding graph structure. The implementation of this module may include a weight management subsystem for dynamically adjusting and storing the weight configurations of each information source.

[0094] The aforementioned modules can be integrated into a unified intelligent driving domain controller, communicating and cooperating via a high-speed bus. Together, they form a robust and adaptive multi-source data fusion processing framework, effectively addressing complex and ever-changing environmental challenges.

[0095] Traditional intelligent driving systems often struggle to effectively identify and resolve conflicts arising from multiple sources, especially in adverse weather or complex traffic scenarios, leading to unreliable or erroneous decisions. The core innovation of the automotive in-vehicle multi-source data fusion processing system proposed in this application lies in its modular system architecture, which enables dynamic evaluation of the reliability of multi-source sensor information and intelligent tracing and correction of conflicting information. Compared to existing systems that simply fuse sensor data or employ fixed-weight strategies, this system constructs and dynamically updates the environmental understanding graph structure. A root cause identification module intelligently identifies the root causes of conflicts, and a root cause weight reduction module applies weight reduction processing to unreliable information sources. This mechanism allows the system to perform deeper logical reasoning and judgment even under high information uncertainty, generating a logically consistent and highly reliable environmental perception description. This significantly improves the decision-making ability and driving safety of intelligent driving systems in adverse environments and complex traffic scenarios, effectively avoiding unnecessary emergency braking or low-speed driving, and improving traffic efficiency and user experience.

[0096] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for fusion processing of multi-source data in automotive vehicles, characterized in that, include: Acquire sensor information about the environment surrounding the vehicle; the sensor information includes preliminary environmental interpretation results from different information sources and reliability scores corresponding to each preliminary environmental interpretation result; The preliminary environmental interpretation results are environmental element identification and attribute estimation results output from different information sources, which at least include environmental element categories and corresponding geometric or occupancy attributes; Based on the preliminary environmental interpretation results and the corresponding reliability scores, an environmental understanding graph structure is constructed; the logical relationships between predefined environmental elements are included in the environmental understanding graph structure. Based on the logical connections, contradictions in the environmental understanding graph structure are identified, and the degree of contradiction is quantified to obtain the degree of contradiction. Obtain the degree of conformity between the preliminary environmental interpretation results and the preset rules; Based on the reliability score, the degree of contradiction, and the degree of conformity, determine the root cause of the contradiction. The root cause is subjected to a weight reduction process to lower the reliability score corresponding to the root cause. The confidence flags of the nodes or logical associations of the environment understanding graph structure are updated based on the reduced reliability score to obtain the weighted environment understanding graph structure, thereby completing the vehicle-mounted multi-source data fusion processing. The nodes of the environment understanding graph structure are environment element nodes, which are used to represent the category and spatial attributes of the corresponding environment element.

2. The method for fusion processing of multi-source data in an automotive vehicle according to claim 1, characterized in that, The step of determining the root cause of the contradiction based on the reliability score, the degree of contradiction, and the degree of conformity includes: Image data from an image sensing device is acquired, and the image data is subjected to dehazing and edge enhancement processing to extract the corresponding visual continuity cues. The visual continuity cues are then used to calculate the visual cue authenticity score, which serves as a reliability score component for the visual source. Point cloud data from a lidar radar is acquired, and echo characteristic analysis is performed on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds. An occupancy credibility score is calculated based on the point cloud occupancy evidence, and this score is used as a reliability score component for the point cloud source. The occupancy credibility score characterizes the reliability of the point cloud occupancy evidence; a higher score indicates higher credibility of the real obstacle occupancy evidence. The point cloud occupancy evidence is the geometric evidence used to update the occupancy probability when the point cloud data is mapped to a 3D raster map. The real obstacle occupancy evidence is the portion of the point cloud occupancy evidence formed by real obstacle point clouds. The visual cue authenticity score, the occupancy credibility score, and the vehicle's physical size and kinematic constraints are mapped onto a 3D grid map to obtain the occupancy probability and empty probability of each grid cell. In the three-dimensional grid map, the passability score of each grid cell is calculated by combining the occupancy probability, the openness probability, and the vehicle's physical size and kinematic constraints, and a topology map of the passable area in front of the vehicle is generated based on the passability score. Based on the topology map of the passable area, the conflict intensity between the passability hypothesis corresponding to the visual continuity cue and the blocking hypothesis corresponding to the point cloud occupancy is calculated, and the conflict intensity and the degree of contradiction are checked for consistency to obtain the conflict intensity that passes the consistency check. Based on the reliability score components of the visual source, the reliability score components of the point cloud source, the conflict intensity of the consistency check, the degree of contradiction, and the degree of conformity, the root source is determined to be either the visual source or the point cloud source.

3. The method for fusion processing of multi-source data in an automotive vehicle according to claim 2, characterized in that, The steps of performing dehazing and edge enhancement on the image data, extracting corresponding visual continuity cues, and calculating a visual cue authenticity score based on the visual continuity cues as a reliability score component of the visual source include: The image data is subjected to dehazing and edge enhancement processing to extract initial visual continuity cues; Obtain candidate regions with brightness variations and geometric shapes from the image data; Based on the geometric features, spatial distribution, and similarity to preset traffic signs or construction markings of the candidate regions, the candidate regions are classified to obtain classification results; Based on the classification results, temporary road marking areas are identified, and the geometric parameters of the temporary road marking areas are extracted. Based on the geometric parameters, the initial visual continuity cues are corrected to obtain the corrected visual continuity cues. Based on the modified visual continuity cues in terms of temporal continuity, geometric consistency, and consistency with traffic rule constraints, a visual cue authenticity score is calculated and output as a reliability score component of the visual source.

4. The method for fusion processing of multi-source data in an automotive vehicle according to claim 1, characterized in that, The step of determining the root cause of the contradiction based on the reliability score, the degree of contradiction, and the degree of conformity includes: Obtain the vehicle's physical dimensions and kinematic constraints under its current motion state, and project the kinematic constraints onto a 3D grid map in front of the vehicle to define the vehicle's spatial envelope; Under the constraints of the spatial envelope, visual continuity cues and point cloud data are acquired, and the visual continuity cues and point cloud data are used as geometric evidence indicating whether the grid cells are empty or occupied. Echo characteristic analysis is performed on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds. A first occupancy evidence weight is assigned to the real obstacle point cloud, and a second occupancy evidence weight is assigned to the multipath reflection phantom point cloud. The first occupancy evidence weight is higher than the second occupancy evidence weight. The visual continuity cues, the real obstacle point cloud, and the multipath reflection phantom point cloud are mapped onto the three-dimensional grid map, and the occupancy probability and the empty probability of each grid cell are updated by weighting based on the first occupancy evidence weight and the second occupancy evidence weight. Based on the weighted updated occupancy and vacancy probabilities, under the constraints of vehicle accessibility and traffic rules, connectivity exploration is performed on the three-dimensional grid map to generate a continuous and unobstructed passable area topology map in front of the vehicle; wherein, the connectivity exploration treats grid cells with occupancy probabilities exceeding a threshold as impassable cells. Based on the topology map of the passable area, the degree of conformity between visual continuity cues and passable areas and the degree of conformity between point cloud data and impassable areas are calculated respectively. Combined with the reliability score and the degree of contradiction, it is determined whether the root cause is caused by the visual continuity cues or by the point cloud data.

5. The method for fusion processing of multi-source data in an automotive vehicle according to claim 4, characterized in that, The echo characteristic analysis is performed on the point cloud data to distinguish between real obstacle point clouds and multipath reflection phantom point clouds, and a first occupancy evidence weight is assigned to the real obstacle point cloud, and a second occupancy evidence weight is assigned to the multipath reflection phantom point cloud. The steps in which the weight of the first occupancy evidence is higher than the weight of the second occupancy evidence include: Obtain environmental parameters and statistical characteristics of radar echo signals; Detect whether the statistical characteristics of the environmental parameters or radar echo signals have changed; When a change in the statistical characteristics of the environmental parameters or radar echo signals is detected, point cloud clusters with geometric consistency and spatial stability are identified in the point cloud data. The echo characteristics of the point cloud clusters are extracted and the correlation analysis between the echo characteristics and the environmental parameters is performed to obtain the correlation analysis results; Based on the correlation analysis results, echo discrimination parameters are generated; wherein, the echo discrimination parameters are a set of discrimination parameters used to distinguish between real obstacle echoes and multipath reflection echoes, including at least a phase consistency threshold, an echo intensity statistical threshold, and / or a Doppler frequency shift distribution discrimination threshold, and are used as the discrimination basis for echo characteristic analysis to output the distinction results between real obstacle point clouds and multipath reflection phantom point clouds; The echo discrimination parameters are used to perform echo characteristic analysis on the point cloud data to output the distinction results between real obstacle point clouds and phantom point clouds; Based on the distinction results, a first occupancy evidence weight is assigned to the real obstacle point cloud, and a second occupancy evidence weight is assigned to the multipath reflection phantom point cloud; the first occupancy evidence weight is higher than the second occupancy evidence weight.

6. The method for fusion processing of multi-source data in an automotive vehicle according to claim 5, characterized in that, The steps for obtaining environmental parameters and statistical characteristics of radar echo signals include: Acquire environmental parameters provided by the onboard environmental sensor array and obtain radar echo signals; Time series analysis was performed on the environmental parameters to identify their changing trends and fluctuation amplitudes. The radar echo signal is subjected to statistical characteristic calculations to obtain statistical characteristics; the statistical characteristics are then subjected to time series analysis to identify the changing trend and fluctuation amplitude of the radar echo signal; the statistical characteristics include average intensity, standard deviation, phase consistency, and Doppler frequency shift distribution. Based on the changing trends and fluctuation ranges of environmental parameters, the environmental parameters are calibrated by combining the vehicle's geographical location and preset meteorological data to obtain calibrated environmental parameters; Based on the changing trend and fluctuation amplitude of the radar echo signal, outlier detection and filtering are performed on the statistical characteristics to suppress false fluctuations caused by sensor noise, transient interference or abnormal data transmission, and the filtered radar echo signal is obtained.

7. The method for fusion processing of multi-source data in an automotive vehicle according to claim 6, characterized in that, The step of calculating the statistical characteristics of the radar echo signal to obtain the statistical characteristics includes: The radar echo signal was subjected to joint time-domain and frequency-domain analysis to identify echo components with different center frequencies and bandwidths. Waveform features are extracted for each echo component to obtain instantaneous amplitude, instantaneous phase, and instantaneous frequency; Based on the instantaneous amplitude, instantaneous phase, and instantaneous frequency, the average intensity, standard deviation, phase consistency, and Doppler frequency shift distribution of the corresponding echo components are calculated and output as statistical characteristics.

8. The method for fusion processing of multi-source data in an automotive vehicle according to claim 6, characterized in that, The step of calculating the statistical characteristics of the radar echo signal to obtain the statistical characteristics includes: The radar echo signal was subjected to joint time-domain and frequency-domain analysis to identify echo components with different center frequencies and bandwidths. Multiple echo components with similar time-frequency characteristics and spatial consistency are clustered into echo clusters; the echo clusters are used to characterize the same obstacle or the same reflector. The average intensity, standard deviation, phase consistency, and Doppler frequency shift distribution of the echo cluster are output as statistical characteristics of the echo cluster by assigning weights based on the signal-to-noise ratio, phase consistency, and time continuity of the echo components. The statistical characteristics of the echo cluster are used as the statistical characteristics of the radar echo signal.

9. The method for fusion processing of multi-source data in an automotive vehicle according to claim 7, characterized in that, The step of performing joint time-domain and frequency-domain analysis on the radar echo signal to identify echo components with different center frequencies and bandwidths includes: Perform a short-time Fourier transform on the radar echo signal to obtain a time-frequency distribution map; Different energy accumulation centers are identified on the time-frequency distribution map to obtain candidate regions for the extended range corresponding to different energy accumulation centers; Boundary extraction and shape analysis are performed on the candidate regions to distinguish between echo component regions generated by a single obstacle and echo component regions generated by the superposition of multiple obstacles, thereby obtaining echo component region distinction results. Adaptive time-frequency window adjustment is performed on the echo component region generated by the superposition of multiple obstacles to separate the independent echo components inside; Based on the echo component region differentiation results and independent echo components, echo components with different center frequencies and bandwidths are identified and output.

10. A vehicle-mounted multi-source data fusion processing system for performing vehicle-mounted multi-source data fusion processing, characterized in that, include: The sensor information acquisition module is used to acquire sensor information about the environment around the vehicle. The sensing information includes preliminary environmental interpretation results from different information sources and reliability scores corresponding to each preliminary environmental interpretation result; An environment structure construction module is used to construct an environment understanding graph structure based on the preliminary environment interpretation results and the corresponding reliability score; the environment understanding graph structure includes predefined logical relationships between environmental elements; The contradiction degree quantification module is used to identify contradictory items in the environment understanding graph structure based on the logical association, and to quantify the degree of contradiction of the contradictory items to obtain the degree of contradiction. The conformity assessment module is used to acquire the degree of conformity between the preliminary environmental interpretation results and preset rules; The root cause determination module is used to determine the root cause of the contradictory item based on the reliability score, the degree of contradiction, and the degree of conformity. The root source weighting module is used to apply weighting processing to the root source to reduce the reliability score corresponding to the root source, and update the confidence flags of the nodes or logical associations of the environment understanding graph structure based on the reduced reliability score to obtain the weighted environment understanding graph structure, so as to complete the vehicle-mounted multi-source data fusion processing.

Images