Multi-source data fusion method and system based on dynamic environment perception and cloud edge collaboration

Abstract

The invention relates to the technical field of vehicles, in particular to a multi-source data fusion method and system based on dynamic environment perception and cloud edge collaboration, and the method comprises the steps: obtaining first multi-source heterogeneous data of a vehicle side and second multi-source heterogeneous data of a road side; effective data in the first multi-source heterogeneous data and the second multi-source heterogeneous data are extracted respectively, and the obtained first effective data and the obtained second effective data are uploaded to a cloud end; the first valid data comprises image data and radar data; the second effective data comprises environment data and traffic data; performing dynamic weight fusion on the image data and the radar data based on the environment data to obtain fusion data; converting the fused data into a data model in a unified format, and performing global path planning based on the data model in combination with real-time traffic data and vehicle intention to generate an optimal path; according to the invention, through advantage complementation of multi-source data, the robustness, real-time performance and cross-system compatibility of data perception can be improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a multi-source data fusion method and system based on dynamic environment perception and cloud-edge collaboration. Background Art

[0002] In the related vehicle-road-cloud integrated architecture technology, collaborative perception of multi-source heterogeneous data mainly relies on the following methods, but there are still significant pain points: (1) Low data fusion efficiency: For example, fixed weight fusion is difficult to adapt to dynamic environments (such as weather changes and equipment failures), resulting in poor robustness of perception results. For example, on-board cameras are prone to failure in rainy and foggy weather, but the existing system cannot dynamically adjust the weight compensation of other sensors (such as roadside lidar).

[0003] (2) Irrational resource allocation: The division of labor between edge computing nodes and the cloud is unclear, resulting in wasted computing resources on edge devices or overload on the cloud. For example, some systems upload raw video streams directly to the cloud, causing transmission delays and bandwidth pressure.

[0004] (3) Conflict between real-time performance and accuracy: Existing technologies have difficulty balancing the real-time performance of data processing and the accuracy of in-depth analysis. For example, in security verification systems, the fusion of multi-source heterogeneous data takes a long time, affecting passenger efficiency.

[0005] (4) Insufficient cross-system collaboration: The data formats of different platforms vary greatly (such as in the financial and medical fields), and there is a lack of standardized interfaces, which makes cross-system data integration difficult.

[0006] Therefore, it is necessary to provide a solution that can improve perception robustness, real-time performance, and cross-system compatibility. Summary of the Invention

[0007] In view of this, the purpose of an embodiment of the present invention is to provide a multi-source data fusion method and system based on dynamic environment perception and cloud-edge collaboration to solve one or more technical problems existing in the prior art, and at least provide a beneficial choice or create conditions.

[0008] In one aspect, an embodiment of the present invention provides a multi-source data fusion method based on dynamic environment perception and cloud-edge collaboration, the method comprising the following steps: Acquire first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side; the first multi-source heterogeneous data includes initial image data and initial radar data; the second multi-source heterogeneous data includes initial environmental data and initial traffic data; Extracting valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data respectively, and uploading the obtained first valid data and the second valid data to the cloud; the first valid data includes image data and radar data; the second valid data includes environmental data and traffic data; Performing dynamic weighted fusion on the image data and the radar data based on the environmental data to obtain fused data; The fused data is converted into a data model in a unified format, combined with real-time traffic data and vehicle intentions, and global path planning is performed based on the data model to generate the optimal path. Optionally, the acquiring of the first multi-source heterogeneous data on the vehicle side and the second multi-source heterogeneous data on the road side includes: Collecting initial image data through a vehicle-mounted camera and collecting initial radar data through a vehicle-mounted radar; the vehicle-mounted camera and vehicle-mounted radar are arranged on the vehicle side; Initial environmental data is collected by meteorological sensors, and initial traffic data is collected by laser radar; the meteorological sensors and laser radar are arranged on the roadside.

[0009] Optionally, extracting valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data respectively, and uploading the obtained first valid data and second valid data to the cloud, includes: A first lightweight target detection model is used by the vehicle-side device to extract first valid data from the first multi-source heterogeneous data and upload the data to the cloud; A second lightweight target detection model is adopted by the roadside equipment to extract second valid data from the second multi-source heterogeneous data and upload it to the cloud.

[0010] Optionally, the dynamically weighted fusion of the image data and the radar data based on the environmental data to obtain fused data includes: Obtaining environmental data, and matching the confidence levels of image data and radar data under the environmental data from a preset dynamic weight allocation table; wherein the dynamic weight allocation table includes preset confidence levels for image data and radar data under various environmental scenarios; determining a first weight for the image data and a second weight for the radar data based on confidence levels of the image data and the radar data, wherein a sum of the first weight and the second weight is 1; The image data is weighted based on the first weight, the radar data is weighted based on the second weight, and fusion data is generated based on the weighted image data and the weighted radar data.

[0011] Optionally, the method further includes: If the performance of the onboard camera is detected to be degraded, the confidence level of the image data is reduced; If the performance of the vehicle's radar is detected to be degraded, the confidence level of the radar data is reduced.

[0012] On the other hand, an embodiment of the present invention provides a multi-source data fusion system based on dynamic environment perception and cloud-edge collaboration, including: Data acquisition layer: used to obtain first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side; the first multi-source heterogeneous data includes initial image data and initial radar data; the second multi-source heterogeneous data includes initial environmental data and initial traffic data; Edge preprocessing layer: used to extract valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data, and upload the obtained first valid data and the second valid data to the cloud; the first valid data includes image data and radar data; the second valid data includes environmental data and traffic data; Dynamic fusion core layer: used to perform dynamic weighted fusion of the image data and radar data based on environmental data to obtain fused data; Cloud-based decision-making layer: used to convert fused data into a data model in a unified format, combine real-time traffic data and vehicle intentions, perform global path planning based on the data model, and generate the optimal path. Optionally, the data acquisition layer is specifically used to: The image data is collected by the vehicle-mounted camera, and the radar data is collected by the vehicle-mounted radar; the vehicle-mounted camera and the vehicle-mounted radar are arranged on the vehicle side; Environmental data is collected by meteorological sensors, and traffic data is collected by laser radar; the meteorological sensors and laser radar are set on the roadside.

[0013] Optionally, the edge preprocessing layer is specifically used to: A first lightweight target detection model is used to extract first valid data from the first multi-source heterogeneous data and upload the data to the cloud; A second lightweight target detection model is used to extract second valid data from the second multi-source heterogeneous data and upload it to the cloud.

[0014] Optionally, the dynamic fusion core layer is specifically used to: Obtaining environmental data, and matching the confidence levels of image data and radar data under the environmental data from a preset dynamic weight allocation table; wherein the dynamic weight allocation table includes preset confidence levels for image data and radar data under various environmental scenarios; determining a first weight for the image data and a second weight for the radar data based on confidence levels of the image data and the radar data, wherein a sum of the first weight and the second weight is 1; The image data is weighted based on the first weight, the radar data is weighted based on the second weight, and fusion data is generated based on the weighted image data and the weighted radar data.

[0015] Optionally, the dynamic fusion core layer is further configured to: If the performance of the onboard camera is detected to be degraded, the confidence level of the image data is reduced; If the performance of the vehicle's radar is detected to be degraded, the confidence level of the radar data is reduced.

[0016] The embodiments of the present invention include the following beneficial effects: The present invention proposes a multi-source data fusion method and system based on dynamic environment perception and cloud-edge collaboration. First, the first multi-source heterogeneous data on the vehicle side and the second multi-source heterogeneous data on the road side are obtained; then, data is screened through the edge pre-processing layer, and valid data is uploaded to the cloud; then, weighted fusion is performed in the dynamic fusion core layer to generate fused data; the fused data is converted into a unified format, and global path planning is performed in combination with real-time traffic data and vehicle intentions, and finally the optimal path is generated to ensure accurate decision-making and efficient operation of the autonomous driving system. The present invention improves fusion accuracy, optimizes path planning, and enhances driving safety and efficiency by dynamically adjusting data weights, and is suitable for complex traffic environments. It effectively solves the limitations of traditional single data sources in complex environments, realizes the complementary advantages of multi-source data, and significantly improves the vehicle's adaptability under changing road conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a schematic diagram of the steps of a multi-source data fusion method based on dynamic environment perception and cloud-edge collaboration provided by an embodiment of the present invention; Figure 2 This is an architecture diagram of a multi-source data fusion system based on dynamic environment perception and cloud-edge collaboration provided by an embodiment of the present invention; Figure 3 This is a diagram of the architecture of the dynamic weight allocation algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] It should be noted that although the device schematics illustrate a modular division and the flowcharts illustrate a logical sequence, in some cases, the steps shown or described may be performed in a different order than the modular division in the device or the sequence in the flowcharts. The terms "first," "second," and so on in the specification, claims, and drawings are used to distinguish similar items and are not necessarily used to describe a specific order or precedence.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0021] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, it will be appreciated by those skilled in the art that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0022] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0023] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0024] In order to solve the technical problems in the background technology, the present invention provides a multi-source data fusion method and system based on dynamic environmental perception and cloud-edge collaboration, which improves perception robustness, real-time performance and cross-system compatibility by dynamically adapting to environmental changes and optimizing resource allocation.

[0025] like Figure 1 As shown, Figure 1 An embodiment of the present invention provides a multi-source data fusion method based on dynamic environment perception and cloud-edge collaboration, the method comprising the following steps: S100, acquiring first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side; the first multi-source heterogeneous data including initial image data and initial radar data; the second multi-source heterogeneous data including initial environmental data and initial traffic data; The data collection layer is used to access multi-source, heterogeneous data from the vehicle (on-board cameras, on-board radar), the roadside (meteorological sensors, lidar), and the cloud (historical data, high-precision maps). The implementation process first requires building the infrastructure for a vehicle-road-cloud collaborative perception platform, including an on-board sensor array, roadside perception devices, and a cloud server cluster. The on-board sensor array includes high-definition cameras, millimeter-wave radar, and lidar, collecting data about the vehicle's surroundings at different frequencies. Roadside perception devices, including meteorological sensors and lidar, are deployed at key traffic nodes to monitor environmental data and road traffic flow in real time. The cloud server cluster connects the vehicle-side and roadside devices via a high-speed network, enabling data fusion and processing to ensure the comprehensiveness and efficiency of the perception system.

[0026] S200, extracting valid data from first and second multi-source heterogeneous data, respectively, and uploading the obtained first and second valid data to the cloud; the first valid data includes image data and radar data; the second valid data includes environmental data and traffic data; The edge preprocessing layer deploys a lightweight target detection model (YOLO-Lite) to filter redundant data in real time and upload only valid data (such as obstacles and traffic accidents) to the cloud.

[0027] S300, performing dynamic weighted fusion on the image data and the radar data based on the environmental data to obtain fused data; The dynamic fusion core layer utilizes an advanced dynamic weighting algorithm. This algorithm accurately calculates the confidence level of each sensor by deeply analyzing and comprehensively considering various environmental data, such as visibility and signal strength, and other key factors. Based on this, the algorithm dynamically adjusts the weighting of each sensor in the data fusion process according to real-time environmental changes to achieve optimal information fusion. Specifically, in adverse weather conditions such as rain and fog, the algorithm intelligently adjusts the weighting of cameras and radar to ensure perception and decision-making accuracy in complex environments, effectively improving overall stability and reliability. For example, in rainy and foggy weather conditions, the algorithm automatically reduces the weight of camera data while increasing the contribution of millimeter-wave radar and lidar data. The fused perception results are transmitted to the vehicle in real time via an encrypted channel, providing reliable environmental perception input for autonomous driving decisions.

[0028] S400 converts the fused data into a data model in a unified format, combines real-time traffic data and vehicle intentions, performs global path planning based on the data model, and generates the optimal path. Specifically, the cross-system data mapping layer defines a common data model (JSON Schema) to convert heterogeneous data into a unified format. The cloud-based decision-making layer combines real-time traffic data and vehicle intent to perform global path planning using game theory algorithms to generate the optimal route. In steps S100 to S400 illustrated in the embodiment of the present invention, first, first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side are obtained; then, data is screened through the edge preprocessing layer, and valid data is uploaded to the cloud; then, weighted fusion is performed in the dynamic fusion core layer to generate fused data; the fused data is then converted into a unified format, and global path planning is performed in combination with real-time traffic data and vehicle intentions, ultimately generating the optimal path to ensure accurate decision-making and efficient operation of the autonomous driving system.

[0029] In the embodiments provided by this invention, edge nodes deploy lightweight preprocessing algorithms to reduce noise and extract features from raw data, significantly reducing transmission bandwidth requirements. A distributed computing architecture is employed in the cloud to enable parallel processing of massive amounts of data, improving system response speed. A unified data interface specification supports access to devices from different manufacturers, ensuring system compatibility and scalability.

[0030] In some embodiments, acquiring the first multi-source heterogeneous data on the vehicle side and the second multi-source heterogeneous data on the road side includes: Collecting initial image data through a vehicle-mounted camera and collecting initial radar data through a vehicle-mounted radar; the vehicle-mounted camera and vehicle-mounted radar are arranged on the vehicle side; Initial environmental data is collected by meteorological sensors, and initial traffic data is collected by laser radar; the meteorological sensors and laser radar are arranged on the roadside.

[0031] Specifically, the first multi-source heterogeneous data includes the initial image data collected by the on-board camera and the initial radar data collected by the on-board radar; the second multi-source heterogeneous data includes the initial environmental data collected by the meteorological sensor and the initial traffic data collected by the lidar; the real-time data streams of the vehicle-side equipment and the roadside equipment are preliminarily integrated through their respective edge computing nodes, and then transmitted to the cloud for in-depth analysis and optimization to ensure the real-time and accuracy of the perception system.

[0032] In some embodiments, extracting valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data respectively, and uploading the obtained first valid data and second valid data to the cloud, includes: A first lightweight target detection model is used by the vehicle-side device to extract first valid data from the first multi-source heterogeneous data and upload the data to the cloud; A second lightweight target detection model is adopted by the roadside equipment to extract second valid data from the second multi-source heterogeneous data and upload it to the cloud.

[0033] Specifically, the data processing phase utilizes a distributed computing framework, deploying a lightweight object detection model (YOLO-Lite) on edge computing nodes, such as onboard and roadside nodes, to complete data preprocessing. This model uses quantization and compression techniques to reduce the number of parameters by 60% while maintaining over 90% detection accuracy, significantly reducing the computational load on edge devices. The preprocessed data is transmitted to the cloud via a wireless network, where timestamp alignment and spatial coordinate transformation techniques are used to achieve spatiotemporal synchronization of multi-source data.

[0034] In some embodiments, the dynamically weighted fusion of the image data and the radar data based on the environmental data to obtain the fused data includes: Obtaining environmental data, and matching the confidence levels of image data and radar data under the environmental data from a preset dynamic weight allocation table; wherein the dynamic weight allocation table includes preset confidence levels for image data and radar data under various environmental scenarios; determining a first weight for the image data and a second weight for the radar data based on confidence levels of the image data and the radar data, wherein a sum of the first weight and the second weight is 1; The image data is weighted based on the first weight, the radar data is weighted based on the second weight, and fusion data is generated based on the weighted image data and the weighted radar data.

[0035] Specifically, environmental data, including weather visibility, weather type, and device health, is fed into the dynamic fusion core layer. The data quality of individual sensors can vary significantly under different environmental conditions. For example, in rainy and foggy weather, the clarity of camera-collected image data decreases significantly. In this case, the system automatically increases the weighting coefficient of millimeter-wave radar data to 0.7 and reduces the weighting of camera data to 0.3. The system's built-in dynamic weighting table contains preset parameters for various typical environmental scenarios, and continuously optimizes and adjusts the strategy through machine learning.

[0036] For example, when a vehicle passes through an intersection during heavy rain, if roadside sensors detect visibility less than 50 meters, dynamic weight adjustment (LiDAR weight increased to 0.8) can be triggered. Edge nodes identify obstacles at the intersection, compress them, and upload them to the cloud. The cloud, combined with historical accident data, generates a detour route and sends it to the vehicle. This can result in a 15% reduction in accident rates and a 20% increase in traffic efficiency.

[0037] Dynamic weight allocation algorithm and its environmental data triggering mechanism; division of labor strategy between lightweight models and complex models in cloud-edge collaborative architecture; standardized interface design for cross-system data mapping.

[0038] In some embodiments, the method further comprises: If the performance of the onboard camera is detected to be degraded, the confidence level of the image data is reduced; If the performance of the vehicle's radar is detected to be degraded, the confidence level of the radar data is reduced.

[0039] During operation, the performance indicators of each sensor are continuously monitored. When sensor anomalies or environmental changes are detected, the weight adjustment mechanism is automatically triggered. At the same time, a data quality assessment system is established to classify the credibility of input data to ensure the reliability of decision-making basis.

[0040] Specifically, when a sensor detects continuous anomalies in data, it automatically switches to a backup data source and sends a device maintenance reminder to operations personnel via the cloud platform. This design significantly improves the system's fault tolerance, ensuring that even in the event of partial device failure, it can still maintain a perception accuracy rate of over 90%.

[0041] like Figure 2 The architecture diagram of the multi-source data fusion system based on dynamic environment perception and cloud-edge collaboration is shown, and the system includes: Data acquisition layer: used to obtain first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side; the first multi-source heterogeneous data includes initial image data and initial radar data; the second multi-source heterogeneous data includes initial environmental data and initial traffic data; The data collection layer is used to access multi-source, heterogeneous data from the vehicle (on-board cameras, on-board radar), the roadside (meteorological sensors, lidar), and the cloud (historical data, high-precision maps). The implementation process first requires building the infrastructure for a vehicle-road-cloud collaborative perception platform, including an on-board sensor array, roadside perception devices, and a cloud server cluster. The on-board sensor array includes high-definition cameras, millimeter-wave radar, and lidar, collecting data about the vehicle's surroundings at different frequencies. Roadside perception devices, including meteorological sensors and lidar, are deployed at key traffic nodes to monitor environmental data and road traffic flow in real time. The cloud server cluster connects the vehicle-side and roadside devices via a high-speed network, enabling data fusion and processing to ensure the comprehensiveness and efficiency of the perception system.

[0042] Edge preprocessing layer: used to extract valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data, and upload the obtained first valid data and the second valid data to the cloud; the first valid data includes image data and radar data; the second valid data includes environmental data and traffic data; The edge preprocessing layer deploys a lightweight target detection model (YOLO-Lite) to filter redundant data in real time and upload only valid data (such as obstacles and traffic accidents) to the cloud.

[0043] Dynamic fusion core layer: used to perform dynamic weighted fusion of the image data and radar data based on environmental data to obtain fused data; The dynamic fusion core layer utilizes an advanced dynamic weighting algorithm. This algorithm accurately calculates the confidence level of each sensor by deeply analyzing and comprehensively considering various environmental data, such as visibility and signal strength, and other key factors. Based on this, the algorithm dynamically adjusts the weighting of each sensor in the data fusion process according to real-time environmental changes to achieve optimal information fusion. Specifically, in adverse weather conditions such as rain and fog, the algorithm intelligently adjusts the weighting of cameras and radar to ensure perception and decision-making accuracy in complex environments, effectively improving overall stability and reliability. For example, in rainy and foggy weather conditions, the algorithm automatically reduces the weight of camera data while increasing the contribution of millimeter-wave radar and lidar data. The fused perception results are transmitted to the vehicle in real time via an encrypted channel, providing reliable environmental perception input for autonomous driving decisions.

[0044] Cloud-based decision-making layer: used to convert fused data into a data model in a unified format, combine real-time traffic data and vehicle intentions, perform global path planning based on the data model, and generate the optimal path. Specifically, the cross-system data mapping layer defines a common data model (JSON Schema) to convert heterogeneous data into a unified format. The cloud-based decision-making layer combines real-time traffic data and vehicle intent to perform global path planning using game theory algorithms to generate the optimal route. In the embodiments provided by this invention, edge nodes deploy lightweight preprocessing algorithms to reduce noise and extract features from raw data, significantly reducing transmission bandwidth requirements. A distributed computing architecture is employed in the cloud to enable parallel processing of massive amounts of data, improving system response speed. A unified data interface specification supports access to devices from different manufacturers, ensuring system compatibility and scalability.

[0045] During operation, the system continuously monitors the performance of each module and automatically triggers weight adjustments when sensor anomalies or environmental changes are detected. A data quality assessment system is also established to grade the credibility of input data, ensuring the reliability of decision-making. By introducing federated learning technology, multi-terminal collaborative training is achieved while protecting data privacy, continuously improving model generalization capabilities.

[0046] In some embodiments, the data collection layer is specifically used to: The image data is collected by the vehicle-mounted camera, and the radar data is collected by the vehicle-mounted radar; the vehicle-mounted camera and the vehicle-mounted radar are arranged on the vehicle side; Environmental data is collected by meteorological sensors, and traffic data is collected by laser radar; the meteorological sensors and laser radar are set on the roadside.

[0047] Specifically, the first multi-source heterogeneous data includes image data collected by on-board cameras and radar data collected by on-board radars; the second multi-source heterogeneous data includes environmental data collected by meteorological sensors and traffic data collected by lidars; the real-time data streams of vehicle-side equipment and roadside equipment are initially integrated through their respective edge computing nodes, and then transmitted to the cloud for in-depth analysis and optimization to ensure the real-time and accuracy of the perception system.

[0048] In some embodiments, the edge pre-processing layer is specifically used to: A first lightweight target detection model is used to extract first valid data from the first multi-source heterogeneous data and upload the data to the cloud; A second lightweight target detection model is used to extract second valid data from the second multi-source heterogeneous data and upload it to the cloud.

[0049] Specifically, the data processing phase utilizes a distributed computing framework, deploying a lightweight object detection model (YOLO-Lite) on edge computing nodes, such as onboard and roadside nodes, to complete data preprocessing. This model uses quantization and compression techniques to reduce the number of parameters by 60% while maintaining over 90% detection accuracy, significantly reducing the computational load on edge devices. The preprocessed data is transmitted to the cloud via a wireless network, where timestamp alignment and spatial coordinate transformation techniques are used to achieve spatiotemporal synchronization of multi-source data.

[0050] refer to Figure 3 In some embodiments, the dynamic fusion core layer is specifically used to: Obtaining environmental data, and matching the confidence levels of image data and radar data under the environmental data from a preset dynamic weight allocation table; wherein the dynamic weight allocation table includes preset confidence levels for image data and radar data under various environmental scenarios; determining a first weight for the image data and a second weight for the radar data based on confidence levels of the image data and the radar data, wherein a sum of the first weight and the second weight is 1; The image data is weighted based on the first weight, the radar data is weighted based on the second weight, and fusion data is generated based on the weighted image data and the weighted radar data.

[0051] Specifically, environmental data, including weather visibility, weather type, and device health, is fed into the dynamic fusion core layer. The data quality of individual sensors can vary significantly under different environmental conditions. For example, in rainy and foggy weather, the clarity of camera-collected image data decreases significantly. In this case, the system automatically increases the weighting coefficient of millimeter-wave radar data to 0.7 and reduces the weighting of camera data to 0.3. The system's built-in dynamic weighting table contains preset parameters for various typical environmental scenarios, and continuously optimizes and adjusts the strategy through machine learning.

[0052] For example, when a vehicle passes through an intersection during heavy rain, if roadside sensors detect visibility less than 50 meters, dynamic weight adjustment (LiDAR weight increased to 0.8) can be triggered. Edge nodes identify obstacles at the intersection, compress them, and upload them to the cloud. The cloud, combined with historical accident data, generates a detour route and sends it to the vehicle. This can result in a 15% reduction in accident rates and a 20% increase in traffic efficiency.

[0053] Dynamic weight allocation algorithm and its environmental data triggering mechanism; division of labor strategy between lightweight models and complex models in cloud-edge collaborative architecture; standardized interface design for cross-system data mapping.

[0054] In some embodiments, the dynamic fusion core layer is further configured to: If the performance of the onboard camera is detected to be degraded, the confidence level of the image data is reduced; If the performance of the vehicle's radar is detected to be degraded, the confidence level of the radar data is reduced.

[0055] Specifically, when a sensor detects continuous anomalies in data, it automatically switches to a backup data source and sends a device maintenance reminder to operations personnel via the cloud platform. This design significantly improves the system's fault tolerance, ensuring that even in the event of partial device failure, it can still maintain a perception accuracy rate of over 90%.

[0056] This paper proposes a multi-source data fusion method and system based on dynamic environment perception and cloud-edge collaboration. Compared with related technologies, this paper has the following advantages: The present invention can automatically adjust the weight of sensor data according to environmental conditions (such as weather and lighting), reduce the risk of single sensor failure, and improve perception accuracy (such as integrating roadside lidar data to compensate for blind spots of on-board cameras in rain and fog); edge nodes perform lightweight preprocessing, and the cloud is responsible for complex model training and global optimization, which can reduce transmission delays by more than 30%; it can standardize data interfaces, support cross-industry data format unification (such as finance and transportation), and reduce integration complexity. An embodiment of the present invention further provides a vehicle, comprising the vehicle control device of the above embodiment.

[0057] The vehicle can be a private car, such as a sedan, SUV, MPV, or pickup truck. It can also be a commercial vehicle, such as a van, bus, small truck, or large trailer. The vehicle must have an electric motor that can output power or store mechanical energy as a generator. If the vehicle is a new energy vehicle, it can be a hybrid or a pure electric vehicle.

[0058] Since the vehicle applies all the technical solutions of the above-mentioned control device or vehicle controller, it has at least all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

[0059] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are used to execute the above method.

[0060] It is worth noting that since the computer-readable storage medium of an embodiment of the present invention can execute the method of any of the above embodiments, the specific implementation methods and technical effects of the computer-readable storage medium of an embodiment of the present invention can refer to the specific implementation methods and technical effects of the method of any of the above embodiments.

[0061] In addition, an embodiment of the present invention also provides a computer program product, including a computer program or computer instructions, the computer program or computer instructions are stored in a computer-readable storage medium, the processor of the computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device performs the above method.

[0062] It is worth noting that since the computer program product of the embodiment of the present invention can execute the method of any of the above embodiments, the specific implementation methods and technical effects of the computer program product of the embodiment of the present invention can refer to the specific implementation methods and technical effects of the method of any of the above embodiments.

[0063] Those skilled in the art will appreciate that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, or any suitable combination thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0064] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

Claims

1. A multi-source data fusion method based on dynamic environment perception and cloud-edge collaboration, characterized by: The method comprises: Acquire first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side; the first multi-source heterogeneous data includes initial image data and initial radar data; the second multi-source heterogeneous data includes initial environmental data and initial traffic data; Extracting valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data respectively, and uploading the obtained first valid data and the second valid data to the cloud; the first valid data includes image data and radar data; the second valid data includes environmental data and traffic data; Performing dynamic weighted fusion on the image data and the radar data based on the environmental data to obtain fused data; The fused data is converted into a data model in a unified format, combined with real-time traffic data and vehicle intentions, and global path planning is performed based on the data model to generate the optimal path.

2. The method according to claim 1, characterized in that The acquiring of first multi-source heterogeneous data on the vehicle side and second multi-source heterogeneous data on the road side includes: Collecting initial image data through a vehicle-mounted camera and collecting initial radar data through a vehicle-mounted radar; the vehicle-mounted camera and vehicle-mounted radar are arranged on the vehicle side; Initial environmental data is collected by meteorological sensors, and initial traffic data is collected by laser radar; the meteorological sensors and laser radar are arranged on the roadside.

3. The method according to claim 1, characterized in that The extracting valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data respectively, and uploading the obtained first valid data and second valid data to the cloud, comprises: A first lightweight target detection model is used by the vehicle-side device to extract first valid data from the first multi-source heterogeneous data and upload the data to the cloud; A second lightweight target detection model is adopted by the roadside equipment to extract second valid data from the second multi-source heterogeneous data and upload it to the cloud.

4. The method according to claim 3, characterized in that The dynamically weighted fusion of the image data and the radar data based on the environmental data to obtain fused data includes: Obtaining environmental data, and matching the confidence levels of image data and radar data under the environmental data from a preset dynamic weight allocation table; wherein the dynamic weight allocation table includes preset confidence levels for image data and radar data under various environmental scenarios; determining a first weight for the image data and a second weight for the radar data based on confidence levels of the image data and the radar data, wherein a sum of the first weight and the second weight is 1; The image data is weighted based on the first weight, the radar data is weighted based on the second weight, and fusion data is generated based on the weighted image data and the weighted radar data.

5. The method according to claim 3, characterized in that The method further comprises: If the performance of the onboard camera is detected to be degraded, the confidence level of the image data is reduced; If the performance of the vehicle's radar is detected to be degraded, the confidence level of the radar data is reduced.

6. A multi-source data fusion system based on dynamic environment perception and cloud-edge collaboration, characterized by: include: Data collection layer: used to obtain the first multi-source heterogeneous data on the vehicle side and the second multi-source heterogeneous data on the road side; The first multi-source heterogeneous data includes initial image data and initial radar data; The second multi-source heterogeneous data includes initial environment data and initial traffic data; Edge preprocessing layer: used to extract valid data from the first multi-source heterogeneous data and the second multi-source heterogeneous data, and upload the obtained first valid data and the second valid data to the cloud; the first valid data includes image data and radar data; the second valid data includes environmental data and traffic data; Dynamic fusion core layer: used to perform dynamic weighted fusion of the image data and radar data based on environmental data to obtain fused data; Cloud-based decision-making layer: used to convert fused data into a data model in a unified format, combine real-time traffic data and vehicle intentions, perform global path planning based on the data model, and generate the optimal path.

7. The system according to claim 6, characterized in that The data collection layer is specifically used for: The image data is collected by the vehicle-mounted camera, and the radar data is collected by the vehicle-mounted radar; the vehicle-mounted camera and the vehicle-mounted radar are arranged on the vehicle side; Environmental data is collected by meteorological sensors, and traffic data is collected by laser radar; the meteorological sensors and laser radar are set on the roadside.

8. The system according to claim 6, wherein: The edge preprocessing layer is specifically used for: A first lightweight target detection model is used to extract first valid data from the first multi-source heterogeneous data and upload the data to the cloud; A second lightweight target detection model is used to extract second valid data from the second multi-source heterogeneous data and upload it to the cloud.

9. The system according to claim 6, wherein: The dynamic fusion core layer is specifically used for: Obtaining environmental data, and matching the confidence levels of image data and radar data under the environmental data from a preset dynamic weight allocation table; wherein the dynamic weight allocation table includes preset confidence levels for image data and radar data under various environmental scenarios; determining a first weight for the image data and a second weight for the radar data based on confidence levels of the image data and the radar data, wherein a sum of the first weight and the second weight is 1; The image data is weighted based on the first weight, the radar data is weighted based on the second weight, and fusion data is generated based on the weighted image data and the weighted radar data.

10. The system according to claim 6, wherein: The dynamic fusion core layer is also used for: If the performance of the onboard camera is detected to be degraded, the confidence level of the image data is reduced; If the performance of the vehicle's radar is detected to be degraded, the confidence level of the radar data is reduced.

Images