Data management system and method based on intelligent network connection centralized electronic and electrical architecture

Abstract

The invention discloses a data management system and method based on an intelligent network connection centralized electronic and electrical architecture, which can process multi-source heterogeneous data. The data management system comprises a data center module integrated on a central computing platform. The data center module comprises a data customization processing unit, a data storage unit and a cloud interaction unit. The data customization processing unit is used for acquiring multi-source heterogeneous data generated by each electronic module of the vehicle, and performing data classification on the multi-source heterogeneous data according to a customized data type; performing data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the multi-source heterogeneous data after data classification to obtain standardized data required by the central computing platform and each electronic module; the data storage unit is used for storing data generated in the running process of the vehicle and storing the whole vehicle behavior data, the fault alarm data and the driver behavior data which are processed by the data customization processing unit.

Description

Technical Field

[0001] The present application relates to the field of automotive technology, and more specifically, to a data management system and method based on an intelligent networked centralized electronic and electrical architecture. Background Art

[0002] China's automobile industry is rapidly evolving along the new four lines of electrification, intelligence, networking and sharing. The automotive electronic and electrical architecture is also gradually changing from the current mature distributed architecture to the distributed domain control architecture, then to the domain centralized architecture, and finally to the centralized architecture. In the future centralized electronic and electrical architecture, the central computing module needs to process a huge amount of data, and different modules will work together in the whole vehicle system, which will lead to data incompatibility, low data retrieval efficiency, storage overflow and other conditions, causing system congestion and then collapse. In terms of data processing, the electronic and electrical architecture in the existing technology can only classify data, and cannot solve the problems faced by the future centralized electronic and electrical architecture, nor does it propose a reliable solution for signal interaction after future networking. Summary of the invention

[0003] The present application provides a data management system and method based on an intelligent networked centralized electronic and electrical architecture to overcome at least one technical problem existing in the prior art.

[0004] The specific technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present application provides a data management system based on an intelligent network-connected centralized electronic and electrical architecture, the data management system comprising: a data center module integrated in a central computing platform, the data center module comprising a data customization processing unit, a data storage unit, and a cloud interaction unit;

[0006] The data customization processing unit is used to obtain multi-source heterogeneous data generated by each electronic module of the vehicle, classify the multi-source heterogeneous data according to the customized data type, and perform data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the multi-source heterogeneous data after data classification, so as to obtain the standardized data required by the central computing platform and each electronic module; wherein the multi-source heterogeneous data includes sensor acquisition data, vehicle internal signal data and cloud data, and the customized data type includes visual information data, vehicle behavior data, communication data, positioning and radar data, fault alarm data, cloud interaction data and driver behavior data;

[0007] The data storage unit is used to store the data generated by the vehicle during operation, and to store the vehicle behavior data, the fault alarm data and the driver behavior data processed by the data customization processing unit;

[0008] The cloud interaction unit is used for data transmission between the cloud control platform and the data customization processing unit, to obtain the cloud data of the cloud control platform, and to upload the standardized data required by the cloud control platform to the cloud control platform.

[0009] In some embodiments of the present application, the data management system also includes: an intelligent driving module, a vehicle control domain module and an intelligent interaction module integrated in the central computing platform, the intelligent driving module, the vehicle control domain module, and the intelligent interaction module are respectively connected to the data center module, and the intelligent driving module, the vehicle control domain module, the intelligent interaction module and the data center module adopt an SPI (Serial Peripheral Interface) communication mode, the intelligent driving module is used to realize the perception, planning and decision-making of the vehicle intelligent driving according to the standardized data output by the data center module, the vehicle control domain module is used to control the whole vehicle body of the vehicle according to the standardized data output by the data center module, and the intelligent interaction module is used to realize the intelligent interaction of the vehicle cabin according to the standardized data output by the data center module.

[0010] In some embodiments of the present application, the sensor collected data includes lidar data, millimeter wave radar data, ultrasonic radar data, camera data, IMU (Inertial Measurement Unit) sensor data, GPS (Global Positioning System) data and V2X (Vehicle to Everything) data; the vehicle internal signal data includes battery status and performance data, environmental perception data, vehicle status data and vehicle behavior data.

[0011] In some embodiments of the present application, the data preprocessing specifically includes: performing data enhancement, filtering and normalization on the multi-source heterogeneous data, and converting the data storage format of the multi-source heterogeneous data; wherein the filtering processing method is one or a combination of several of Kalman filtering, median filtering, and machine learning algorithms.

[0012] In some embodiments of the present application, the preset format conversion processing specifically includes: converting the color format, image format, and pixel bit depth of the video data in the multi-source heterogeneous data into a specified format; the specified format of the color format is one or a combination of YUV444 format, YUV422 format, YUV420 format, and RGB format, and the specified format of the image format is interlaced, planar, or semi-planar.

[0013] In some embodiments of the present application, the feature extraction processing specifically includes: based on the OpenCV (Open Source Computer Vision Library) algorithm library and convolutional neural network, HOG (Histogram of Oriented Gradient) features, LBP (Local Binary Patterns) features, Haar features are extracted from the multi-source heterogeneous data, and / or target detection and extraction algorithms are used to extract targets from the multi-source heterogeneous data.

[0014] In some embodiments of the present application, the data fusion processing specifically includes: performing multi-sensor fusion on the multi-source heterogeneous data based on a random fusion algorithm and an artificial intelligence fusion algorithm, wherein the random fusion algorithm includes a weighted average method, a Kalman filter method, a multi-Bayesian estimation method, a D–S (Dempster–Shafer envidence theory) evidence reasoning method and a production rule, and the artificial intelligence fusion algorithm includes a fuzzy logic thrust algorithm and an artificial neural network algorithm.

[0015] In some embodiments of the present application, the data fusion processing of the visual information data specifically includes:

[0016] Obtain visual image data collected by the camera, point cloud data collected by the laser radar, depth radar image data collected by the millimeter wave radar, and close-range image data collected by the ultrasonic radar;

[0017] According to the weights of different environmental background influencing factors of the visual image data in the application scenario, the visual image data of different environments are weightedly fused to obtain visual fusion data, and the visual fusion data is fused with the point cloud data, the depth radar image data, and the close-range image data to obtain a fused output image; wherein the environmental background influencing factors include illumination factors, road marking line factors, road sign factors, and road obstacle factors, and the calculation formula of the fused output image is:

[0018]

[0019] Wherein, S represents the fusion output image; u(x) represents the visual image data; u(y) represents the point cloud data; u(z) represents the depth radar image data; u(l) represents the close-range image data; n represents the total number of environmental background influencing factors, n=4; W1 represents the weight coefficient of the illumination factor, and its value is determined by the illumination level; W2 represents the weight coefficient of the road marking line factor, and its value is determined by the recognition rate of the road marking line; W3 represents the weight coefficient of the road sign factor, and its value is determined by the recognition rate of the road sign; W4 represents the weight coefficient of the road obstacle factor, and its value is determined by the distance of the obstacle.

[0020] In some embodiments of the present application, the data compression processing specifically includes: performing PCA (Principal Component Analysis) dimensionality reduction processing on the multi-source heterogeneous data, or acquiring and reconstructing sparse signals based on compressed sensing technology; at the same time, using an image reconstruction algorithm to restore and enhance the visual information data.

[0021] In a second aspect, an embodiment of the present application provides a data management method based on an intelligent network-connected centralized electronic and electrical architecture, the data management method comprising:

[0022] Acquire multi-source heterogeneous data generated by each electronic module of the vehicle, classify the multi-source heterogeneous data according to the customized data type, and perform data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the multi-source heterogeneous data after data classification, so as to obtain standardized data required by the central computing platform and each electronic module; wherein the multi-source heterogeneous data includes sensor acquisition data, vehicle internal signal data and cloud data obtained from the cloud control platform, and the customized data type includes visual information data, vehicle behavior data, communication data, positioning and radar data, fault alarm data, cloud interaction data and driver behavior data;

[0023] storing the data generated during the operation of the vehicle and the vehicle behavior data, the fault alarm data, and the driver behavior data processed by the data customization processing unit;

[0024] The standardized data required by the cloud control platform is uploaded to the cloud control platform.

[0025] The innovative features of the embodiments of the present application include but are not limited to the following:

[0026] 1. In this embodiment, a data center module is built into the central computing platform, which is one of the innovative points of the embodiment of this application. It provides a standardized platform for unified reading, unified calling, calculation and storage of data from different modules, and is not only oriented to the future intelligent network centralized electronic and electrical architecture, but also backward compatible with the distributed domain control architecture.

[0027] 2. In this embodiment, the data center module is capable of processing multi-source heterogeneous data and has high computing power. The data volume and data type are sufficient to support AI model training. This is one of the innovative points of the embodiment of the present application, which can reduce the reaction time of autonomous driving at the decision-making level and improve driving safety.

[0028] 3. In this embodiment, the data center module can perform standardized customized processing on the data of different modules, which is one of the innovative points of the embodiment of the present application. It can generate standardized data required by the central computing platform or other modules, and has high data retrieval efficiency.

[0029] 4. In this embodiment, a storage ROM is built into the data center module, which is one of the innovative features of the embodiment of the present application. It compresses data, has high storage efficiency, and can realize classified storage of data, making it convenient for other modules to retrieve data on demand.

[0030] 5. In this embodiment, the data storage unit updates and stores fault alarm data in real time, which is one of the innovative features of the embodiment of the present application. It can obtain fault alarm signals intuitively and simply, improve the efficiency of developers, and clearly troubleshoot faults.

[0031] 6. One of the innovative points of the embodiments of the present application is that data transmission between the cloud control platform and the data center module is realized through the cloud interaction unit. The Internet of Vehicles communication link in the prior art is too lengthy, and is basically vehicle-side MCU (Microcontroller Unit) – domain controller – gateway – Internet of Vehicles intelligent terminal TBOX – (4 / 5G) – cloud (TSP, Telematics Service Provider, automotive remote service provider / cloud platform). Data quality problems are prone to occur during transmission, and the communication efficiency is low. The embodiments of the present application adopt a data center deployment method, and the cloud control platform is connected to the data center module deployed on the central computing platform. The communication link is shortened and the transmission efficiency is high, which effectively solves the problems in the prior art and is conducive to data sampling.

[0032] The beneficial effects of the embodiments of the present application are as follows:

[0033] The central computing platform has a built-in data center module, which can perform customized processing on signal data collected by sensors, data interacting with the central computing platform, data interacting with the cloud control platform, and compressed and stored data, forming an interaction center for vehicle signals and data, and providing a standardized platform for unified reading, unified calling, calculation, and storage of data from different modules. It solves the problems of poor compatibility, weak liquidity, low efficiency, and easy congestion of data flow under a large number of signal interactions, and solves the problem of CPU and GPU resource occupation on the central computing platform, freeing up the CPU's I / O channels, allowing the central computing platform to leave more resources for other applications. The data processed by the data center module is standardized, easy to analyze, easy to store, and highly flexible, which not only improves the efficiency of the entire vehicle system operation, but also helps developers obtain necessary information for R&D verification and engineering testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A structural block diagram of a data center module in a data management system based on an intelligent network-connected centralized electronic and electrical architecture provided in an embodiment of the present application;

[0036] Figure 2 A structural block diagram of a data management system based on an intelligent networked centralized electronic and electrical architecture provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0038] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The terms "including" and "having" in the embodiments of the present application and the accompanying drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0039] The embodiment of the present application discloses a data management system based on an intelligent network-connected centralized electronic and electrical architecture, which is described in detail below.

[0040] Figure 1 and Figure 2 A data management system based on an intelligent network-connected centralized electronic and electrical architecture is shown according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the data management system includes: a data center module 2, an intelligent driving module 3, a vehicle control domain module 4 and an intelligent interaction module 5 integrated in a central computing platform 1. Among them, the data center module 2 classifies, customizes and integrates large-scale data under the large-scale data interaction of vehicle-road-cloud collaboration to realize the automatic driving and intelligent interaction of the vehicle. It is integrated in the central computing platform 1, closely interacts with the central computing platform 1, and forms data intercommunication with other modules, that is, the intelligent driving module 3, the vehicle control domain module 4, and the intelligent interaction module 5 are respectively connected to the data center module 2. The intelligent driving module 3 is used to realize the perception, planning and decision-making of the vehicle intelligent driving according to the standardized data output by the data center module 2. The vehicle control domain module 4 is used to control the whole vehicle body of the vehicle according to the standardized data output by the data center module 2. The intelligent interaction module 5 is used to realize the intelligent interaction of the vehicle cockpit according to the standardized data output by the data center module 2. In some specific embodiments, the intelligent driving module 3, the vehicle control domain module 4, the intelligent interaction module 5 and the data center module 2 adopt the SPI (Serial Peripheral Interface) communication mode, which can transmit and process data in parallel. In addition, as Figure 2 As shown, connected to the central computing platform 1 are vehicle area controllers based on physical partitioning, namely, a first area controller 6 distributed at the left front of the vehicle body, a second area controller 7 distributed at the left rear of the vehicle body, a third area controller 8 distributed at the right front of the vehicle body, and a fourth area controller 9 distributed at the right rear of the vehicle body. In the specific implementation process, the connection form can be a ring network connected by high-speed Ethernet or a point-to-point star network.

[0041] In the embodiment of the present application, a data center module 2 with a data customization processing function is set on the central computing platform 1, which is one of the innovations of the embodiment of the present application. It can process multi-source heterogeneous data at the same time and realize unified data processing. The data center module 2 is a separate data processing module in the data management system, such as Figure 1 As shown, the data center module 2 mainly includes a data customization processing unit 21, a data storage unit 22 and a cloud interaction unit 23. The data customization processing unit 21 is used for customized processing of multi-source heterogeneous data, the data storage unit 22 is used for system data storage, and the cloud interaction unit 23 is used for data transmission between the cloud control platform and the data customization processing unit 21, to obtain the cloud data of the cloud control platform, and to upload the standardized data required by the cloud control platform to the cloud control platform, so that the data center module 2 has three major functions of data customization processing, data storage and cloud interaction, and uniformly manages and processes the data generated by different modules under the collaborative work of the whole vehicle system, so as to solve the problem of system congestion and then collapse caused by data incompatibility, low data retrieval efficiency, storage overflow and other conditions.

[0042] Specifically, the data customization processing unit 21 is used to obtain the multi-source heterogeneous data generated by each electronic module of the vehicle, classify the multi-source heterogeneous data according to the customized data type, and perform data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the multi-source heterogeneous data after data classification to obtain the standardized data required by the central computing platform 1 and each electronic module. Among them, the multi-source heterogeneous data includes sensor acquisition data, vehicle internal signal data and cloud data, that is, the data source of the data center module 2 comes not only from sensors such as laser radar, millimeter wave radar, ultrasonic radar, camera, IMU (Inertial Measurement Unit, inertial measurement unit) sensor, GPS (Global Positioning System, global positioning system) and V2X (Vehicle to Everything, vehicle wireless communication), but also from the signals generated inside the car and the data interacting with the cloud control platform. In some specific embodiments, the vehicle internal signal data includes battery status and performance data, environmental perception data, vehicle status data and vehicle behavior data, and the battery management and charging strategy optimization of the vehicle are performed by collecting and processing the battery status and performance data, and the autonomous decision-making and driving control of the autonomous driving vehicle are supported by collecting and processing the environmental perception data, vehicle status data and vehicle behavior data. In further detail, the sources of multi-source heterogeneous data include but are not limited to: vehicle mode signals, battery power and voltage signals, distance, direction and speed measurement signals, vehicle direction and speed planning based on environmental factors and missing data signals inferred based on experience, longitudinal acceleration signals, engine torque and estimation signals, steering angle and steering wheel angle signals, engine speed, odometer and pedal throttle signals, throttle and brake signals, front and rear wheel speed signals, rear left and right wheel speed signals, car dashboard camera signals, GPS signals, speed, drift and heading angle signals, estimated attitude signals, gyroscope and accelerometer indicator data signals, vertical, horizontal and heading accuracy signals, lane number signals, road type signals, intersection type and distance signals, weather index and road condition signals, wind speed and visibility signals, etc.

[0043] The data customization processing unit 21 can process and analyze a variety of data types to provide more comprehensive information support. In the embodiment of the present application, the customized data types include visual information data, vehicle behavior data, communication data, positioning and radar data, fault alarm data, cloud interaction data, and driver behavior data. The data customization processing unit 21 classifies the acquired multi-source heterogeneous data according to the customized data type to facilitate the subsequent customized processing of the data. Among them, for visual information data, the collected image information is standardized, and the huge video data is processed by dimensionality reduction, video data is compressed, and feature extraction is performed on video frames. Image recognition, target tracking, motion detection and other processing can also be performed; for vehicle behavior data, the entire process of each driving cycle of the car is recorded, including the vehicle status from power-on to power-off, driving cycle, vehicle power data, thermal management information, etc.; for communication data, including data information such as communication quality, rate, signal-to-noise ratio of the Ethernet loop and controller in-loop information, so as to assist in quickly determining which communication fault is caused during fault analysis; for positioning and radar data, the vehicle location information is updated and recorded in a timely manner; for fault alarm data, abnormal data under preset conditions is detected, and stored and reported to relevant personnel outside the vehicle in a timely manner; for cloud interaction data, the data required by the cloud control platform is processed and retrieved, and the data obtained from the cloud control platform can also be processed and input into the vehicle; for driver behavior data, the driver behavior data is recorded to provide training samples for the intelligent cockpit interaction simulator's driving behavior, which are stored in the data center module 2.

[0044] In some embodiments, during the data preprocessing process, the data customization processing unit 21 can perform data enhancement, filtering and normalization processing on the multi-source heterogeneous data to eliminate noise and outliers, realize data filtering, and enable the data center module 2 to have a noise reduction function. In detail, the data enhancement transformation includes image flipping, rotation, scaling, cropping, translation, etc., which can effectively expand the scale of the data set and improve the generalization ability of the model. In addition, the filtering processing method is a combination of one or more of Kalman filtering, median filtering, and machine learning algorithms to use filtering processing to filter out interference signals and improve the signal-to-noise ratio of the signal. The data customization processing unit 21 can also convert the data storage format of multi-source heterogeneous data, for example, converting a 32-bit double-precision floating point type to a 16-bit floating point type, thereby saving memory space and enabling the data center module 2 to have the function of simplifying the complex.

[0045] In the existing technology, different automobile manufacturers and technology suppliers use different data formats and interface standards, which leads to interoperability issues in data collection, making data collection and integration complicated and limiting the effective use and sharing of data.

[0046] In the embodiment of the present application, the preset format conversion processing is one of the main functions of the data customization processing unit 21. The data center module 2 can realize data standardization of multi-source heterogeneous data through customized processing of preset format conversion. Specifically, the data types and entries required by the national standards are uniformly organized, and different processing is performed according to the needs of different data to achieve more extensive data standardization. It provides basic work for the calling, collection and re-injection of other modules, improves the intercommunication and interoperability of data, solves problems in the prior art, and collects data through the Internet of Vehicles or equipment to enable timely transmission of data to specific scenarios such as the cloud. In some specific embodiments, when the data customization processing unit 21 performs preset format conversion processing on the visual information data, the color format, image format, and pixel bit depth of the video data in the multi-source heterogeneous data are converted into a specified format so that the subsequent ISP (Image Signal Process) module, encoder or perception module can efficiently receive and use the output pixels, wherein the specified format of the color format (i.e., data and color format) can be selected from the format in the video codec standard, specifically one or a combination of YUV444 format, YUV422 format, YUV420 format, and RGB format, and the specified format of the image format is interlaced, plane or semi-plane. For the pixel bit depth, the same number of bits of pixels can be used for each color channel, but for the bandwidth constraints in the perception engine and system architecture, each color channel may need to use a different bit depth. In addition, it should be noted that for some color formats, the color information needs to be sub-sampled and scaled to avoid artifacts in the image.

[0047] In addition, the data center module 2 has a feature extraction function, that is, using a deep learning model to automatically mine relevant features from multimodal data. In some specific embodiments, the feature extraction process specifically includes: based on the OpenCV (OpenSource Computer Vision Library) algorithm library and convolutional neural network, HOG (Histogram of Oriented Gradient) features, LBP (Local Binary Patterns) features, Haar features are extracted from multi-source heterogeneous data, and / or the target detection extraction algorithm is used to extract targets from multi-source heterogeneous data, omitting unwanted image information, wherein the target detection extraction algorithm can use an open source convolutional neural network. For ordinary scenes, a first-order YOLO (You Only Look Once: Unified, Real-Time Object Detection) model can be used for iteration, and its objective function for:

[0048]

[0049] in, represents the category loss, represents the intersection-over-union loss, represents the confidence loss, and λ is The balance coefficient of The balance coefficient.

[0050] In the embodiment of the present application, data fusion processing is also one of the main functions of the data customization processing unit 21, which fuses data information from different sensors and modules according to a preset algorithm. These data information include but are not limited to laser radar information, millimeter wave radar information, ultrasonic radar information, visual information, positioning information, driver information, map and environmental information, etc. The data customization processing unit 21 can efficiently integrate the data together through data fusion processing, while minimizing noise, redundancy and inconsistency. In some embodiments, the data fusion processing specifically includes: based on random fusion algorithms and artificial intelligence fusion algorithms, multi-sensor fusion of multi-source heterogeneous data, wherein the random fusion algorithms include weighted averaging, Kalman filtering, multi-Bayesian estimation, D-S (Dempster-Shafer envidence theory) evidence reasoning and production rules, and the artificial intelligence fusion algorithms include fuzzy logic thrust algorithms and artificial neural network algorithms.

[0051] Furthermore, in some specific implementation processes, the data fusion processing of the visual information data specifically includes the following steps:

[0052] A. Obtain visual image data collected by the camera, point cloud data collected by the lidar, depth radar image data collected by the millimeter wave radar, and close-range image data collected by the ultrasonic radar.

[0053] In the embodiment of the present application, the data sources of visual information data fusion are mainly laser radar, millimeter wave radar, camera and ultrasonic radar. Among them, the laser radar collects point cloud data, the millimeter wave radar collects depth radar information containing angular velocity, that is, depth radar image data, the camera collects image information, that is, visual image data, and the ultrasonic radar collects obstacle information, that is, close-range image data.

[0054] B. According to the weights of different environmental background influencing factors of visual image data in the application scenario, weighted fusion is performed on visual image data of different environments to calculate visual fusion data, and the visual fusion data is fused with point cloud data, depth radar image data, and close-range image data to calculate a fused output image; wherein, environmental background influencing factors include illumination factors, road marking line factors, road sign factors, and road obstacle factors, and the calculation formula for the fused output image is:

[0055]

[0056] Wherein, S represents the fusion output image; u(x) represents visual image data; u(y) represents point cloud data; u(z) represents depth radar image data; u(l) represents close-range image data; n represents the total number of environmental background influencing factors, n=4; W1 represents the weight coefficient of the illumination factor, and its value is determined by the illumination level; W2 represents the weight coefficient of the road marking line factor, and its value is determined by the recognition rate of the road marking line; W3 represents the weight coefficient of the road sign factor, and its value is determined by the recognition rate of the road sign; W4 represents the weight coefficient of the road obstacle factor, and its value is determined by the distance of the obstacle.

[0057] In the embodiment of the present application, the data fusion of visual information data mainly considers two factors, namely, complex background factors and three-dimensional information factors (i.e., information collected by laser radar, millimeter-wave radar, camera, and ultrasonic radar). The camera is mainly used to identify the color of objects, but it is affected by rainy weather, that is, affected by light. The millimeter-wave radar can make up for the disadvantage of the camera being affected by rainy days, and can identify obstacles at a relatively long distance, such as pedestrians, roadblocks, etc., but cannot identify the specific shape of the obstacles. The laser radar can make up for the disadvantage that the millimeter-wave radar cannot identify the specific shape of the obstacles. In addition, the ultrasonic radar can identify close obstacles of the vehicle body, so that a panoramic perception can be formed by fusing the external data collected by different sensors. Further, since the visual image data collected by the camera is affected by light, the complex background factor mainly considers the light intensity and the distance of the obstruction to improve the accuracy of the fusion. As shown in the calculation formula of the above-mentioned fusion output image, the application scenarios are divided into several categories, and the fusion weight distribution of the visual image data of each category of application scenarios is different. The illumination factor is determined by the illumination intensity and the level is determined by the photosensitive sensor. It can be divided into three levels: low illumination, medium illumination and high illumination. The visual image data of the low illumination level application scene needs to be integrated with more point cloud data to overcome the influence of the illumination factor on the object color recognition. That is, the higher the illumination intensity, the less the visual image data is affected. Therefore, the weight coefficient W1 of the visual image data of the three levels of low illumination, medium illumination and high illumination application scenes increases. The road marking line factor is determined by the recognition rate of the road marking line in the visual image data. If the road marking line information obtained through the visual image data is unclear, specifically, the recognition rate of the road marking line is lower than 80%, it is necessary to integrate the deep radar image data to obtain more complete road marking line information. That is to say, the higher the recognition rate of the road marking line, the less the visual image data is affected. Therefore, the weight coefficient W2 of the road marking line factor can be positively correlated with the recognition rate of the road marking line. The road sign factor is determined by the recognition rate of the road sign. In the process of road sign identification, it is mainly determined by visual image data. The higher the recognition rate of the road sign, the higher the completeness of the fused road sign information. Therefore, the weight coefficient W3 of the road sign factor can also be positively correlated with the recognition rate of the road sign. The road obstacle factor is determined by the size of the obstacle distance. The obstacle is detected by visual image data, but the approximate distance of the obstacle is determined by the close-range image data collected by the ultrasonic radar. If the obstacle distance is determined to be medium or short, the depth radar image data collected by the millimeter wave radar can also be integrated to determine the position and speed of the obstacle. In other words, the closer the obstacle distance, the higher the reliability of the visual image data. Therefore, the weight coefficient W4 of the road obstacle factor is negatively correlated with the size of the obstacle distance.

[0058] In the calculation of the lidar u(y), the full-view information obtained by the lidar is the information of the three-dimensional space coordinates represented by the point cloud. Therefore, the data fusion of the lidar and the camera is actually the combination of the depth formed by the projection of the lidar point cloud on the camera image plane and the depth estimated by the image. The calibration of the lidar and the camera is completed synchronously, so the lidar point cloud can be calibrated and projected onto the camera's image plane to form a relatively sparse depth map. In this process, the point cloud data needs to be projected to form a bird's-eye view and a front view, etc. After that, CNN (Convolutional Neural Networks) in 2D is used for feature extraction, and then the point cloud is converted into RGBMap (RGB color map, specified as a c×3 numerical matrix composed of values ​​in the range [0,1]) by projection to complete the feature output.

[0059] In the present application, the image fusion and feature extraction method plays an important role in the data processing level of the data center module 2. The method makes a judgment on the application scenario and classifies the scenes with complex backgrounds. Different image fusion methods are applied in different classification scenarios. It is mainly to superimpose visual image data with different weights according to the reliability of the image data, and finally form a more reliable fusion effect.

[0060] In addition, the data customization processing unit 21 also has a data compression processing function. In some embodiments, the data compression processing specifically includes: performing PCA (Principal Component Analysis) dimensionality reduction processing on multi-source heterogeneous data, or acquiring and reconstructing sparse signals based on compressed sensing technology; at the same time, using an image reconstruction algorithm to restore and enhance the visual information data. In detail, the data customization processing unit 21 performs sparse representation processing on the data through PCA dimensionality reduction processing, which can effectively compress the data and reduce the amount of model calculation for establishing the autonomous driving mode. It can also manage data through compressed sensing technology and effectively solve the image restoration problem in the case of few samples. In addition, the data customization processing unit 21 can also restore and enhance the image data through image reconstruction algorithms such as super-resolution reconstruction. The data center module 2 effectively compresses the data and can also reduce the storage pressure on the vehicle side.

[0061] The data storage unit 22 is used to store the data generated by the vehicle during operation, and to store the vehicle behavior data, fault alarm data and driver behavior data processed by the data customization processing unit 21. It should be understood that the "data generated by the vehicle during operation" in this application not only includes the cached data during the interaction between the vehicle and the cloud control platform, but also includes the data required to be stored by the vehicle itself. In some embodiments, the data storage unit 22 may be a ROM (Read-Only Memory) used to store specific data, which is convenient for other modules to retrieve when there is a data demand. The data center module 2 has a built-in ROM, and the system can store data freely. It can automatically save the data that the vehicle needs to record automatically, and each module can retrieve the required data. In the behavior of interacting with the cloud control platform, the car usually generates a large amount of cached data, which is temporarily cached in the memory when there is no V2X or 5G signal. At this time, the ROM of the data center module 2 can assume the obligation of data caching to avoid data loss when there is no signal and the data to be uploaded cannot be processed well.

[0062] In this application, the data center module 2 collects data from sensors and single ECU calculations. For basic data, sensor data is directly collected and obtained directly from the bus. For data calculated by a single ECU, the data is usually cumbersome and the data volume is huge. It is usually calculated by the controller module itself and obtained from the central gateway. The data interaction between the data center module 2 and the cloud control platform is collected or sent by the vehicle-mounted OBU (T-BOX). The data center module 2 serves as a data transmission transfer station for the data management system to achieve unified scheduling and standardization of data. It performs standardized and customized processing on the collected data. These standardized and customized data are usually data required by other modules in the central computing platform 1. For example, the data generated by the intelligent driving module 3 is required by both the cloud control platform and the intelligent interaction module 5. The cloud control platform needs data after feature extraction, and the intelligent interaction module 5 needs data after compression. In this way, the data center module 2 provides the required data for other modules in the central computing platform 1. Data that cannot be transmitted temporarily will be stored in the data storage unit 22. For example, the driving data and driver data of the vehicle need to be recorded. Data that cannot be sent through V2X or 5G when the signal is not good can be compressed and cached in the data storage unit 22 to avoid data loss.

[0063] Corresponding to the above system embodiment, another embodiment of the present application provides a data management method based on an intelligent network-connected centralized electronic and electrical architecture, the data management method comprising the following steps:

[0064] Step S110, acquiring multi-source heterogeneous data generated by each electronic module of the vehicle, classifying the multi-source heterogeneous data according to the customized data type, and performing data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the classified multi-source heterogeneous data to obtain standardized data required by the central computing platform 1 and each electronic module; wherein the multi-source heterogeneous data includes sensor acquisition data, vehicle internal signal data and cloud data obtained from the cloud control platform, and the customized data type includes visual information data, vehicle behavior data, positioning and radar data, fault alarm data, cloud interaction data and driver behavior data;

[0065] Step S120, storing the data generated during the operation of the vehicle and the vehicle behavior data, fault alarm data, and driver behavior data processed by the data customization processing unit 21;

[0066] Step S130, uploading the standardized data required by the cloud control platform to the cloud control platform.

[0067] It should be noted that the method embodiment corresponds to the aforementioned system embodiment and has the same technical effect as the aforementioned system embodiment. Please refer to the system embodiment, and the method embodiment is obtained based on the system embodiment. Please refer to the system embodiment part for specific descriptions, which will not be repeated here.

[0068] In summary, the embodiments of the present application provide a data management system and method based on an intelligent network-connected centralized electronic and electrical architecture. A data center module is built into the central computing platform, which can customize the signal data collected by sensors, the data interacting with the central computing platform, the data interacting with the cloud control platform, and the compressed and stored data to form an interaction center for automobile signals and data, and provide a standardized platform for unified reading, unified calling, calculation and storage of data from different modules, solving the problems of poor compatibility, weak circulation, low efficiency, and easy congestion of data flow under a large number of signal interactions, and solving the problem of resource occupation of the central computing platform CPU (Central Processing Unit) and GPU (Graphics Processing Unit), freeing up the CPU's I / O channels, so that the central computing platform can leave more resources for other applications, and the data processed by the data center module has the characteristics of standardization, easy analysis, easy storage, and strong flexibility, which not only improves the efficiency of the operation of the entire vehicle system, but also helps developers obtain necessary information for R&D verification and engineering testing.

[0069] Those skilled in the art will understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. In addition, the modules in the embodiment device can be distributed in the embodiment device according to the embodiment description, or can be changed accordingly and located in one or more devices different from the present embodiment. The modules of the above-mentioned embodiments can be combined into one module, or can be further split into multiple sub-modules.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data management system based on intelligent network-connected centralized electronic and electrical architecture, characterized in that: The data management system includes: a data center module integrated in a central computing platform, the data center module including a data customization processing unit, a data storage unit and a cloud interaction unit; The data customization processing unit is used to obtain multi-source heterogeneous data generated by each electronic module of the vehicle, classify the multi-source heterogeneous data according to the customized data type, and perform data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the multi-source heterogeneous data after data classification, so as to obtain the standardized data required by the central computing platform and each electronic module; wherein the multi-source heterogeneous data includes sensor acquisition data, vehicle internal signal data and cloud data, and the customized data type includes visual information data, vehicle behavior data, communication data, positioning and radar data, fault alarm data, cloud interaction data and driver behavior data; The data storage unit is used to store the data generated by the vehicle during operation, and to store the vehicle behavior data, the fault alarm data and the driver behavior data processed by the data customization processing unit; The cloud interaction unit is used for data transmission between the cloud control platform and the data customization processing unit, to obtain the cloud data of the cloud control platform, and to upload the standardized data required by the cloud control platform to the cloud control platform.

2. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1 is characterized in that: The data management system also includes: an intelligent driving module, a vehicle control domain module and an intelligent interaction module integrated in the central computing platform, the intelligent driving module, the vehicle control domain module and the intelligent interaction module are respectively connected to the data center module, and the intelligent driving module, the vehicle control domain module, the intelligent interaction module and the data center module adopt an SPI communication mode, the intelligent driving module is used to realize the perception, planning and decision-making of the intelligent driving of the vehicle according to the standardized data output by the data center module, the vehicle control domain module is used to control the whole vehicle body of the vehicle according to the standardized data output by the data center module, and the intelligent interaction module is used to realize the intelligent interaction of the vehicle cabin according to the standardized data output by the data center module.

3. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The sensor collected data includes lidar data, millimeter wave radar data, ultrasonic radar data, camera data, IMU sensor data, GPS data and V2X data; the vehicle internal signal data includes battery status and performance data, environmental perception data, vehicle status data and vehicle behavior data.

4. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The data preprocessing specifically includes: performing data enhancement, filtering and normalization processing on the multi-source heterogeneous data, and converting the data storage format of the multi-source heterogeneous data; wherein the filtering processing method is one or a combination of several of Kalman filtering, median filtering, and machine learning algorithms.

5. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The preset format conversion processing specifically includes: converting the color format, image format, and pixel bit depth of the video data in the multi-source heterogeneous data into a specified format; the specified format of the color format is one or a combination of YUV444 format, YUV422 format, YUV420 format, and RGB format, and the specified format of the image format is interlaced, planar, or semi-planar.

6. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The feature extraction process specifically includes: extracting HOG features, LBP features, and Haar features from the multi-source heterogeneous data based on the OpenCV algorithm library and convolutional neural network, and / or extracting targets from the multi-source heterogeneous data using a target detection extraction algorithm.

7. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The data fusion processing specifically includes: performing multi-sensor fusion on the multi-source heterogeneous data based on a random fusion algorithm and an artificial intelligence fusion algorithm, wherein the random fusion algorithm includes a weighted average method, a Kalman filter method, a multi-Bayesian estimation method, a D-S evidence reasoning method and a production rule, and the artificial intelligence fusion algorithm includes a fuzzy logic thrust algorithm and an artificial neural network algorithm.

8. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The data fusion processing of the visual information data specifically includes: Obtain visual image data collected by the camera, point cloud data collected by the laser radar, depth radar image data collected by the millimeter wave radar, and close-range image data collected by the ultrasonic radar; According to the weights of different environmental background influencing factors of the visual image data in the application scenario, the visual image data of different environments are weightedly fused to obtain visual fusion data, and the visual fusion data is fused with the point cloud data, the depth radar image data, and the close-range image data to obtain a fused output image; wherein the environmental background influencing factors include illumination factors, road marking line factors, road sign factors, and road obstacle factors, and the calculation formula of the fused output image is: Among them, S represents the fused output image; u(x) represents visual image data; u(y) represents point cloud data; u(z) represents depth radar image data; u(l) represents close-range image data; n represents the total number of environmental background influencing factors, n=4; W1 represents the weight coefficient of the illumination factor, and its value is determined by the illumination level; W2 represents the weight coefficient of the road marking line factor, and its value is determined by the recognition rate of the road marking line; W3 represents the weight coefficient of the road sign factor, and its value is determined by the recognition rate of the road sign; W4 represents the weight coefficient of the road obstacle factor, and its value is determined by the distance of the obstacle.

9. The data management system based on intelligent network-connected centralized electronic and electrical architecture according to claim 1, characterized in that: The data compression processing specifically includes: performing PCA dimensionality reduction processing on the multi-source heterogeneous data, or acquiring and reconstructing sparse signals based on compressed sensing technology; at the same time, using an image reconstruction algorithm to restore and enhance the visual information data.

10. A data management method based on intelligent network-connected centralized electronic and electrical architecture, characterized in that: The data management method comprises: Acquire multi-source heterogeneous data generated by each electronic module of the vehicle, classify the multi-source heterogeneous data according to the customized data type, and perform data preprocessing, preset format conversion processing, feature extraction processing, data fusion processing and data compression processing on the multi-source heterogeneous data after data classification, so as to obtain standardized data required by the central computing platform and each electronic module; wherein the multi-source heterogeneous data includes sensor acquisition data, vehicle internal signal data and cloud data obtained from the cloud control platform, and the customized data type includes visual information data, vehicle behavior data, communication data, positioning and radar data, fault alarm data, cloud interaction data and driver behavior data; storing the data generated during the operation of the vehicle and the vehicle behavior data, the fault alarm data, and the driver behavior data processed by the data customization processing unit; The standardized data required by the cloud control platform is uploaded to the cloud control platform.

Images