Vehicle early warning method and device facing multi-source heterogeneous data collaboration, equipment and medium

Abstract

This application discloses a vehicle early warning method, device, equipment, and medium for multi-source heterogeneous data collaboration, relating to the field of computer technology. The method includes: receiving multi-source heterogeneous vehicle data reported by multi-source on-board devices and parsing the data to determine its corresponding data type; storing the multi-source heterogeneous vehicle data in a corresponding storage area of ​​a preset data lake according to its data type, and performing semantic analysis on the data according to a preset semantic analysis model to determine several corresponding data semantics; matching the data semantics with a preset alarm rule base; and if the data semantics successfully match a target alarm rule in the preset alarm rule base, then issuing a vehicle early warning based on the target alarm rule. Thus, multi-source heterogeneous data can be converted into a unified format of data semantics, giving previously discrete data computable attributes, thereby realizing vehicle early warning based on multi-source heterogeneous data.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a vehicle early warning method, device, equipment, and medium for multi-source heterogeneous data collaboration. Background Technology

[0002] The classic architecture of vehicle-to-everything (V2X) safety operation platforms has been solidified into a three-stage process: "MQTT / Kafka access, Flink streaming deduplication / rule collision, and Doris alarm storage." This process only handles structured or semi-structured security events such as CAN (Controller Area Network) messages, Ethernet images, and system logs. Unstructured data such as images, audio, and video are typically discarded or only cold-stored due to significant format differences and a lack of a unified semantic model, and cannot participate in real-time correlation analysis.

[0003] Current multi-source heterogeneous automotive security data fusion solutions only reach the text alert stage and do not involve content parsing of multimodal data or cross-modal semantic alignment mechanisms. When attack behavior involves unstructured data of various forms, detection blind spots are likely to occur. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a vehicle early warning method, device, equipment, and medium for multi-source heterogeneous data collaboration. This method can convert multi-source heterogeneous data into a unified data semantic format, enabling previously discrete data to acquire computable attributes, thereby achieving vehicle early warning based on multi-source heterogeneous data. The specific solution is as follows: Firstly, this application discloses a vehicle early warning method for multi-source heterogeneous data collaboration, applied to a vehicle networking platform, including: Receive multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices, and parse the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data; The multi-source heterogeneous vehicle data is stored in the corresponding storage area of ​​the preset data lake according to the data type, and semantic analysis is performed on the multi-source heterogeneous vehicle data according to the preset semantic analysis model to determine several data semantics of the multi-source heterogeneous vehicle data. The data semantics are matched with a preset alarm rule library. If the data semantics successfully match the target alarm rule in the preset alarm rule library, a vehicle warning is issued based on the target alarm rule.

[0005] Optionally, the step of receiving multi-source heterogeneous vehicle data reported by multi-source on-board devices and parsing the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data includes: The system receives multi-source heterogeneous vehicle data reported by multiple vehicle-mounted devices through a preset multi-protocol gateway; the multi-source heterogeneous vehicle data refers to vehicle data in different formats collected by different vehicle-mounted devices. The decoder in the preset multi-protocol gateway parses the multi-source heterogeneous vehicle data to identify the data type corresponding to the multi-source heterogeneous vehicle data.

[0006] Optionally, storing the multi-source heterogeneous vehicle data into a corresponding storage area in a preset data lake according to the data type includes: Based on the data type, the multi-source heterogeneous data is classified, and the structured data in the multi-source heterogeneous data is stored in a structured data stream; And, write the binary objects in the multi-source heterogeneous data into a preset original data storage bucket; And, write the metadata events in the multi-source heterogeneous data into a preset metadata stream; And, extract the text key-value pairs from the text logs in the multi-source heterogeneous data, and write the text key-value pairs into the text data stream.

[0007] Optionally, the step of performing semantic analysis on the multi-source heterogeneous vehicle data according to a preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data includes: The multi-source heterogeneous vehicle data is augmented to obtain augmented data; The data type of the enhanced data is determined, and corresponding semantic analysis is performed on the enhanced data based on the data type to determine several data semantics corresponding to the multi-source heterogeneous vehicle data.

[0008] Optionally, determining the data type of the enhanced data and performing corresponding semantic analysis on the enhanced data based on the data type to determine several data semantics corresponding to the multi-source heterogeneous vehicle data includes: The enhanced data is analyzed by a preset semantic analysis model. If the enhanced data is image data, inference is performed on the image data to determine the image data features corresponding to the image data. If the enhanced data is audio data, the audio data is converted into corresponding audio text data, and keyword matching is performed on the text data to determine the audio text features corresponding to the text data. If the enhanced data is video data, then the video data is subjected to frame extraction processing, and the obtained video frames are inferred to determine the video features corresponding to the video frames. If the enhanced data is text data, then the text data is classified to determine the text features corresponding to the text data; Data semantics in a preset format are constructed based on the image data features, and / or the audio text features, and / or the video features, and / or the text features.

[0009] Optionally, matching the data semantics with a preset alarm rule base, and if the data semantics successfully match a target alarm rule in the preset alarm rule base, then issuing a vehicle warning based on the target alarm rule, includes: The data semantics are sent to a preset neural network model for analysis to determine several prediction probabilities corresponding to the data semantics and each alarm rule in the preset alarm rule base. Determine the target prediction probability with the highest probability value among the several predicted probabilities. If the target probability value corresponding to the target prediction probability is not less than a preset probability threshold, then the alarm rule corresponding to the target prediction probability is taken as the target alarm rule, and vehicle warning is performed based on the target alarm rule.

[0010] Optionally, after matching the data semantics with a preset alarm rule base, and if the data semantics successfully match a target alarm rule in the preset alarm rule base, the process further includes: If the response duration of the preset neural network model is greater than the preset response duration threshold, then the confidence value corresponding to the data semantics is determined based on the preset rule confidence value, and whether to issue a vehicle warning is determined based on the confidence value.

[0011] Secondly, this application discloses a vehicle early warning device for multi-source heterogeneous data collaboration, applied to a vehicle networking platform, comprising: The data parsing module is used to receive multi-source heterogeneous vehicle data reported by multi-source on-board equipment, and to parse the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data. The semantic analysis module is used to store the multi-source heterogeneous vehicle data into the corresponding storage area in the preset data lake according to the data type, and to perform semantic analysis on the multi-source heterogeneous vehicle data according to the preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data. The vehicle warning module is used to match the data semantics with a preset alarm rule library. If the data semantics successfully match the target alarm rule in the preset alarm rule library, a vehicle warning is issued based on the target alarm rule.

[0012] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute the computer program to implement the vehicle early warning method for multi-source heterogeneous data collaboration as described above.

[0013] Fourthly, this application discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned vehicle early warning method for multi-source heterogeneous data collaboration.

[0014] In this application, multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices can be received and parsed to determine the data type corresponding to the multi-source heterogeneous vehicle data; the multi-source heterogeneous vehicle data is stored in the corresponding storage area of ​​a preset data lake according to the data type, and semantic analysis is performed on the multi-source heterogeneous vehicle data according to a preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data; the data semantics are matched with a preset alarm rule base, and if the data semantics successfully match the target alarm rule in the preset alarm rule base, a vehicle warning is issued based on the target alarm rule.

[0015] Therefore, the method described in this application requires analyzing multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices after receipt. Then, based on the data type determined by the analysis, the multi-source heterogeneous vehicle data is stored in the corresponding storage area of ​​a pre-defined data lake. After storage, semantic analysis is performed on the multi-source heterogeneous vehicle data to determine its corresponding semantics. This semantics is then matched against a pre-defined alarm rule database, and finally, vehicle warnings are issued based on the successfully matched target alarm rules. This allows data of any format to be converted into a unified semantic object and enables cross-modal data to be correlated. This transforms previously discarded or isolated multimedia data into analyzable data semantics, thereby achieving collaboration of multi-source heterogeneous data and improving the reliability of vehicle warnings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a flowchart of a vehicle early warning method for multi-source heterogeneous data collaboration disclosed in this application; Figure 2 This is a schematic diagram of an early warning process disclosed in this application; Figure 3This application discloses a schematic diagram of a vehicle network platform processing principle. Figure 4 This is a schematic diagram of a multi-protocol gateway processing principle disclosed in this application; Figure 5 This is a schematic diagram of a vehicle early warning device for multi-source heterogeneous data collaboration disclosed in this application; Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Current multi-source heterogeneous automotive security data fusion solutions only reach the text alert stage and do not involve content parsing of multimodal data or cross-modal semantic alignment mechanisms. When attack behavior involves unstructured data of various forms, detection blind spots are likely to occur.

[0020] To overcome the aforementioned technical problems, this application discloses a vehicle early warning method, device, equipment, and medium for multi-source heterogeneous data collaboration, which can convert multi-source heterogeneous data into data semantics in a unified format, enabling the originally discrete data to acquire computable attributes, thereby realizing vehicle early warning based on multi-source heterogeneous data.

[0021] See Figure 1 As shown, this embodiment of the invention discloses a vehicle early warning method for multi-source heterogeneous data collaboration, applied to a vehicle networking platform, including: Step S11: Receive multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices, and parse the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data.

[0022] In this embodiment, as Figure 2 As shown, after the on-board devices report multi-source heterogeneous vehicle data to the vehicle-to-everything (V2X) platform, the multi-source heterogeneous data needs to be parsed to determine its corresponding data type. Specifically, a pre-set multi-protocol gateway is needed to receive the multi-source heterogeneous vehicle data reported by the multi-source on-board devices; the multi-source heterogeneous vehicle data refers to vehicle data in different formats collected by different on-board devices. It should be noted that the V2X platform is an enhanced VSOC (Vehicle Security Operations Center) system, such as... Figure 3As shown, the system is divided into five vertical functional zones. The left side uses a unidirectional arrow to indicate the data flow direction, and the right side uses a bidirectional arrow to indicate the control / configuration loop.

[0023] It should be further explained that the default multi-protocol gateway supports multiple protocols, including but not limited to MQTT (Message Queuing Telemetry Transport), HTTP (Hypertext Transfer Protocol), TCP (Transmission Control Protocol), and S3 (Simple Storage Service). Taking MQTT data as an example, the data would be as follows: { "head":{ "msgId":"1", "time":"2026-05-06 00:00:00", "vin":"virtual vehicle identification number" }, "data":"123" }

[0024] In the code above, the "virtual VIN" is an encrypted real VIN. Its function in the code is only to interpret parameters, not to be used as a real parameter.

[0025] It should be noted that by parsing multi-source heterogeneous vehicle data, the data type corresponding to the multi-source heterogeneous vehicle data can be determined based on a format recognizer. The data parsing process is as follows: Figure 4 As shown, data parsing requires a protocol decoder, implemented using Netty 4.x. Both MQTT and HTTP reuse the same EventLoop thread pool, reducing context switching. Different decoding is performed based on the protocol and format of the reported data. Taking image and audio data as examples: Image sample data: { "source": "camera_front", "protocol": "RTSP", "format": "H.264", "payload": "0x4D5A9F...", "timestamp": 1746601333210 }; Protocol decoding: RTSP → RTP; Format parsing: H.264 → YUV; Content Extraction: Engine vibration, low tachometer reading.

[0026] Audio sample data: { "source": "mic_driver", "protocol": "ALSA", "format": "PCM 16bit 16kHz", "payload": "0x7E3B21...", "timestamp": 1746601333650 }; Protocol decoding: ALSA frame synchronization; Format parsing: PCM → Mel; Content extraction: Abnormal low-frequency jitter.

[0027] It should be further noted that the Netty gateway in this embodiment can be replaced by the open-source Nginx+Stream Lua module, which uses lua-resty-kafka and lua-resty-mqtt to perform protocol parsing and libmagic to complete the format recognition.

[0028] In this way, data parsing through a multi-protocol gateway can effectively improve data processing efficiency.

[0029] Step S12: Store the multi-source heterogeneous vehicle data in the corresponding storage area of ​​the preset data lake according to the data type, and perform semantic analysis on the multi-source heterogeneous vehicle data according to the preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data.

[0030] In this embodiment, multi-source heterogeneous vehicle data needs to be stored in corresponding storage areas within a pre-defined data lake according to data type. Specifically, for example... Figure 4As shown, it is necessary to classify multi-source heterogeneous data based on data type. Here, "major / minor" refers to the format structure of MIME (Multipurpose Internet Mail Extensions) types: Major type: represents the major category of data, such as text, image, application, etc. Minor type: the specific format under the major type, such as html, jpeg, json, etc. The two combine to form a complete MIME type, with the format: major / minor, for example: text / html, image / jpeg, application / json.

[0031] Furthermore, the structured data from the multi-source heterogeneous data needs to be stored in a structured data stream via a content dispatcher. Specifically, it needs to be stored in the `struct_stream` within the Kafka Topic, i.e., the structured data stream. Binary objects from the multi-source heterogeneous data need to be written to a pre-defined raw data storage bucket. Specifically, the binary objects need to be written to the data lake's `raw_bucket`. Metadata events from the multi-source heterogeneous data need to be written to a pre-defined metadata stream. Additionally, text key-value pairs (key=value) need to be extracted from the text logs in the multi-source heterogeneous data and written to the text data stream `text_stream`. It should be noted that the data lake is divided into three levels of storage: raw area, semantic area, and index area. The raw area only performs append-only writes, the semantic area stores the JSON (JavaScript Object Notation) features after AI (Artificial Intelligence) inference, and the index area provides row-level storage acceleration, balancing compliant archiving and real-time querying, to overcome the lack of semantic meaning in cold data in existing Hadoop lakes.

[0032] The next step requires data augmentation and semantic analysis of the augmented data. Specifically, data augmentation is needed for multi-source heterogeneous vehicle data to obtain augmented data; the data type of the augmented data is determined, and corresponding semantic analysis is performed based on the data type to identify several semantic meanings of the multi-source heterogeneous vehicle data. Augmentation is implemented through AI semantic augmentation ETL (Extract-Load-Transform), using an Airflow DAG (Directed Acyclic Graph) format, with a cycle of 1 minute and 3 retries on failure. This involves scanning incremental original files and then analyzing the augmented data using a pre-defined semantic analysis model. If the augmented data is image data, inference is performed on the image data to determine its corresponding image features, primarily using the YOLOv8 (You Only Look Once v8) model for image inference. If the augmented data is audio data, Whisper transformation is performed on the audio data to convert it into corresponding audio-text data, and keyword matching is performed on the text data to determine its corresponding audio-text features. If the enhanced data is video data, then FFmpeg (Fast Forward Moving Picture Experts Group) is used to extract frames from the video data, and YOLOv8 is used to infer the video frames to determine the corresponding video features. If the enhanced data is text data, then BERT (Bidirectional Encoder Representations from Transformers) is used to classify the text data to determine the corresponding text features. Data semantics in a preset format are constructed based on image data features, and / or audio text features, and / or video features, and / or text features. The preset format is JSON, and the output JSON data semantics format is as follows: {file_id, media_type, timestamp, objects[], transcript[], threat_score}. Taking image data enhancement as an example, the process is as follows: { "source": "camera_front", "protocol": "RTSP", "format": "H.264", "payload": "0x4D5A9F...", "timestamp": 1746601333210, "content": "Engine vibration, low tachometer reading" } Generate feature vectors based on the engine rotation and tachometer features in the content. { "semantic_vector": [0.031, -0.128, 0.842, ..., 0.476], "dimension": 768, "confidence": 0.93, "label": "engine_stall_risk" }

[0033] Furthermore, YOLOv8, Whisper, and Distilled BERT can be encapsulated as gRPC (Google Remote Procedure Call) microservices. These can be scheduled on demand via Airflow DAG to perform image object detection, audio transcription and keyword matching, and text threat classification, outputting unified Avro Schema semantic events. This fills the gap where unstructured content cannot enter the real-time security pipeline. Moreover, cloud vendor serverless functions can replace Kubernetes microservices, with event-triggered inference and results written back to Kafka.

[0034] Step S13: Match the data semantics with the preset alarm rule library. If the data semantics successfully match the target alarm rule in the preset alarm rule library, then issue a vehicle warning based on the target alarm rule.

[0035] In this embodiment, the data semantics stored in the data lake need to be matched with a preset alarm rule base. If the data semantics successfully match the target alarm rule in the preset alarm rule base, a vehicle warning is issued based on the target alarm rule. Specifically, the data semantics need to be sent to a preset neural network model for analysis to determine several predicted probabilities corresponding to each alarm rule in the preset alarm rule base. The target predicted probability with the highest probability value among the several predicted probabilities is determined. If the target probability value is not less than a preset probability threshold, the alarm rule corresponding to the target predicted probability is taken as the target alarm rule, and a vehicle warning is issued based on the target alarm rule.Taking the joint alarm scenario of "camera detects occlusion" and "CAN message abnormal unlock" as an example, at the event starting point T0, the camera deployed on the vehicle captures video frame #F, which is processed by the access gateway and written to the raw_bucket object storage; at the same time, the CAN bus continuously collects messages from each ECU and writes them to the struct_stream message queue; all data is globally timestamped with T0 as the reference when it is written; visual semantic analysis is performed on video frame #F in raw_bucket, and a semantic event E1 is generated at time T0+200, whose data structure is {type: "image", objects: ["face_occlusion"], score: 0.8}; the CAN message in struct_stream is parsed in real time, and at time T0+220, {id: 0x421, value: 0xFF, flag:} is extracted from CAN message C1. "unlock"} generates a structured event stream; at time T0+250, the rule engine layer performs sliding window pattern matching on the structured event stream generated in step S2 based on the preset CEP rule base. When the message id=0x421 and flag="unlock" is detected and meets the abnormal unlocking rule, a primary alarm event A1 is generated; at time T0+300, the fusion inference layer receives the semantic event E1 generated in step S2 and the primary alarm event A1 generated in step S3, maps the two to node feature vectors in the heterogeneous graph neural network, calculates the spatiotemporal relationship features and then performs multi-layer graph convolution operation; at time T0+400, the graph neural network outputs the association relationship edge E1-[:close_to]-A1 between E1 and A1, with a comprehensive confidence level of 0.92; at time T0+450, the event Based on the correlation output in step S4, the response layer merges semantic event E1 with primary alarm event A1 to generate a higher-order composite event H1, named "Suspected Unauthorized Intrusion," with a CRITICAL level. At time T0+500, the event response layer executes a tiered response strategy based on the CRITICAL level determined in step S5: a visual pop-up alarm is displayed on the vehicle's central control screen, a door locking command is issued via SOAR orchestration, and video data from 30 seconds before and after the event is extracted and uploaded to the cloud. At time T0+600, the event response layer writes the higher-order composite event H1 generated in step S5 into the Doris distributed relational database for persistent storage, and simultaneously writes H1 and its constituent events E1 and A1 into the knowledge graph in the form of knowledge triples, triggering incremental learning and updates of the graph neural network model. Example data is shown below. { "semantic_vector": [0.031, -0.128, 0.842, ..., 0.476], "dimension": 768, "confidence": 0.93, "label": "engine_stall_risk" }; Based on the above data, the feature vector [0.031, -0.128, 0.842, ..., 0.476] represents the feature values ​​reported by the image. The trained GraphSAGE can then make a judgment based on these feature values, indicating an 80% probability of engine failure. Based on this 80% probability, an alarm is issued, the vehicle is locked via OTA (Over-The-Air technology), and an image is uploaded.

[0036] The fusion engine is based on Flink CEP+Async I / O, calling the GNN (Graph Neural Network) service with a timeout of 100ms. Upon timeout, it degrades to multiplying rule confidence scores. The GNN model uses a two-layer GraphSAGE, inputting node features as semantic JSON vector concatenation and outputting edge prediction probabilities. Higher-order events employ a "UUID + source event list" mechanism to ensure traceability. The SOAR playbook uses OpenAPI 3.0 for description, supporting hot-swapping of ECU (Engine Control Unit) adapters from different vehicle manufacturers. It should be noted that GNN can be abandoned in favor of traditional machine learning feature concatenation combined with XGBoost classification, directly concatenating structured fields with image / audio vectors into a wide table. Furthermore, instead of using a Kubernetes CRD Playbook, an Ansible Agent can be used to distribute SSH (Secure Shell) scripts to the vehicle's T-Box (Telematics BOX) to perform vehicle locking and recording.

[0037] In this way, Flink CEP handles structured alarm filtering, while GraphSAGE-GNN handles cross-modal correlation. The two are concatenated via asynchronous I / O to output high-order attack events, overcoming the bottleneck of a single rule engine being unable to describe cross-modal attacks. These high-order events carry a UUID (Universally Unique Identifier) ​​and a list of source events, directly triggering a Kubernetes CRD-based Playbook. Actions such as locking car doors, uploading camera recordings, and disabling communication modules are then automatically triggered via OTA channels, achieving full automation and overcoming the limitations of traditional VSOCs that only display information without taking action.

[0038] Further explanation is needed regarding the vehicle networking platform in this application, which adopts a two-tier physical deployment architecture of "cloud-based central server cluster + vehicle-mounted edge computing box." All nodes are uniformly based on the Kubernetes container orchestration platform to achieve elastic scheduling and resource isolation of software functions. Two 1U rack-mount servers are deployed in the cloud (serving as dual master-slave hot standby for each other). The hardware configuration of a single server is as follows: Central Processing Unit: Intel Xeon Silver 4314 processor to provide ample general-purpose computing power. Memory: 256 GB DDR4 ECC memory. Storage Subsystem: Hot Data Lake: Equipped with two NVMe 4 TB solid-state drives; Cold Data Lake: Equipped with four SATA 16 TB hard disk drives. Network Interface: Equipped with dual 10 Gigabit SFP+ optical ports, using LACP link aggregation and binding as logical ports. Graphics Processing Unit: Equipped with two single-width NVIDIA RTX A4000 professional graphics cards (16 GB VRAM / card), dedicated to batch inference, incremental learning training, and complex visual semantic analysis tasks of graph neural network models. Operating System and Container Platform: A unified Ubuntu 22.04 LTS operating system is used, with a Kubernetes 1.28 container orchestration cluster deployed. Core SoC of the in-vehicle edge computing box: NVIDIA Jetson Orin NX 16 GB system-level module; External interfaces: 2 CAN-FD interfaces, 2 1000BASE-T Gigabit Ethernet ports, 4 GMSL2 serializer / deserializer camera interfaces; Local storage: Configured with a 1 TB NVMe solid-state drive; Containerized deployment: Sharing the same KubernetesYAML resource orchestration files with the cloud, running the same microservice images; Differentiated restrictions are only applied to resource quotas, such as setting the GPU computing power limit to 30% of the cloud's limit.

[0039] In this embodiment, after receiving multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices, the data needs to be analyzed. Then, based on the data type determined by the analysis, the multi-source heterogeneous vehicle data is stored in the corresponding storage area of ​​a preset data lake. After storage, semantic analysis is performed on the multi-source heterogeneous vehicle data to determine its corresponding semantics. The semantics are then matched against a preset alarm rule database, and finally, vehicle warnings are issued based on the successfully matched target alarm rules. This allows data of any format to be converted into a unified semantic object and enables cross-modal data to be correlated. This transforms previously discarded or isolated multimedia data into analyzable data semantics, thereby achieving collaboration of multi-source heterogeneous data and improving the reliability of vehicle warnings.

[0040] See Figure 5 As shown, this embodiment of the invention discloses a vehicle early warning method for multi-source heterogeneous data collaboration, applied to a vehicle networking platform, including: The data parsing module 11 is used to receive multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices, and to parse the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data. The semantic analysis module 12 is used to store the multi-source heterogeneous vehicle data into the corresponding storage area in the preset data lake according to the data type, and to perform semantic analysis on the multi-source heterogeneous vehicle data according to the preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data. The vehicle warning module 13 is used to match the data semantics with a preset alarm rule library. If the data semantics successfully match the target alarm rule in the preset alarm rule library, a vehicle warning is issued based on the target alarm rule.

[0041] In some embodiments, the data parsing module 11 may specifically include: The data acquisition unit is used to receive multi-source heterogeneous vehicle data reported by multi-source on-board devices through a preset multi-protocol gateway; the multi-source heterogeneous vehicle data refers to vehicle data in different formats collected by different on-board devices. The data type identification unit is used to parse the multi-source heterogeneous vehicle data based on the decoder in the preset multi-protocol gateway in order to identify the data type corresponding to the multi-source heterogeneous vehicle data.

[0042] In some embodiments, the semantic analysis module 12 may specifically include: The first data storage unit is used to classify the multi-source heterogeneous data based on the data type, and store the structured data in the multi-source heterogeneous data into a structured data stream. The second data storage unit is used to write binary objects from the multi-source heterogeneous data into a preset raw data storage bucket. The third data storage unit is used to write metadata events from the multi-source heterogeneous data into a preset metadata stream; The fourth data storage unit is used to extract text key-value pairs from the text logs in the multi-source heterogeneous data and write the text key-value pairs into the text data stream.

[0043] In some embodiments, the semantic analysis module 12 may specifically include: The data augmentation submodule is used to augment the multi-source heterogeneous vehicle data to obtain augmented data; The semantic analysis submodule is used to determine the data type of the enhanced data and perform corresponding semantic analysis on the enhanced data according to the data type to determine several data semantics corresponding to the multi-source heterogeneous vehicle data.

[0044] In some embodiments, the semantic analysis submodule may specifically include: The first data reasoning unit is used to analyze the enhanced data through a preset semantic analysis model. If the enhanced data is image data, it performs reasoning on the image data to determine the image data features corresponding to the image data. The keyword matching unit is used to convert the audio data into corresponding audio text data if the enhanced data is audio data, and to perform keyword matching on the text data to determine the audio text features corresponding to the text data. The second data reasoning unit is used to perform frame extraction processing on the video data and reason on the obtained video frames if the enhanced data is video data, so as to determine the video features corresponding to the video frames. A data classification unit is used to classify the text data if the enhanced data is text data, so as to determine the text features corresponding to the text data; The data semantic determination unit is used to construct data semantics in a preset format based on the image data features, and / or the audio text features, and / or the video features, and / or the text features.

[0045] In some embodiments, the vehicle warning module 13 may specifically include: The prediction probability determination unit is used to send the data semantics to a preset neural network model for analysis, so as to determine a number of prediction probabilities corresponding to the data semantics and each alarm rule in the preset alarm rule base. The vehicle warning unit is used to determine the target prediction probability with the highest probability value among the plurality of predicted probabilities. If the target probability value corresponding to the target prediction probability is not less than a preset probability threshold, the alarm rule corresponding to the target prediction probability is taken as the target alarm rule, and a vehicle warning is performed based on the target alarm rule.

[0046] In some embodiments, the vehicle warning module for multi-source heterogeneous data collaboration may specifically include: The confidence determination unit is used to determine the confidence value corresponding to the data semantics based on the preset rule confidence value if the response duration of the preset neural network model is greater than the preset response duration threshold, and to determine whether to issue a vehicle warning based on the confidence value.

[0047] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0048] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the vehicle early warning method for multi-source heterogeneous data collaboration disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0049] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0050] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0051] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the vehicle warning method for multi-source heterogeneous data collaboration executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0052] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned vehicle early warning method for multi-source heterogeneous data collaboration. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0053] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0054] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0055] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0056] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0057] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A vehicle early warning method for multi-source heterogeneous data collaboration, characterized in that, Applications in vehicle networking platforms include: Receive multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices, and parse the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data; The multi-source heterogeneous vehicle data is stored in the corresponding storage area of ​​the preset data lake according to the data type, and semantic analysis is performed on the multi-source heterogeneous vehicle data according to the preset semantic analysis model to determine several data semantics of the multi-source heterogeneous vehicle data. The data semantics are matched with a preset alarm rule library. If the data semantics successfully match the target alarm rule in the preset alarm rule library, a vehicle warning is issued based on the target alarm rule.

2. The vehicle early warning method for multi-source heterogeneous data collaboration according to claim 1, characterized in that, The process of receiving multi-source heterogeneous vehicle data reported by multi-source vehicle-mounted devices and parsing the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data includes: The system receives multi-source heterogeneous vehicle data reported by multiple vehicle-mounted devices through a preset multi-protocol gateway; the multi-source heterogeneous vehicle data refers to vehicle data in different formats collected by different vehicle-mounted devices. The decoder in the preset multi-protocol gateway parses the multi-source heterogeneous vehicle data to identify the data type corresponding to the multi-source heterogeneous vehicle data.

3. The vehicle early warning method for multi-source heterogeneous data collaboration according to claim 1, characterized in that, The step of storing the multi-source heterogeneous vehicle data into the corresponding storage area in the preset data lake according to the data type includes: Based on the data type, the multi-source heterogeneous data is classified, and the structured data in the multi-source heterogeneous data is stored in a structured data stream; And, write the binary objects in the multi-source heterogeneous data into a preset raw data storage bucket; And, write the metadata events in the multi-source heterogeneous data into a preset metadata stream; And, extract the text key-value pairs from the text logs in the multi-source heterogeneous data, and write the text key-value pairs into the text data stream.

4. The vehicle early warning method facing multi-source heterogeneous data collaboration according to claim 1, characterized in that, The step of performing semantic analysis on the multi-source heterogeneous vehicle data according to a preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data includes: The multi-source heterogeneous vehicle data is augmented to obtain augmented data; The data type of the enhanced data is determined, and corresponding semantic analysis is performed on the enhanced data based on the data type to determine several data semantics corresponding to the multi-source heterogeneous vehicle data.

5. The vehicle early warning method for multi-source heterogeneous data collaboration according to claim 4, characterized in that, The step of determining the data type of the enhanced data and performing corresponding semantic analysis on the enhanced data based on the data type to determine several data semantics corresponding to the multi-source heterogeneous vehicle data includes: The enhanced data is analyzed by a preset semantic analysis model. If the enhanced data is image data, inference is performed on the image data to determine the image data features corresponding to the image data. If the enhanced data is audio data, the audio data is converted into corresponding audio text data, and keyword matching is performed on the text data to determine the audio text features corresponding to the text data. If the enhanced data is video data, then the video data is subjected to frame extraction processing, and the obtained video frames are inferred to determine the video features corresponding to the video frames. If the enhanced data is text data, then the text data is classified to determine the text features corresponding to the text data; Data semantics in a preset format are constructed based on the image data features, and / or the audio text features, and / or the video features, and / or the text features.

6. The vehicle early warning method for multi-source heterogeneous data collaboration according to any one of claims 1 to 5, characterized in that, The step of matching the data semantics with a preset alarm rule base, and if the data semantics successfully match a target alarm rule in the preset alarm rule base, then issuing a vehicle warning based on the target alarm rule, includes: The data semantics are sent to a preset neural network model for analysis to determine several prediction probabilities corresponding to the data semantics and each alarm rule in the preset alarm rule base. Determine the target prediction probability with the highest probability value among the several predicted probabilities. If the target probability value corresponding to the target prediction probability is not less than a preset probability threshold, then the alarm rule corresponding to the target prediction probability is taken as the target alarm rule, and vehicle warning is performed based on the target alarm rule.

7. The vehicle early warning method for multi-source heterogeneous data collaboration according to claim 6, characterized in that, The step of matching the data semantics with a preset alarm rule base, and if the data semantics successfully match a target alarm rule in the preset alarm rule base, and then issuing a vehicle warning based on the target alarm rule, further includes: If the response duration of the preset neural network model is greater than the preset response duration threshold, then the confidence value corresponding to the data semantics is determined based on the preset rule confidence value, and whether to issue a vehicle warning is determined based on the confidence value.

8. A vehicle early warning device oriented to multi-source heterogeneous data collaboration, characterized in that, Applications in vehicle networking platforms include: The data parsing module is used to receive multi-source heterogeneous vehicle data reported by multi-source on-board equipment, and to parse the multi-source heterogeneous vehicle data to determine the data type corresponding to the multi-source heterogeneous vehicle data. The semantic analysis module is used to store the multi-source heterogeneous vehicle data into the corresponding storage area in the preset data lake according to the data type, and to perform semantic analysis on the multi-source heterogeneous vehicle data according to the preset semantic analysis model to determine several data semantics corresponding to the multi-source heterogeneous vehicle data. The vehicle warning module is used to match the data semantics with a preset alarm rule library. If the data semantics successfully match the target alarm rule in the preset alarm rule library, a vehicle warning is issued based on the target alarm rule.

9. An electronic device, comprising: include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the vehicle early warning method for multi-source heterogeneous data collaboration as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the vehicle early warning method for multi-source heterogeneous data collaboration as described in any one of claims 1 to 7.

Images