Vehicle and road cloud service panoramic knowledge graph construction method based on various data driving

Abstract

The invention provides a vehicle and road cloud service panoramic knowledge graph construction method based on various data driving, and relates to the technical field of vehicle and road cloud integration in an intelligent traffic system, and the method comprises the steps: obtaining multi-source heterogeneous data from a plurality of data sources in the traffic system; cleaning and standardizing the multi-source heterogeneous data to obtain a data set; entities and relations are extracted from the data set; aligning and fusing the extracted entities and relationships to obtain an entity relationship set; constructing a graph structure of the knowledge graph by utilizing the entity relationship set, and storing the graph structure in a graph database; dynamically updating the knowledge graph in the graph database based on incremental updating; based on a knowledge graph in a graph database, a cloud-edge collaborative panoramic knowledge graph platform is constructed, an accurate and structured knowledge graph is constructed, and the semantic expression ability of traffic system data is enhanced; and through an incremental updating mechanism, dynamic updating of the knowledge graph is realized, and timeliness and accuracy of the knowledge graph are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle-road cloud integration technology in intelligent transportation systems, and in particular to a vehicle-road cloud business panoramic knowledge graph construction method based on multiple data driving. BACKGROUND

[0002] With the rapid development of intelligent transportation systems, vehicle-road cloud integration technology puts forward higher requirements for multi-source data fusion and intelligent analysis. The current transportation system involves the collection and fusion of multi-source heterogeneous data such as vehicle terminals, roadside devices, cloud platforms, etc. These data include vehicle operating status, road environment perception, traffic management services, and other multi-dimensional information, and effective technical means are urgently needed to realize the deep integration and intelligent analysis of data.

[0003] The prior art mainly uses data warehouse or relational database for storage management, and realizes simple data analysis through rule engine. This method has obvious shortcomings: first, it is difficult to effectively handle the complex correlation between massive heterogeneous data; second, data cleaning and standardization processing is not perfect, resulting in unstable data quality; in addition, the knowledge graph updating mechanism is lagging behind, and lacks spatiotemporal dynamic modeling capability. These problems seriously restrict the practical application effect of knowledge graph in intelligent transportation systems. SUMMARY

[0004] The present application provides a vehicle-road cloud business panoramic knowledge graph construction method based on multiple data driving, to solve the technical problem that the knowledge graph in the prior art cannot accurately reflect the state of the transportation system in a timely manner.

[0005] In one aspect, the present application provides a vehicle-road cloud business panoramic knowledge graph construction method based on multiple data driving, comprising: Obtaining multi-source heterogeneous data from multiple data sources in the transportation system; Cleaning and standardizing the multi-source heterogeneous data to obtain a data set; According to the attributes of the data in the data set, classifying the data in the data set, and extracting entities and relationships in the classified data; Aligning and fusing the extracted entities and relationships to obtain an entity relationship set; Using the entity relationship set to construct a graph structure of the knowledge graph, and storing the graph structure in a graph database; When new data is generated in the data source, dynamically updating the knowledge graph in the graph database; Based on the knowledge graph in the graph database, a cloud-edge collaborative panoramic knowledge graph platform is constructed.

[0006] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data driving. According to the types of the multi-source heterogeneous data in the data set, the data set is subjected to first clustering, and a plurality of initial clusters are obtained. According to the semantic information in each initial cluster, the initial cluster is subjected to second clustering, and a plurality of sub-clusters are obtained. For each sub-cluster, data points having similar features are identified as the same type of entity, and the interaction mode between entities is identified as a relationship.

[0007] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data driving. When new data is generated in the data source, the running situation of the current traffic system is evaluated to obtain an evaluation result, and the priority of the current update task is determined according to the evaluation result; wherein the priority includes high priority and low priority. The entities and relationships of the new data are extracted. According to the running situation, high entities and high relationships corresponding to the high priority and low entities and low relationships corresponding to the low priority are screened out from the new entities and relationships. The high entities and the high relationships are preferentially updated into the knowledge graph. The low entities and the low relationships are updated into the knowledge graph.

[0008] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data driving. The similarity of the high entities and the existing entities and the similarity of the high relationship and the existing relationship semantics are determined respectively. If the similarity of the high entities and the high relationships is greater than a first similarity threshold, the high entities and the high relationships are merged with the respective corresponding entities and relationships respectively, and if the similarity is less than or equal to the first similarity threshold, the high entities and the high relationships are taken as new entities and new edges respectively. The low entities and the low relationships are updated into the knowledge graph. The similarity of the low entities and the existing entities and the similarity of the low relationship and the existing relationship semantics are determined respectively. If the similarity of the low entity and the low relation is greater than a second similarity threshold, the low entity and the low relation are merged with the respective corresponding entity and relation respectively, and if the similarity is less than or equal to the second similarity threshold, the low entity and the low relation are respectively taken as a new entity and a new edge; wherein the second similarity threshold is less than the first similarity threshold.

[0009] According to the vehicle-road cloud business panoramic knowledge graph construction method based on multiple data driving provided by the application, the graph structure of the knowledge graph is constructed by using the entity relation set, which comprises: According to the entity relation set, each entity is taken as an entity node in the graph, and the relation between the entities is taken as a relation edge in the graph, so as to construct the basic graph structure of the knowledge graph. For each entity node, the entity nodes with spatio-temporal association are connected through the spatio-temporal edge according to the spatio-temporal information thereof. For each relation edge, a spatio-temporal weight is added to the relation edge according to the time and space range of the occurrence of the relation edge. The entity nodes in the basic graph structure are subjected to multi-modal feature enhancement in combination with the multi-modal data features in the multi-source heterogeneous data.

[0010] According to the vehicle-road cloud business panoramic knowledge graph construction method based on multiple data driving provided by the application, the entity nodes in the basic graph structure are subjected to multi-modal feature enhancement in combination with the multi-modal data features in the multi-source heterogeneous data, which comprises: For each entity node, the features of the entity node in different modalities are extracted from the multi-source heterogeneous data. The features in different modalities are fused to generate a multi-modal feature vector. The generated multi-modal feature vector is embedded into the representation of the entity node.

[0011] According to the vehicle-road cloud business panoramic knowledge graph construction method based on multiple data driving provided by the application, the features in different modalities are fused to generate a multi-modal feature vector, which comprises: For each entity node, scene correlation weights are assigned to the features in different modalities according to the business scenario to which the entity node belongs. The features in different modalities are weighted and averaged according to the scene correlation weights to obtain preliminary fusion features. The preliminary fusion features are subjected to dimension reduction processing to generate a final multi-modal feature vector. The generated multi-modal feature vector is embedded into the representation of the entity node, which comprises: The multi-modal feature vector is combined with the attributes of the entity node to form an entity node representation. The entity node representation is mapped to a semantic space of the knowledge graph. In the knowledge graph, the mapped entity node is connected with the related entity nodes through the relation edges.

[0012] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data, which aligns and fuses extracted entities and relations to obtain an entity relation set, comprising: standardizing the extracted entities to unify the entity representation in different data sources to the same standard; performing semantic alignment on the extracted relations to combine the relations with a semantic similarity exceeding a preset similarity threshold but different expressions; associating entities and relations with the same context to obtain an entity relation set.

[0013] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data, which constructs a cloud-edge collaborative panoramic knowledge graph platform based on a knowledge graph in the graph database, comprising: deploying a graph database on the cloud to centrally store graph structure data of the knowledge graph; deploying a knowledge graph cache module and a real-time calculation module on the edge to process real-time data generated by the edge device and feed back the processing result to the cloud; constructing a visual interface of the panoramic knowledge graph platform to provide a graph display function and an interactive function.

[0014] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data, which cleans and standardizes the multiple source heterogeneous data to obtain a data set, comprising: removing abnormal values and noise data in the multiple source heterogeneous data; filling missing values in the multiple source heterogeneous data; converting data in different formats in the multiple source heterogeneous data into a unified format; normalizing the multiple source heterogeneous data to obtain a data set.

[0015] The application provides a vehicle-road cloud service panoramic knowledge graph construction method based on multiple data, which dynamically updates the knowledge graph in the graph database when new data is generated in the data source, comprising: when new data is generated in the data source, extracting entities and relations of the new data; for the new entity, checking the similarity of the new entity with existing entities, if the similarity is lower than a preset threshold, adding the new entity as a new node to the knowledge graph, if the similarity is higher than or equal to the preset threshold, combining the features of the new entity with existing nodes; For the newly added relationship, check its semantic consistency with the existing relationship, if inconsistent, add it as a new edge to the knowledge graph, if consistent, update the attributes of the existing relationship according to the newly added relationship.

[0016] According to the car road cloud business panoramic knowledge graph construction method based on multiple data driving provided by the application, the multiple data sources include: A vehicle end data source is used to provide vehicle running state data. A roadside sensing device data source is used to provide road environment data. A cloud service system data source is used to provide traffic management and service data. An external knowledge base data source is used to provide knowledge data related to traffic. A user equipment data source is used to provide user behavior and preference data. A third-party service data source is used to provide third-party service data related to traffic.

[0017] The car road cloud business panoramic knowledge graph construction method based on multiple data driving provided by the application effectively integrates multi-dimensional information by obtaining massive data from multiple heterogeneous data sources and performing cleaning and standardization processing, improves the quality and availability of data, constructs a precise and structured knowledge graph by using entity and relationship extraction, alignment and fusion technology, enhances the semantic expression ability of traffic system data, realizes dynamic updating of the knowledge graph through an incremental updating mechanism, ensures its timeliness and accuracy, and constructs a panoramic knowledge graph platform based on a cloud-edge collaborative architecture, which not only supports efficient storage and query, but also can process real-time data of edge devices in real time, and provides strong support for monitoring, decision-making and optimization of intelligent transportation systems. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0019] Figure 1 is a flowchart of the car road cloud business panoramic knowledge graph construction method based on multiple data driving provided by the embodiment of the present application; Figure 2 is a structural schematic diagram of the car road cloud business panoramic knowledge graph construction device based on multiple data driving provided by the embodiment of the present application; Figure 3 is a structural schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0021] Figure 1 is a flowchart of a method for constructing a vehicle-road cloud service panoramic knowledge graph based on multiple data provided by an embodiment of the present application.

[0022] Referring to Figure 1 The method for constructing a vehicle-road cloud service panoramic knowledge graph based on multiple data can include the following steps 101 to 107.

[0023] Step 101, obtaining multi-source heterogeneous data from multiple data sources in a traffic system.

[0024] In this step, the multiple data sources can include a vehicle-side data source, a roadside perception device data source, a cloud-side service system data source, an external knowledge base data source, a user device data source, and a third-party service data source, etc.

[0025] The vehicle-side data source is used to provide running state data of a vehicle. For example, speed and acceleration data of the vehicle, engine state and fuel consumption data of the vehicle, driving mileage and brake frequency data of the vehicle, and data generated by an automatic driving system of the vehicle, such as environment perception data (radar and camera data).

[0026] The roadside perception device data source is used to provide road environment data. For example, traffic flow data collected by a camera on the road, vehicle speed and distance data detected by a radar device, weather data (such as temperature, humidity, and wind speed) collected by a roadside weather sensor, and road condition sensor data (such as ice and water accumulation detection).

[0027] The cloud-side service system data source is used to provide traffic management and service data. For example, control data of a traffic signal, operation schedule and dispatch data of public transportation, real-time monitoring and analysis data of traffic congestion, and traffic accident record and processing data.

[0028] The external knowledge base data source is used to provide knowledge data related to traffic. For example, map data (such as Gaode Map and Baidu Map), traffic regulations and policy data, historical traffic data (such as annual traffic accident statistics), and data and reports published by traffic research institutions.

[0029] User device data sources are used to provide user behavior and preference data. For example, user's navigation preferences (such as frequently chosen routes), user's travel time preferences (such as commuting times), user's feedback data on traffic congestion, user's operation records in traffic applications (such as querying road conditions, booking parking spaces).

[0030] Third-party service data sources are used to provide traffic-related third-party service data. For example, order data of shared travel platforms, parking space usage data provided by parking lot management systems, freight vehicle operation data of logistics companies, and vehicle insurance claim data provided by insurance companies.

[0031] Step 102, cleaning and standardizing the multi-source heterogeneous data to obtain a data set.

[0032] Step 102 can specifically include: Step one, removing outliers and noise data in multi-source heterogeneous data; In this step, outliers in the data can be identified through statistical analysis (such as mean, standard deviation) or machine learning algorithms (such as IsolationForest). Filter algorithms (such as median filtering) or rule-based methods (such as threshold filtering) are used to remove noise.

[0033] Suppose the speed value of a certain data point in the speed data obtained from the vehicle sensor is 300 km / h, which obviously does not conform to the actual situation (may be sensor failure). By setting a reasonable speed threshold (such as 150 km / h), this outlier can be identified and removed. For traffic flow data collected by road cameras, if the flow data at a certain time point is significantly higher than other time points (such as suddenly increasing by 10 times), a median filtering algorithm can be used to smooth this mutation and remove noise.

[0034] Step two, filling in missing values in multi-source heterogeneous data; In this step, there are many methods for filling in missing values, such as mean filling, median filling, interpolation filling, using machine learning models (such as KNN) to predict missing values, etc.

[0035] Suppose there are some missing fuel consumption data for a vehicle. If these data are recorded in time series, linear interpolation method can be used to estimate the missing values according to the fuel consumption values of adjacent time points. For missing user behavior data in user device data, KNN algorithm can be used to predict and fill in the missing values according to the behavior patterns of similar users.

[0036] Step three, converting different formats of data in multi-source heterogeneous data into a unified format; In this step, it can generally refer to format conversion, or data structure adjustment, or encoding unification. For example, converting data from one format (such as XML, JSON) to another format (such as CSV), unfolding nested data structures (such as nested objects in JSON) into flat structures, and unifying character encoding in data to a standard format such as UTF-8.

[0037] Step four, normalizing the multi-source heterogeneous data to obtain a data set.

[0038] In this step, the normalization method can be Min-Max normalization, which scales the data to the interval [0, 1], or Z-Score standardization, which converts the data to a distribution with a mean of 0 and a standard deviation of 1.

[0039] Step 103, according to the attributes of the data in the data set, classifying the data in the data set and extracting entities and relationships in the classified data.

[0040] Step 103 can specifically include: Step one, according to the types of multi-source heterogeneous data in the data set, performing a first clustering on the data set to obtain a plurality of initial clusters; In this step, clustering algorithms (such as K-Means, DBSCAN) can be used to preliminarily group data according to their sources or types. For example, vehicle-side data, roadside sensing device data, and cloud-side business system data are classified into different clusters.

[0041] Step two, for each initial cluster, according to the semantic information therein, performing a second clustering to obtain a plurality of sub-clusters; In this step, natural language processing (NLP) techniques or semantic analysis algorithms can be used to analyze text, image, and other information in the data. For example, location information and speed information in vehicle-side data are clustered separately. For example, in the vehicle-side data cluster, a second clustering is further performed according to the driving state of the vehicle (such as high-speed driving, low-speed driving, and parking) to obtain a high-speed driving vehicle data sub-cluster (speed > 60 km / h), a low-speed driving vehicle data sub-cluster (speed ≤ 60 km / h), and a parked vehicle data sub-cluster (speed = 0 km / h). In the roadside sensing device data cluster, a second clustering is performed according to the camera image content (such as traffic congestion and accident scene) to obtain a traffic congestion image sub-cluster and an accident scene image sub-cluster.

[0042] Step three, for each sub-cluster, identifying data points with similar features therein as the same class of entities, and identifying interaction patterns between entities as relationships.

[0043] In this step, feature extraction techniques such as TF-IDF, Word2Vec can be used to process text data and extract key features; image recognition techniques such as CNN can be used to process image data and extract image features; association rule mining or graph neural network (GNN) techniques can be used to identify relationships between entities.

[0044] Each of the above steps gradually refines and classifies complex multi-source heterogeneous data through two clustering processes (first by data type, second by semantic information), which can effectively reduce the complexity of data processing and improve the efficiency of data processing.

[0045] Step 104, align and fuse the extracted entities and relationships to obtain an entity relationship set.

[0046] Step 104 can specifically include: Step one, standardize the extracted entities to unify the representation of entities in different data sources to the same standard; In this step, the name of the entity can be standardized to ensure that the same entity in different data sources uses the same name; the attributes of the entity (such as time, location, numerical value, etc.) can be formatted to ensure consistency; the ambiguity of the entity name can be solved through context information or external knowledge base.

[0047] Step two, perform semantic alignment on the extracted relationships to merge relationships with a semantic similarity exceeding a preset similarity threshold but different expressions; In this step, natural language processing techniques such as word embedding and semantic similarity algorithms are used to calculate the similarity between relationships, and relationships with a semantic similarity exceeding a preset threshold are merged into one relationship. For example, suppose the relationship "Vehicle A travels on Road X" is extracted from vehicle-side data, and the relationship "Vehicle A travels on Road X" is extracted from road-side perception device data. Through semantic alignment, the semantic similarity of the two relationships is calculated (assuming the threshold is 0.8), and it is found that their similarity is 0.9, so they are merged into one relationship "Vehicle A travels on Road X".

[0048] Step three, associate entities and relationships with the same context to obtain an entity relationship set; In this step, by analyzing the context information of entities and relationships (such as time, location, event, etc.), it is determined whether they belong to the same scene. Entities and relationships with the same context are combined into a complete entity relationship set. For example, assume that within a certain time period, vehicle A has overtaking behavior on road X, and a roadside perception device records this event. Through context analysis, it is found that vehicle A and road X have the same context in time (9:00 am) and location (a certain section of road X). Therefore, the entity "vehicle A", "road X" and the relationship "vehicle A overtaking on road X" are associated to form a complete entity relationship set.

[0049] Step 105, using the entity relationship set, constructing the graph structure of the knowledge graph, and storing the graph structure in the graph database.

[0050] Step 105 can specifically include: Step one, according to the entity relationship set, taking each entity as an entity node in the graph, and taking the relationship between entities as a relationship edge in the graph, constructing the basic graph structure of the knowledge graph; For example, assume that the entities include vehicle A, road X, and traffic signal Y; the relationships include vehicle A driving on road X, and road X being controlled by traffic signal Y; the basic graph structure can be vehicle A -- (driving on) --> road X -- (controlled by) --> traffic signal Y.

[0051] Step two, for each entity node, according to its spatio-temporal information, connecting the entity nodes with spatio-temporal association through spatio-temporal edges; In this step, spatio-temporal information can generally include time stamp and geographic location, etc. For example, assume that vehicle A passes road X at 9:00 am, and vehicle B passes the same road X at 9:10 am. According to the spatio-temporal information, vehicle A and vehicle B can be connected through spatio-temporal edges, indicating that they pass the same location within the same time period.

[0052] Step three, for each relationship edge, according to the time and space range of its occurrence, adding spatio-temporal weight to it; In this step, the weight can generally be assigned according to the time interval or spatial distance. The relationship between time interval or spatial distance and weight size is usually an inverse relationship, that is, the shorter the time interval or the closer the spatial distance, the greater the weight. This relationship reflects the closeness of entities in time and space, and the greater the weight indicates that the association between entities is stronger, and the more important it is for the semantic expression and analysis of the knowledge graph.

[0053] The above steps significantly improve the spatio-temporal semantic expression ability and multi-modal information fusion ability of the knowledge graph by constructing a basic graph structure, adding spatio-temporal edges and spatio-temporal weights, and enhancing multi-modal features. This not only improves the quality and accuracy of the knowledge graph, but also provides stronger support for the diversification of intelligent transportation systems, enabling the knowledge graph to better serve traffic management and intelligent decision-making.

[0054] Step four, multi-modal feature enhancement of entity nodes in the basic graph structure by combining multi-modal data features in multi-source heterogeneous data.

[0055] Step 106, when new data is detected in the data source, dynamically update the knowledge graph in the graph database.

[0056] Step 106 specifically includes: Step one, when new data is detected in the data source, evaluate the running situation of the current transportation system to obtain an evaluation result, and determine the priority of this update task according to the evaluation result; wherein the priority includes high priority and low priority; In this step, the running situation can include, for example, holidays and special events. Special events can include marathons or important exams, etc. High priority is generally for roads and venues that are greatly affected by special events and holidays. Low priority is for roads and venues that are less affected.

[0057] Step two, extract entities and relationships from the new data; In this step, data extraction tools such as NLP tools and data mining algorithms can be used to extract entities and relationships from the new data.

[0058] Step three, according to the running situation, filter out high entities and high relationships corresponding to high priority and low entities and low relationships corresponding to low priority from the new entities and relationships; Step four, update the high entities and high relationships to the knowledge graph first; Step five, update the low entities and low relationships to the knowledge graph.

[0059] For example, suppose a large marathon is held in a city, and this special event is detected as a high-priority situation. The new data may include temporary traffic control on the marathon route, spectator gathering points, etc., and will be marked as high priority, while other routine maintenance or regular traffic information will be marked as low priority. First, process the data related to the marathon to ensure that the road conditions and traffic flow along the race route are updated in a timely manner.

[0060] Among them, updating the high entities and high relationships to the knowledge graph first includes: respectively determine the similarity of the high entity and the existing entity and the similarity of the high relationship and the existing relationship semantics; If the similarity of the high entity and the high relationship is greater than the first similarity threshold, they are merged with the respective corresponding entity and relationship respectively, and if it is less than or equal to the first similarity threshold, they are respectively taken as a new entity and a new edge.

[0061] updating the low entity and the low relationship into the knowledge graph, including: respectively determine the similarity of the low entity and the existing entity and the similarity of the low relationship and the existing relationship semantics; If the similarity of the low entity and the low relationship is greater than the second similarity threshold, they are merged with the respective corresponding entity and relationship respectively, and if it is less than or equal to the second similarity threshold, they are respectively taken as a new entity and a new edge; wherein the second similarity threshold is less than the first similarity threshold.

[0062] When judging the similarity, the cosine similarity between entities can be calculated to determine the degree of similarity according to the size of the cosine similarity. The semantic alignment technology (such as semantic similarity algorithm) can be used to check the semantic consistency of the new relationship and the existing relationship. For example, suppose the new relationship is "vehicle A is at position P2 at time T2", and the existing relationship is "vehicle A is at position P1 at time T1". After checking the semantic consistency, it is found that both describe the position information of the same entity at different times. Therefore, the attributes of the existing relationship are updated, and the timestamp and position information are updated to: vehicle A--(position update)--> position P1 (time T1), position P2 (time T2).

[0063] In addition to the above-mentioned ways of updating the knowledge graph, a graph evolution and life cycle management mechanism can also be designed to dynamically adjust the structure and content of the knowledge graph according to the usage and data update frequency of the knowledge graph, ensuring the timeliness and usability of the knowledge graph.

[0064] Step 107, based on the knowledge graph in the graph database, a cloud-edge collaborative panoramic knowledge graph platform is constructed.

[0065] Step 107 can specifically include: Step one, deploy a graph database in the cloud for centralized storage of graph structure data of the knowledge graph; In this step, a graph database (such as Neo4j, OrientDB) is used to deploy a knowledge graph storage system in the cloud. The constructed knowledge graph data is stored in the graph database, supporting complex graph queries and analysis. Assuming that a knowledge graph containing entities such as vehicles, roads, traffic lights, and relationships has been constructed, these data are stored in the Neo4j graph database in the cloud. For example, querying all vehicles and their driving status on a certain road can be quickly completed through the query language (such as Cypher) of the graph database.

[0066] Step two, deploy knowledge graph caching module and real-time computing module at the edge end to process real-time data generated by edge devices and feed back the processing results to the cloud; In this step, for example, a caching module is deployed at the edge end (such as a roadside device or a vehicle terminal) to temporarily store real-time data; a real-time computing module (such as Apache Flink or Spark Streaming) is deployed to quickly process and analyze real-time data. The processing and analysis can refer to the previous embodiments, such as data cleaning, feature extraction, event detection, and correlation analysis between data.

[0067] Suppose a roadside perception device collects real-time traffic flow data for a certain section of the road, the caching module at the edge end temporarily stores these data, and the real-time computing module quickly analyzes the traffic flow changes. If a traffic jam is detected, the real-time computing module generates an alert message and sends relevant information (such as the location of the jam and the timestamp) to the cloud to update the relevant nodes and relationships in the knowledge graph.

[0068] Step three, build a visual interface for the panoramic knowledge graph platform to provide graph display functions and interactive functions; In this step, a visual interface can be built using web technologies such as HTML5, CSS3, and JavaScript. Graph visualization libraries such as D3.js or Cytoscape.js can be integrated to display the structure of the knowledge graph. Interactive functions can include node queries, path searches, and real-time data updates.

[0069] For example, suppose the visual interface of the panoramic knowledge graph platform has been built. Users can access the interface through a browser to view the knowledge graph of the transportation system. Users can click on a node (such as vehicle A) to view its detailed information (such as license plate number, vehicle type, and driving path), and also query the relationship between vehicle A and other vehicles (such as whether they are driving on the same road) through interactive functions. If the edge end detects a new traffic event (such as an accident), the visual interface will be updated in real-time to display the accident location and the dynamic changes of the relevant vehicles.

[0070] In this embodiment, by obtaining massive data from multiple heterogeneous data sources and performing cleaning and standardization processing, multi-dimensional information is effectively integrated, improving the quality and usability of the data; by using entity and relationship extraction, alignment, and fusion technology, a precise and structured knowledge graph is constructed, enhancing the semantic expression capability of the transportation system data; through an incremental update mechanism, dynamic updating of the knowledge graph is realized, ensuring its timeliness and accuracy; based on the cloud-edge collaborative architecture, the panoramic knowledge graph platform is built, which not only supports efficient storage and query, but also can process real-time data from edge devices in real-time, providing strong support for monitoring, decision-making, and optimization of the intelligent transportation system.

[0071] In an embodiment of the present specification, in combination with the multi-modal data features in multi-source heterogeneous data, the multi-modal feature enhancement of the entity nodes in the basic graph structure can include: For each entity node, extract its features in different modalities from multi-source heterogeneous data; fuse the features of different modalities to generate a multi-modal feature vector; embed the generated multi-modal feature vector into the representation of the entity node.

[0072] In the present embodiment, the features of different modalities are fused to generate a multi-modal feature vector, and the generated multi-modal feature vector is embedded into the representation of the entity node, which enriches the semantic information of the entity node and improves the accuracy and expressiveness of the knowledge graph.

[0073] In an embodiment of the present specification, fusing the features of different modalities to generate a multi-modal feature vector can include: Step 1, for each entity node, according to the business scenario it belongs to, assign a scene correlation weight to the features of different modalities; In the present step, analyze the requirements of the business scenario, determine which modalities of features are more important in the current scenario, and assign a weight value to each modality of features, the more important the corresponding weight value is larger. For example, in the traffic congestion analysis scenario, the speed data of the vehicle (sensor modality) is more important than the color of the vehicle (image modality). Therefore, a higher weight can be assigned to the speed data and a lower weight to the color data.

[0074] Step 2, weight average the features of different modalities according to the scene correlation weight to obtain the preliminary fusion features; In the present step, multiply the feature vector of each modality by the corresponding weight, and then add the weighted feature vectors to obtain the preliminary fusion features.

[0075] Step 3, dimension reduction processing is performed on the preliminary fusion features to generate the final multi-modal feature vector; In the present step, assuming that the preliminary fusion features are [70.3, 0.5, 0.2, 0.1], through dimension reduction processing, the first two most important feature dimensions are retained, and the final multi-modal feature vector is [70.3, 0.5].

[0076] In the present embodiment, according to the business scenario to which the entity node belongs, the features of different modalities are assigned weights to highlight important features. Through dimension reduction processing, the feature dimension is reduced, the calculation efficiency is improved, and redundant information is removed.

[0077] In an embodiment of the present specification, embedding the generated multi-modal feature vector into the representation of the entity node can include: Step one, combining the multi-modal feature vector with the attributes of the entity node to form the entity node representation; Step two, mapping the entity node representation into the semantic space of the knowledge graph; In this step, the mapping process can be as follows: Define semantic space: Semantic space is a formal framework for describing the semantic information of entity nodes, usually including: ontology, which defines the categories, attributes and relationship types of entities; semantic model, which describes the semantic structure of entity nodes, including the semantic types of attributes and the semantic meanings of relationships; semantic mapping rules, which define how to convert the attributes and feature vectors of entity nodes into semantic representations in the knowledge graph.

[0078] Attribute semanticization: mapping the attribute values of the entity node to the ontology of the knowledge graph. For example, map the "vehicle type" attribute to the "vehicle type" semantic category; map the "color" attribute to the "appearance color" semantic category. This can be achieved through a predefined mapping table or semantic matching algorithm.

[0079] Feature semanticization: mapping the feature values in the multi-modal feature vector to the semantic space. For example, map the speed feature of the vehicle to the "driving speed" semantic category; map the image feature of the road to the "road condition" semantic category. Similarly, this can be done through predefined mapping rules or semantic analysis algorithms.

[0080] Overall mapping: mapping the integrated entity node representation to the semantic space of the knowledge graph. This can be achieved by constructing a mapping function that takes the representation of the entity node as input and outputs its position in the semantic space. For example, assuming the entity node representation is x1, x2, x3, the mapping function is f(x), where x represents the entity node to be input into the function, then the mapped semantic representation is f(x1, x2, x3). This mapping function can be linear or non-linear, depending on the complexity of the semantic space and the characteristics of the entity node representation.

[0081] Step three, in the knowledge graph, connect the mapped entity node with related entity nodes through relationship edges.

[0082] In the embodiment, the multimodal feature vector is combined with the attribute of the entity node, the semantic information of the entity node is enriched, the entity node representation is mapped to the semantic space of the knowledge graph, and the semantic consistency and accuracy of the entity node in the knowledge graph are ensured. By connecting the mapped entity node and the related entity node through the relationship edge in the knowledge graph, the structuring and correlation of the knowledge graph are enhanced.

[0083] In an embodiment of the present specification, the preliminary fusion features are dimensionally reduced to generate the final multimodal feature vector, including: Step one, the preliminary fusion features are divided into multiple levels of feature sets according to their inherent similarity, and each level of feature set represents different granularity of feature information; For example, a multimodal dataset containing vehicle operation data and road environment data is being processed. The preliminary fusion features may include vehicle speed, acceleration, fuel consumption, road traffic flow, weather conditions, etc. First level (fine granularity): the speed and acceleration of the vehicle are classified into a feature set because they are closely related to the dynamic behavior of the vehicle. For example, speed and acceleration can be used to describe the motion state of the vehicle. Second level (medium granularity): the fuel consumption of the vehicle and the traffic flow of the road are classified into another feature set because they are related to the operation efficiency of the vehicle and the use of the road. Third level (coarse granularity): the weather conditions (such as temperature, humidity, wind speed) are classified into a feature set because they have a macro influence on vehicle operation and road environment. In the above manner, the preliminary fusion features are divided into multiple levels of feature sets, and each level represents different granularity of feature information.

[0084] Step two, in each level of feature set, identify and retain local features that most reflect the feature information of that level; For example, first level (vehicle dynamic behavior): in the feature set of speed and acceleration, assume that the range of speed change is larger and can better reflect the motion state of the vehicle, so retain speed as a local key feature. Second level (vehicle operation efficiency and road use): in the feature set of fuel consumption and traffic flow, assume that traffic flow can better reflect the use of the road, so retain traffic flow as a local key feature. Third level (weather conditions): in the feature set of temperature, humidity and wind speed, assume that temperature has the greatest influence on vehicle operation and road environment, so retain temperature as a local key feature.

[0085] Step three, adjust the relationship between the retained local features to reduce the feature dimension; For example, adjusting the feature relationship can be specifically understood as follows: assuming that there is a certain correlation between the speed and the traffic flow, for example, the increase of the traffic flow can cause the decrease of the vehicle speed, the weight between the features can be adjusted by analyzing the correlation. For example, if the change of the traffic flow has a greater impact on the vehicle speed, the weight of the traffic flow can be appropriately increased.

[0086] For reducing the feature dimension, it can be specifically understood that, by adjusting the relationship between the features, it is found that the temperature has a relatively small impact on the vehicle operation, and thus the weight of the temperature can be reduced or even discarded in some cases to further reduce the feature dimension.

[0087] In the embodiment, by hierarchical aggregation, local reservation and global optimization, the feature dimension is gradually reduced while the key semantic information is reserved, thereby providing a high-quality feature vector for the construction of the knowledge graph.

[0088] Based on the same overall inventive concept, the application also protects a multi-data-driven vehicle-road cloud service panoramic knowledge graph construction device, as shown in Figure 2 Figure 2 is a structural schematic diagram of the multi-data-driven vehicle-road cloud service panoramic knowledge graph construction device provided by the embodiment of the application. The multi-data-driven vehicle-road cloud service panoramic knowledge graph construction device provided by the application is described below, and the multi-data-driven vehicle-road cloud service panoramic knowledge graph construction device described below can be correspondingly referred to the multi-data-driven vehicle-road cloud service panoramic knowledge graph construction method described above.

[0089] The multi-data-driven vehicle-road cloud service panoramic knowledge graph construction device comprises a data acquisition module 201, a preprocessing module 202, an extraction module 203, a merging module 204, a graph construction module 205, a graph updating module 206 and a platform construction module 207.

[0090] The data acquisition module 201 is configured to acquire multi-source heterogeneous data from a plurality of data sources in a traffic system. The preprocessing module 202 is configured to clean and standardize the multi-source heterogeneous data to obtain a data set. The extraction module 203 is configured to extract entities and relationships in the data set. The merging module 204 is configured to align and fuse the extracted entities and relationships to obtain an entity relationship set. The graph construction module 205 is configured to construct a graph structure of a knowledge graph by using the entity relationship set, and store the graph structure in a graph database. The graph updating module 206 is configured to dynamically update the knowledge graph in the graph database based on incremental updating. ​The platform construction module 207 is used to build a cloud-edge collaborative panoramic knowledge graph platform based on the knowledge graph in the graph database.

[0091] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0092] like Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communication interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a method for constructing a panoramic knowledge graph of vehicle-road-cloud services based on multiple data drives.

[0093] Furthermore, the logic instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0094] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data-driven methods provided by the above methods.

[0095] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives provided by the above methods.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0097] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives, characterized by: include: Acquire multi-source heterogeneous data from multiple data sources in the transportation system; Cleaning and standardizing the multi-source heterogeneous data to obtain a data set; Classifying the data in the data set according to attributes of the data in the data set, and extracting entities and relationships from the classified data; Align and fuse the extracted entities and relationships to obtain an entity relationship set; Using the entity relationship set, constructing a graph structure of a knowledge graph, and storing the graph structure in a graph database; When new data is detected in the data source, the knowledge graph in the graph database is dynamically updated; Based on the knowledge graph in the graph database, a cloud-edge collaborative panoramic knowledge graph platform is constructed.

2. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: The step of classifying the data in the data set according to the attributes of the data in the data set and extracting entities and relationships from the classified data includes: Performing a first clustering on the data set according to the types of multi-source heterogeneous data in the data set to obtain a plurality of initial clusters; For each of the initial clusters, a second clustering is performed based on the semantic information therein to obtain multiple subclusters; For each of the sub-clusters, data points with similar features are identified as entities of the same type, and interaction patterns between entities are identified as relationships.

3. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: When new data is detected in the data source, the knowledge graph in the graph database is dynamically updated, including: When new data is detected in the data source, the operating situation of the current traffic system is evaluated to obtain the evaluation results, and the priority of the update task is determined according to the evaluation results; wherein the priority includes high priority and low priority; Extracting entities and relationships of the newly added data; According to the operation scenario, select high-priority high entities and high relationships and low-priority low entities and low relationships from the newly added entities and relationships; Prioritize updating the high-level entities and the high-level relationships into the knowledge graph; Then update the low entity and the low relationship into the knowledge graph.

4. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: The method of constructing a graph structure of a knowledge graph by using the entity relationship set includes: According to the entity relationship set, each entity is regarded as an entity node in the graph, and the relationship between entities is regarded as the relationship edge in the graph to construct the basic graph structure of the knowledge graph; For each entity node, according to its spatiotemporal information, the entity nodes with spatiotemporal association are connected through spatiotemporal edges; For each relationship edge, add a spatiotemporal weight based on the time and space range in which it occurs; Combining the multimodal data features in multi-source heterogeneous data, multimodal feature enhancement is performed on the entity nodes in the basic graph structure.

5. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 4 is characterized in that: The multimodal data features in the multi-source heterogeneous data are combined to enhance the multimodal features of the entity nodes in the basic graph structure, including: For each entity node, its features in different modalities are extracted from multi-source heterogeneous data; Fuse the features of different modalities to generate a multimodal feature vector; The generated multimodal feature vector is embedded into the representation of the entity node.

6. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 5 is characterized in that: The method of fusing features of different modalities to generate a multimodal feature vector includes: For each entity node, assign scenario-relevance weights to features of different modalities based on the business scenario to which they belong; The features of different modalities are weighted averaged according to the scene relevance weights to obtain the preliminary fusion features; Perform dimensionality reduction on the preliminary fusion features to generate the final multimodal feature vector; The step of embedding the generated multimodal feature vector into the representation of the entity node includes: Combine the multimodal feature vector with the attributes of the entity node to form the entity node representation; Map entity node representations into the semantic space of the knowledge graph; In the knowledge graph, the mapped entity nodes are connected to the related entity nodes through relationship edges.

7. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: The extracted entities and relationships are aligned and merged to obtain an entity relationship set, including: Standardize the extracted entities to unify the entity representations in different data sources to the same standard; Semantic alignment is performed on the extracted relationships to merge relationships whose semantic similarity exceeds a preset similarity threshold but has different expressions; Entities and relationships with the same context are associated to obtain an entity relationship set.

8. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: The cloud-edge collaborative panoramic knowledge graph platform is constructed based on the knowledge graph in the graph database, including: Deploy a graph database in the cloud to centrally store graph structure data of the knowledge graph; Deploy a knowledge graph cache module and a real-time computing module at the edge to process the real-time data generated by edge devices and feed the processing results back to the cloud. Build a visual interface for the panoramic knowledge graph platform, providing graph display and interactive functions.

9. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: The multiple data sources include: Vehicle-side data source, used to provide vehicle operating status data; Roadside sensing equipment data source, used to provide road environment data; Cloud-based business system data source, used to provide traffic management and service data; External knowledge base data source, used to provide traffic-related knowledge data; User device data sources, used to provide user behavior and preference data; Third-party service data source, used to provide third-party service data related to traffic.

10. The method for constructing a panoramic knowledge graph of vehicle-road-cloud business based on multiple data drives according to claim 1 is characterized in that: The cleaning and standardization of the multi-source heterogeneous data to obtain a data set includes: Removing outliers and noise data from the multi-source heterogeneous data; Filling missing values ​​in the multi-source heterogeneous data; Converting data of different formats in the multi-source heterogeneous data into a unified format; Normalize the multi-source heterogeneous data to obtain the data set.

Images