A data lake warehouse integrated management method and system based on port traffic

Abstract

This invention discloses a data lake warehouse integrated management method and system based on port traffic, specifically relating to the field of data management technology. It involves collecting multi-source heterogeneous traffic data, performing unified timestamp annotation and format standardization to obtain preprocessed multi-source heterogeneous traffic data, extracting semantic features and performing hierarchical analysis based on predefined business ontology and semantic meta-model, and constructing semantic mapping rules. It also generates semantic drift correction parameters by combining historical semantic drift accumulated deviations; generates alignment strategies by constructing a dynamic semantic alignment and correction mechanism; and performs semantic correction and fusion based on the alignment strategies to output semantically aligned lake warehouse fusion data, thereby improving the accuracy of lake warehouse collaborative management and the consistency of data analysis.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and more specifically, to a data lake warehouse integrated management method and system based on port traffic. Background Technology

[0002] Existing port traffic data management systems have significant limitations in handling integrated lake-warehouse processing in complex and heterogeneous data environments. Although current data lake and data warehouse architectures can respectively handle raw data accumulation and structured analysis tasks, the lack of a unified semantic alignment mechanism becomes a key bottleneck affecting data consistency during actual lake-warehouse transfer. This leads to semantic drift during the switching and translation of data from the lake to the warehouse, causing semantic distortion in the processing chains such as metadata inheritance, field transformation, and data fusion. Summary of the Invention

[0003] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a data lake warehouse integrated management method and system based on port traffic to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A data lake warehouse integrated management method based on port traffic includes the following steps:

[0006] S1: Collect multi-source heterogeneous traffic data, and unify the timestamps and standardize the format to obtain preprocessed multi-source heterogeneous traffic data;

[0007] S2: Based on the predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on the preprocessed heterogeneous traffic data to generate a semantic feature set;

[0008] S3: Construct a semantic mapping rule set based on the semantic feature set, and generate initial mapping rules through ontology alignment, multi-source label mapping and field caliber mapping;

[0009] S4: Perform historical semantic drift cumulative deviation analysis on the semantic feature set to generate semantic drift correction parameters;

[0010] S5: Combine the initial mapping rules with semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism and generate an alignment strategy;

[0011] S6: Based on the alignment strategy, perform semantic correction and fusion on multi-source heterogeneous traffic data in the data lake and data warehouse switching process, and output semantically aligned lake warehouse fused data.

[0012] In a preferred embodiment, S1 specifically refers to:

[0013] Vehicle navigation data is collected from the automated guided vehicle navigation system, parking space scheduling log data is collected from the parking space scheduling system, and real-time vehicle coordinate data is collected from the vehicle positioning system.

[0014] Timestamp alignment is performed on vehicle navigation data, parking space scheduling log data, and real-time vehicle coordinate data respectively.

[0015] Reconstruct the fields of the vehicle navigation data to generate a navigation data table;

[0016] Convert the data type of the berth scheduling log data to generate a berth status code table;

[0017] Perform coordinate system transformation on real-time vehicle coordinate data to generate a standard geographic coordinate sequence;

[0018] Based on navigation data tables, berth status coding tables, and standard geographic coordinate sequences, preprocessed multi-source heterogeneous traffic data is generated.

[0019] In a preferred embodiment, S2 specifically refers to:

[0020] Based on the port traffic concept set in the predefined business ontology library, the semantics of the fields in the navigation data table are parsed to extract vehicle speed feature values, steering angle feature values ​​and path planning feature values;

[0021] Based on the attribute relationship tree in the predefined business ontology library, the field semantics of the berth status coding table are parsed to extract berth occupancy status feature values, operation timeliness feature values ​​and equipment association feature values;

[0022] The spatial location parsing rules in the predefined semantic meta-model are invoked to parse the field semantics of the standard geographic coordinate sequence and extract the feature values ​​of the coordinate point sequence and the movement trajectory feature values.

[0023] Integrate all feature values ​​to generate a semantic feature set.

[0024] In a preferred embodiment, S3 specifically refers to:

[0025] Ontology alignment is performed on vehicle speed feature values, steering angle feature values, and path planning feature values ​​in the semantic feature set. The attribute definitions of vehicle trajectory concepts in the predefined business ontology library are matched and multi-source label mapping rules are generated.

[0026] Field mapping is performed on the berth occupancy status feature value, operation timeliness feature value and equipment association feature value in the semantic feature set, and constraint rules between feature values ​​and data warehouse fields are established based on the attribute relationship tree of the predefined business ontology library;

[0027] The spatial relationship parsing rules of the predefined semantic meta-model are invoked on the coordinate point sequence feature values ​​and movement trajectory feature values ​​in the semantic feature set to generate a standardized conversion protocol for the geographic coordinate field;

[0028] An initial mapping rule set is generated based on multi-source label mapping rules, constraint rules, and standardized conversion protocols.

[0029] In a preferred embodiment, S4 specifically refers to:

[0030] The vehicle speed feature value, steering angle feature value, and path planning feature value in the semantic feature set are divided into time slices, and the feature mean drift of historical data is calculated based on a predefined time window.

[0031] A status coding distribution analysis was performed on the berth occupancy status feature value, operation timeliness feature value and equipment association feature value in the semantic feature set, and the status feature offset rate was calculated based on the historical frequency distribution.

[0032] The spatial offset calculation rule is applied to the coordinate point sequence feature value and the movement trajectory feature value in the semantic feature set to calculate the spatial position drift vector based on the historical geographic coordinate reference point sequence.

[0033] The mean drift of vehicle speed features, mean drift of steering angle features, mean drift of path planning features, offset rate of parking space status features, offset rate of equipment association features, and spatial position drift vector are summarized to generate semantic drift correction parameters.

[0034] In a preferred embodiment, S5 specifically refers to:

[0035] Based on the mean drift of vehicle speed features, mean drift of steering angle features, and mean drift of path planning features in the semantic drift correction parameters, the weight allocation coefficients in the multi-source label mapping rules are adjusted.

[0036] Based on the berth status feature offset rate and equipment association feature offset rate in the semantic drift correction parameters, the allowable floating range of the status code in the extended constraint rules is determined.

[0037] Update the coordinate compensation parameters in the standardized transformation protocol based on the spatial position drift vector in the semantic drift correction parameters;

[0038] Based on the multi-source label mapping rules after adjusting the weight allocation coefficients, the constraint rules after expanding the allowed floating range, and the standardized transformation protocol after updating the coordinate compensation parameters, an alignment strategy corresponding to the dynamic semantic alignment and correction mechanism is generated.

[0039] In a preferred embodiment, S6 specifically refers to:

[0040] Multi-source heterogeneous traffic data is extracted from the data lake. Based on the multi-source label mapping rules in the alignment strategy, field-level weights are assigned to the vehicle navigation data, and the first round of semantic correction is performed.

[0041] The constraint rules in the alignment strategy are invoked to perform status encoding and allowable floating range verification on the berth scheduling log data, and a second round of semantic correction is performed.

[0042] The standardized transformation protocol in the alignment strategy is applied to perform coordinate compensation calculation on real-time vehicle coordinate data, and a third round of semantic correction is executed.

[0043] By integrating the vehicle navigation data after the first round of semantic correction, the parking space scheduling log data after the second round of semantic correction, and the real-time vehicle coordinate data after the third round of semantic correction, semantically aligned lake-warehouse fusion data is generated.

[0044] On the other hand, the present invention provides an integrated data lake warehouse management system based on port traffic, comprising:

[0045] Data acquisition module: Collects multi-source heterogeneous traffic data, unifies timestamps and standardizes formats to obtain pre-processed multi-source heterogeneous traffic data;

[0046] Semantic extraction module: Based on predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on preprocessed heterogeneous traffic data to generate a semantic feature set;

[0047] Mapping rule module: Constructs a semantic mapping rule set based on semantic feature set, and generates initial mapping rules through ontology alignment, multi-source label mapping and field caliber mapping;

[0048] Drift analysis module: Performs historical semantic drift cumulative deviation analysis on the semantic feature set and generates semantic drift correction parameters;

[0049] Alignment mechanism module: Combines initial mapping rules and semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism and generate alignment strategies;

[0050] Fusion Output Module: Based on the alignment strategy, semantic correction and fusion of multi-source heterogeneous traffic data in the data lake and data warehouse switching process are performed, and semantically aligned lake warehouse fused data is output.

[0051] The technical effects and advantages of the integrated data lake warehouse management method and system based on port traffic of this invention are as follows:

[0052] By unifying the preprocessing of multi-source heterogeneous traffic data, the consistency of data structure is improved; semantic feature extraction is performed by combining predefined business ontology and semantic meta-model, enhancing the parsability of data at the semantic layer; by constructing semantic mapping rules, the labels and definitions of data from different sources are unified, solving the correspondence problem of heterogeneous fields; by analyzing the cumulative deviation of semantic drift, inconsistency trends in the data evolution process are identified; by adjusting mapping rules and constraint strategies, the semantic correction capability to adapt to data changes is achieved; by completing the semantic alignment and fusion between the data lake and the data warehouse, the usability and consistency of data after migration are improved, enhancing the semantic stability, query consistency, and analytical accuracy of port traffic data in the cross-system flow process. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of a data lake warehouse integrated management method based on port traffic according to the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of a data lake warehouse integrated management system based on port traffic according to the present invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. Example 1

[0056] Figure 1 This invention presents a data lake warehouse integrated management method based on port traffic, which includes the following steps:

[0057] S1: Collect multi-source heterogeneous traffic data, and unify the timestamps and standardize the format to obtain preprocessed multi-source heterogeneous traffic data;

[0058] S2: Based on the predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on the preprocessed heterogeneous traffic data to generate a semantic feature set;

[0059] S3: Construct a semantic mapping rule set based on the semantic feature set, and generate initial mapping rules through ontology alignment, multi-source label mapping and field caliber mapping;

[0060] S4: Perform historical semantic drift cumulative deviation analysis on the semantic feature set to generate semantic drift correction parameters;

[0061] S5: Combine the initial mapping rules with semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism and generate an alignment strategy;

[0062] S6: Based on the alignment strategy, perform semantic correction and fusion on multi-source heterogeneous traffic data in the data lake and data warehouse switching process, and output semantically aligned lake warehouse fused data.

[0063] S1: Collect multi-source heterogeneous traffic data, and standardize the timestamps and formats to obtain preprocessed multi-source heterogeneous traffic data, including:

[0064] Vehicle navigation data is collected from the automated guided vehicle navigation system, parking space scheduling log data is collected from the parking space scheduling system, and real-time vehicle coordinate data is collected from the vehicle positioning system.

[0065] Timestamp alignment is performed on vehicle navigation data, parking space scheduling log data, and real-time vehicle coordinate data respectively.

[0066] Reconstruct the fields of the vehicle navigation data to generate a navigation data table;

[0067] Convert the data type of the berth scheduling log data to generate a berth status code table;

[0068] Perform coordinate system transformation on real-time vehicle coordinate data to generate a standard geographic coordinate sequence;

[0069] Based on navigation data tables, berth status coding tables, and standard geographic coordinate sequences, preprocessed multi-source heterogeneous traffic data is generated.

[0070] Specifically, the Automated Guided Vehicle (AVR) system is a management system used in ports to guide and control the operating path of automated guided vehicles; the berth scheduling system is an operational system used to control and manage the status of each berth in the port; and the vehicle positioning system is a positioning management system used to monitor and record the geographical location information of vehicles in real time. First, vehicle navigation trajectory information is collected from the automated guided vehicle (AGV) navigation system using a preset data interface or data transmission method. This trajectory information includes, but is not limited to, vehicle number, current location, current speed, current direction and angle of travel, and planned route information. For example, the AGV navigation system can output in real-time the location coordinates of an AGV named "AGV-001" at a specific time point: 30.12345°N, 120.54321°E, speed of 2.5 meters per second, direction of travel 15 degrees due north, and the set of planned route nodes for the next 10 minutes. Simultaneously, berth allocation status information is collected from the berth scheduling system using a preset data interface or data transmission method. This status information includes, but is not limited to, berth number, berth occupancy status indicator, identification number of the vehicle or vessel occupying the berth, and duration of berth occupancy. The system also provides real-time data on the expected release time of berths. For example, the berth scheduling system can output in real time that a berth numbered "B-12" is currently occupied, the vehicle occupying the berth is numbered "TC-003", the berth has been occupied for 25 minutes, and the berth is expected to be released in the next 45 minutes. Real-time geographic coordinate information of vehicles is collected from the vehicle positioning system using a preset data interface or data transmission method. This real-time geographic coordinate information includes, but is not limited to, the vehicle number, the longitude and latitude coordinates of the vehicle's current location, the identifier of the port area where the vehicle is located, and the time of current location data collection. For example, the vehicle positioning system can output in real-time the latitude and longitude coordinates of a port transport vehicle numbered "TRUCK-005" as 30.22222 degrees North latitude and 120.33333 degrees East longitude, the port area where the vehicle is located as "Store Yard Area A", and mark the location data collection time as 10:23:45 on April 10, 2025. The above-mentioned automated guided vehicle navigation system, parking space scheduling and vehicle positioning system collect data and generate three different types of data respectively: vehicle navigation data, parking space scheduling log data and real-time vehicle coordinate data.

[0071] Each data record of vehicle navigation data, berth scheduling log data, and real-time vehicle coordinate data is stamped with a standardized timestamp, accurate to the millisecond level. The timestamp time base is uniformly adopted as the international standard time base, i.e., Coordinated Universal Time. For example, vehicle navigation data records the position and movement status of a vehicle at a certain moment, so a timestamp is added to each position and movement status data record. Similarly, berth scheduling log data records the time of berth occupancy information, and a standardized timestamp is added to each berth scheduling log record. Real-time vehicle coordinate data is stamped with a corresponding time mark according to a standardized timestamp format; for example, when the vehicle positioning system records the real-time coordinates of a vehicle, it is uniformly marked with a standardized timestamp of "2025-04-10 02:23:45.123 UTC". Through the above process of aligning the timestamps of vehicle navigation data, berth scheduling log data, and real-time vehicle coordinate data, the consistency of the data collection and recording times is ensured.

[0072] The vehicle navigation data undergoes field reconstruction, which involves standardizing field names and adjusting field order according to a preset standard format. For example, the "Vehicle Identifier" field is uniformly reconstructed as "Vehicle Number," and the "Current Location Coordinates" field is uniformly reconstructed as "Current Navigation Coordinates" to unify field names. Secondly, the berth scheduling log data undergoes data type conversion. For instance, fields representing occupancy duration, which might be string types, are converted to integer or floating-point types. Finally, the real-time vehicle coordinate data undergoes coordinate system conversion, transforming the WGS84 geographic coordinate system used in the real-time vehicle coordinate data into the port's unified standard geographic coordinate system. After field reconstruction, data type conversion, and coordinate system conversion, a navigation data table, a berth status code table, and a standard geographic coordinate sequence conforming to unified specifications are generated, respectively.

[0073] The navigation data table, berth status coding table, and standard geographic coordinate sequence are integrated to generate preprocessed multi-source heterogeneous traffic data.

[0074] S2: Based on a predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on the preprocessed heterogeneous traffic data to generate a semantic feature set, including:

[0075] Based on the port traffic concept set in the predefined business ontology library, the semantics of the fields in the navigation data table are parsed to extract vehicle speed feature values, steering angle feature values ​​and path planning feature values;

[0076] Based on the attribute relationship tree in the predefined business ontology library, the field semantics of the berth status coding table are parsed to extract berth occupancy status feature values, operation timeliness feature values ​​and equipment association feature values;

[0077] The spatial location parsing rules in the predefined semantic meta-model are invoked to parse the field semantics of the standard geographic coordinate sequence and extract the feature values ​​of the coordinate point sequence and the movement trajectory feature values.

[0078] Integrate all feature values ​​to generate a semantic feature set.

[0079] Specifically, the navigation data table includes vehicle number, vehicle position, vehicle speed, vehicle steering angle, and planned vehicle path. The field semantics of the navigation data table refers to the business meaning corresponding to each field; for example, the vehicle speed field corresponds to the vehicle's real-time speed. The port traffic concept set is a predefined set of concepts based on actual port operations, including vehicle navigation, vehicle movement mode, vehicle steering, path planning, trajectory tracking, and path nodes. Each concept has a unified definition, attribute description, and attribute value type specification. For example, in the port traffic concept set, vehicle speed is defined as the distance a vehicle moves per unit time, with the unit being meters per second and the data type being floating-point; steering angle is defined as the angle between the vehicle's current direction and due north, with the unit being degrees; and path planning is defined as the sequence of nodes the vehicle is expected to pass through within a preset future time, with nodes represented by latitude and longitude coordinates. Based on the concept definitions and attribute rules in the port traffic concept set of the predefined business ontology library, the semantic meaning of the corresponding fields in the navigation data table is parsed to obtain the business semantics. For example, the field named "Current Navigation Speed" in the navigation data table is parsed according to the vehicle speed concept definition in the port traffic concept set to obtain the vehicle speed feature value; the "Direction Angle" field in the navigation data table is parsed according to the "Steering Angle" concept definition to obtain the steering angle feature value; and the "Future Path" field in the navigation data table is parsed according to the "Path Planning" concept to obtain the path planning feature value, which is a sequence of latitude and longitude coordinate nodes arranged in order. After parsing, the vehicle speed feature value, steering angle feature value, and path planning feature value were successfully extracted from the navigation data table.

[0080] The berth status coding table is a data table generated after converting berth scheduling log data to different data types. It records the berth's occupancy status, the number of the occupying vehicle, the occupancy duration, and the estimated release time. The attribute relationship tree in the predefined business ontology library is a hierarchical tree structure defining the relationships between conceptual attributes based on the actual business situation of port berth management. It reflects the correspondence between higher-level and lower-level attributes of berth status. For example, the berth occupancy status attribute may include lower-level attributes such as occupancy / idle status, occupancy duration, and estimated release time. The operation timeliness attribute represents the berth occupancy timeliness and operation efficiency, with lower-level attributes including average occupancy duration and occupancy frequency. The equipment association attribute represents the associated operation equipment number information when the berth is occupied. Based on the attribute relationship tree in the predefined business ontology library, the business semantics of each field in the berth status coding table are parsed field by field to obtain feature values. For example, the "Current Status" field in the berth status code table, after parsing according to the definition of the berth occupancy status attribute in the attribute relationship tree, yields the berth's current occupied or vacant status information, i.e., the berth occupancy status characteristic value; the "Occupancy Duration" field, parsed according to the "Operation Timeliness" attribute, yields the "Operation Timeliness" characteristic value; and the "Associated Equipment Number" field, parsed according to the "Equipment Association" attribute, yields the "Equipment Association" characteristic value. For instance, if the berth status code table records berth "B-12" with the currently occupied equipment number "CRANE-03," this equipment association number is the equipment association characteristic value. Through parsing, the berth occupancy status characteristic value, operation timeliness characteristic value, and equipment association characteristic value are extracted from the berth status code table.

[0081] The standard geographic coordinate sequence is a data sequence obtained by transforming real-time vehicle coordinate data into a coordinate system, recording the real-time geographic location coordinates of vehicles in the port. The spatial location resolution rules in the predefined semantic meta-model are a set of geospatial semantic resolution rules defined according to the port operation scenario, including spatial location point definition rules, spatial trajectory rules, and location point sequence sorting rules. For example, the spatial location point definition rule specifies that a single latitude and longitude coordinate point is resolved as a spatial location identifier within the port area, such as the coordinates 30.22222 degrees North latitude and 120.33333 degrees East longitude corresponding to the identifier of yard area A; the spatial trajectory rule specifies that a continuous location point sequence is resolved as a vehicle movement path; and the location point sequence sorting rule specifies that vehicle location point data are arranged sequentially from earliest to latest according to timestamps. By calling the spatial location resolution rules to parse the semantic meaning of each record's fields in the standard geographic coordinate sequence, the feature values ​​of the coordinate point sequence and the movement trajectory feature values ​​are extracted. For example, a vehicle's standard geographic coordinate sequence includes three consecutive location points: "30.22222°N, 120.33333°E", "30.22333°N, 120.33444°E", and "30.22444°N, 120.33555°E". After being analyzed using spatial location analysis rules, the feature values ​​of the coordinate point sequence arranged in chronological order and the feature values ​​of the vehicle's movement trajectory formed by these coordinates are obtained, describing the vehicle's position change pattern.

[0082] The vehicle speed feature value, steering angle feature value, path planning feature value, parking space occupancy status feature value, operation timeliness feature value, equipment association feature value, coordinate point sequence feature value and movement trajectory feature value obtained from the above analysis are summarized and integrated to generate a semantic feature set.

[0083] S3: Construct a semantic mapping rule set based on the semantic feature set, and generate initial mapping rules through ontology alignment, multi-source label mapping, and field caliber mapping, including:

[0084] Ontology alignment is performed on vehicle speed feature values, steering angle feature values, and path planning feature values ​​in the semantic feature set. The attribute definitions of vehicle trajectory concepts in the predefined business ontology library are matched and multi-source label mapping rules are generated.

[0085] Field mapping is performed on the berth occupancy status feature value, operation timeliness feature value and equipment association feature value in the semantic feature set, and constraint rules between feature values ​​and data warehouse fields are established based on the attribute relationship tree of the predefined business ontology library;

[0086] The spatial relationship parsing rules of the predefined semantic meta-model are invoked on the coordinate point sequence feature values ​​and movement trajectory feature values ​​in the semantic feature set to generate a standardized conversion protocol for the geographic coordinate field;

[0087] An initial mapping rule set is generated based on multi-source label mapping rules, constraint rules, and standardized conversion protocols.

[0088] Specifically, ontology alignment involves matching feature values ​​against the defined attributes of a predefined port traffic business ontology database, based on the concept of vehicle trajectories. This determines whether a matching relationship exists between each feature value and the attributes of the vehicle trajectory concept. The attribute definitions for the vehicle trajectory concept in the business ontology database include three attributes: vehicle speed, vehicle direction angle, and the planned path nodes the vehicle will traverse. Each attribute definition includes a data type and a numerical range standard. For example, the vehicle speed attribute is defined as a floating-point number in meters per second, with a range of 0 to 10 meters per second; the vehicle direction angle attribute is defined as a floating-point number in degrees, with a range of 0 to 360 degrees; and the planned path node attribute is defined as a floating-point pair of latitude and longitude coordinates, requiring each node to have at least five decimal places and to be a continuous sequence of path nodes. The system matches vehicle speed feature values ​​with vehicle operating speed attribute definitions to generate vehicle speed label mapping rules; it matches steering angle feature values ​​with vehicle direction angle attribute definitions to generate steering angle label mapping rules; and it matches path planning feature values ​​with vehicle planned path node attribute definitions to generate path planning label mapping rules. These three rules are combined to form multi-source label mapping rules, which serve as the rule basis for semantically unified mapping of vehicle navigation data.

[0089] Field mapping is based on the attribute relationship tree of berth management in the predefined business ontology library. The attribute relationship tree defines the correspondence and numerical constraint standards between each berth-related attribute and the fields in the data warehouse. In the attribute relationship tree, the berth occupancy status attribute corresponds to the data warehouse field "Occupancy Status Field," with a value range constraint of "occupied" or "idle"; the operation timeliness attribute corresponds to the data warehouse field "Berth Occupancy Duration Field," with a value type of integer or floating-point number in minutes; the equipment association attribute corresponds to the data warehouse field "Associated Equipment Field," with a constraint that the field data type is string and must conform to the port equipment number standard format. Mapping the berth occupancy status feature value to the data warehouse occupancy status field generates berth status constraint rules; mapping the operation timeliness feature value to the data warehouse berth occupancy duration field generates operation timeliness constraint rules; mapping the equipment association feature value to the data warehouse associated equipment field generates equipment association constraint rules. These three constraint rules combine to form the overall constraint rule set between the berth status coding table feature values ​​and the data warehouse fields, serving as the rule basis for the unified semantic constraint processing of the data warehouse.

[0090] The predefined spatial relationship parsing rules of the semantic meta-model predefine rules for converting between spatial location coordinates and standard geographic coordinate systems. These rules specify how the feature values ​​of each coordinate point and its trajectory should be accurately mapped and uniformly processed between geographic information system coordinate systems. The spatial relationship parsing rules are invoked to parse the feature values ​​of the coordinate point sequence and the movement trajectory feature values, performing spatial coordinate transformation point by point and confirming that the transformed coordinate point sequence is consistent with the standard coordinate system. For example, the coordinate sequence is uniformly converted from the WGS84 coordinate system to the port's unified UTM coordinate system, and the accuracy of the coordinate points is confirmed to meet standard requirements, generating a standardized conversion protocol for geographic coordinate fields.

[0091] The initial mapping rule set is generated by combining multi-source label mapping rules, feature value and data warehouse field constraint rules, and standardized conversion protocol of geographic coordinate fields, which serves as the unified rule basis for dynamic semantic alignment processing and integrated data lake warehouse processing.

[0092] S4: Perform historical semantic drift cumulative bias analysis on the semantic feature set to generate semantic drift correction parameters, including:

[0093] The vehicle speed feature value, steering angle feature value, and path planning feature value in the semantic feature set are divided into time slices, and the feature mean drift of historical data is calculated based on a predefined time window.

[0094] A status coding distribution analysis was performed on the berth occupancy status feature value, operation timeliness feature value and equipment association feature value in the semantic feature set, and the status feature offset rate was calculated based on the historical frequency distribution.

[0095] The spatial offset calculation rule is applied to the coordinate point sequence feature value and the movement trajectory feature value in the semantic feature set to calculate the spatial position drift vector based on the historical geographic coordinate reference point sequence.

[0096] The mean drift of vehicle speed features, mean drift of steering angle features, mean drift of path planning features, offset rate of parking space status features, offset rate of equipment association features, and spatial position drift vector are summarized to generate semantic drift correction parameters.

[0097] Specifically, vehicle speed characteristic values, steering angle characteristic values, and path planning characteristic values ​​are divided into time slices according to predefined port data analysis standard time windows. The time window is a duration uniformly set in the port traffic data analysis process. For example, a standard time window is 30 minutes. The vehicle speed characteristic value, steering angle characteristic value, and path planning characteristic value data collected during port vehicle operation are divided into each corresponding 30-minute time segment, and the characteristic value data in each time segment are analyzed independently.

[0098] Taking vehicle speed characteristic values ​​as an example, the actual speed characteristic value data collected by automated guided vehicles (AGVs) in the port are divided and analyzed. For instance, AGV-001 operates between 7:00 AM and 8:00 AM, collecting 120 vehicle speed characteristic values, ranging from 2.1 meters per second to 2.9 meters per second. These 120 vehicle speed characteristic values ​​are divided into two time segments, each lasting 30 minutes: the first time window is from 7:00 AM to 7:30 AM, and the second time window is from 7:30 AM to 8:00 AM. Within each time window, the average value of all vehicle speed characteristic values ​​is calculated. The average value for the first time window is 2.5 meters per second, and the average value for the second time window is 2.7 meters per second. Based on historical data comparison, for example, using the average speed calculated from the vehicle speed data of the same time period over the past 10 consecutive days as a historical benchmark of 2.4 meters per second, the deviation between the vehicle speed characteristic value in the current window and the historical benchmark value is the characteristic mean drift. For example, the characteristic mean drift in the first window is 0.1 meters per second, and in the second window it is 0.3 meters per second. The same method also applies to steering angle characteristic values ​​and path planning characteristic values. For example, the historical average steering angle is 12 degrees north, and the current window value is 15 degrees, with a deviation of 3 degrees; the historical deviation number of nodes for the path planning characteristic value is 1, and the deviation number in the current window increases to 2. Therefore, the vehicle speed characteristic mean drift, steering angle characteristic mean drift, and path planning characteristic mean drift are obtained through the above methods.

[0099] The berth occupancy status characteristic value indicates whether the berth is occupied within a specific time period, such as "occupied" or "idle" status; the operation timeliness characteristic value indicates the duration of berth occupancy, such as the occupancy time of berth number B-12 being 25 minutes; the equipment association characteristic value indicates the equipment number associated with the berth when it is occupied, such as CRANE-03.

[0100] According to the port berth operation analysis specifications, status coding is carried out. That is, each status of the berth occupancy status characteristic value, such as "occupied" or "idle", is assigned a unified and unique code. For example, "occupied" is coded as "1" and "idle" is coded as "0". The operation timeliness characteristic value is coded according to the predetermined timeliness interval. For example, the occupancy timeliness of less than 30 minutes is coded as "short timeliness (01)", the occupancy timeliness of 30 to 60 minutes is coded as "medium timeliness (02)", and the occupancy timeliness of more than 60 minutes is coded as "long timeliness (03)". The equipment association characteristic value is coded according to the unified format of port equipment number. For example, the unified format of port hoisting equipment number is "CRANE-XX".

[0101] The frequency of each coded status in historical data was statistically analyzed using historical frequency distribution analysis. Taking berth occupancy status as an example, the frequency of berth occupancy code "1" in historical data was an average of 60 times per day. In the current data window, the frequency of berth occupancy code "1" is 80 times. Therefore, the frequency of the current status code "1" has increased by 20 times compared to the historical frequency, with an offset rate of 33.3%. The same analysis was performed on the operational timeliness characteristic value and the equipment association characteristic value to calculate the berth status characteristic offset rate and the equipment association characteristic offset rate.

[0102] The coordinate point sequence feature value and the movement trajectory feature value are the actual location point sequence collected by the vehicle within the port area. For example, the continuous geographic coordinate sequence that the vehicle TRUCK-005 passes through at a specific time includes three points: 30.22222 degrees north latitude and 120.33333 degrees east longitude, 30.22333 degrees north latitude and 120.33444 degrees east longitude, and 30.22444 degrees north latitude and 120.33555 degrees east longitude.

[0103] The system invokes predefined spatial offset calculation rules in the field of port data analysis. These rules stipulate that the spatial position drift is calculated by comparing the feature values ​​of each location point sequence collected within the current window with a historical sequence of geographic coordinate reference points for the same location, thus obtaining a vector of spatial coordinate offset. For example, if the historical geographic coordinate reference point sequence is 30.22220 degrees North latitude and 120.33330 degrees East longitude, then the spatial position drift of the first location point is a two-dimensional drift vector of 0.00002 degrees North and 0.00003 degrees East. After comparing each point in the entire location point sequence, the complete spatial position drift vector is obtained.

[0104] By systematically organizing and summarizing all feature drift amounts and offset rates—namely, the average drift amount of vehicle speed features, the average drift amount of steering angle features, the average drift amount of path planning features, the offset rate of berth status features, the offset rate of equipment association features, and the spatial position drift vector—a set of semantic drift correction parameters is generated. This serves as the input data foundation for the dynamic semantic correction mechanism, ensuring data consistency and accuracy in port data warehouse management.

[0105] S5: Combine the initial mapping rules with semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism, generating alignment strategies, including:

[0106] Based on the mean drift of vehicle speed features, mean drift of steering angle features, and mean drift of path planning features in the semantic drift correction parameters, the weight allocation coefficients in the multi-source label mapping rules are adjusted.

[0107] Based on the berth status feature offset rate and equipment association feature offset rate in the semantic drift correction parameters, the allowable floating range of the status code in the extended constraint rules is determined.

[0108] Update the coordinate compensation parameters in the standardized transformation protocol based on the spatial position drift vector in the semantic drift correction parameters;

[0109] Based on the multi-source label mapping rules after adjusting the weight allocation coefficients, the constraint rules after expanding the allowed floating range, and the standardized transformation protocol after updating the coordinate compensation parameters, an alignment strategy corresponding to the dynamic semantic alignment and correction mechanism is generated.

[0110] Specifically, the multi-source label mapping rule is a unified mapping rule formed by matching vehicle speed feature values, steering angle feature values, and path planning feature values ​​with the vehicle trajectory concept attribute definitions in the business ontology library through ontology alignment. It includes weight allocation coefficients for each feature value. Taking vehicle speed feature values ​​as an example, the original multi-source label mapping rule assigned a weight coefficient of 0.3 to the vehicle speed feature value. Now, based on the mean drift of vehicle speed feature values ​​being 0.1 meters per second, the weights are adjusted. Specifically, the original weight coefficients are corrected using a dynamic weight adjustment formula, for example, the adjustment formula is:

[0111] Adjusted weight coefficient = original weight coefficient × (1 + feature mean drift / historical benchmark value);

[0112] For example, if the historical baseline is 2.4 meters per second and the characteristic mean drift is 0.1 meters per second, then the adjusted weighting coefficient = 0.3 × (1 + 0.1 / 2.4) = 0.3125.

[0113] The updated weighting coefficient for the vehicle speed feature value is 0.3125. Similarly, the weighting coefficients for the steering angle and path planning feature values ​​are calculated and adjusted as described above. For example, in the original rule, the weighting coefficient for the steering angle feature value was 0.4, and the weighting coefficient for the path planning feature value was 0.3. After calculating the feature mean drift, the new weighting coefficients are 0.41 and 0.325, respectively. This adjustment results in the adjusted weighting coefficients of the multi-source label mapping rule, enhancing its adaptability to real-time vehicle data drift.

[0114] Constraint rules refer to the restrictive rules applied when mapping berth occupancy status characteristics, operational timeliness characteristics, and equipment-related characteristics to corresponding fields in the data warehouse. For example, in the original rule, the allowed frequency fluctuation range for the berth status "occupancy" was within ±20% of the historical baseline value. Now, based on a feature offset rate of 33.3%, the allowable fluctuation range is expanded to ±40% of the historical baseline value. For instance, the original allowed fluctuation range was 48 to 72 times (baseline 60 times ± 20%), but after adjustment, the allowed fluctuation range is expanded to 36 to 84 times (baseline 60 times ± 40%). The same method is applied to equipment-related characteristics, such as "CRANE-03," whose allowed fluctuation range is expanded from the original 40 to 60 times to 30 to 70 times to adapt to the current actual data change trend.

[0115] The spatial position drift vector is a vector calculated from the deviation between the currently measured real-time geographic coordinates of a vehicle and the historical reference coordinates. For example, if the vehicle's current real-time geographic coordinates are 30.22222°N, 120.33333°E, and the historical reference coordinates are 30.22220°N, 120.33330°E, then the difference in latitude (0.00002°N) and longitude (0.00003°E) constitutes the spatial position drift vector.

[0116] The coordinate compensation parameters in the normalized transformation protocol are updated based on the spatial position drift vector. The normalized transformation protocol is a coordinate transformation protocol generated through the spatial relationship parsing rules of the semantic meta-model, and includes compensation parameters for spatial coordinate calibration. Taking the update formula for the compensation parameters as an example:

[0117] Updated compensation parameters = original compensation parameters + spatial position drift vector × correction coefficient;

[0118] The correction coefficient is preset. For example, if the correction coefficient is 0.8 and the original compensation parameter is defined as 0.00000, then the new coordinate compensation parameter after calculation is:

[0119] North latitude compensation parameter = 0 + 0.00002 × 0.8 = 0.000016 degrees;

[0120] East longitude compensation parameter = 0 + 0.00003 × 0.8 = 0.000024 degrees;

[0121] The spatial accuracy of vehicle coordinate data is improved by using updated coordinate compensation parameters.

[0122] The alignment strategy is formed by integrating the multi-source label mapping rules after adjusting the weighting coefficients, the constraint rules after expanding the floating range, and the standardized transformation protocol after updating the coordinate compensation parameters to create a dynamic semantic alignment and correction mechanism. This alignment strategy is used to perform accurate semantic correction of data between the data lake and the data warehouse, dynamically tracking the actual changing trends of port traffic data and ensuring data accuracy, real-time performance, and consistency between the data lake and the data warehouse.

[0123] S6: Based on alignment strategies, perform semantic correction and fusion on multi-source heterogeneous traffic data in the data lake and data warehouse switching process, and output semantically aligned lake-warehouse fused data, including:

[0124] Multi-source heterogeneous traffic data is extracted from the data lake. Based on the multi-source label mapping rules in the alignment strategy, field-level weights are assigned to the vehicle navigation data, and the first round of semantic correction is performed.

[0125] The constraint rules in the alignment strategy are invoked to perform status encoding and allowable floating range verification on the berth scheduling log data, and a second round of semantic correction is performed.

[0126] The standardized transformation protocol in the alignment strategy is applied to perform coordinate compensation calculation on real-time vehicle coordinate data, and a third round of semantic correction is executed.

[0127] By integrating the vehicle navigation data after the first round of semantic correction, the parking space scheduling log data after the second round of semantic correction, and the real-time vehicle coordinate data after the third round of semantic correction, semantically aligned lake-warehouse fusion data is generated.

[0128] Specifically, a data lake is a data platform for storing multi-source heterogeneous traffic data collected at ports. This includes vehicle navigation data, berth scheduling log data, and real-time vehicle coordinate data collected by automated guided vehicle (AGV) navigation systems, berth scheduling systems, and vehicle positioning systems. Initially, the data is stored in a standardized format. However, during actual data collection, due to the diversity of data sources and the differences in the performance of collection equipment, the accuracy and completeness of various fields can vary. To ensure accurate alignment of the data before it enters the data warehouse, various types of multi-source heterogeneous traffic data need to be extracted from the data lake using pre-defined data interfaces or data retrieval rules. For example, vehicle navigation data collected by the AGV navigation system between 7:00 AM and 7:30 AM can be extracted from the data lake. This data includes multiple fields such as vehicle number, real-time vehicle speed, real-time vehicle turning angle, and planned path node sequence. The data volume is approximately 10 records per vehicle per minute, totaling 300 records over 30 minutes. Based on the multi-source label mapping rules in the alignment strategy, field-level weights are assigned to each field in the vehicle navigation trajectory information data. The multi-source label mapping rule includes a weight coefficient for each field, which is dynamically adjusted based on semantic drift correction parameters. For example, the weight coefficient for the vehicle speed feature value has been updated to 0.3125, the weight coefficient for the steering angle feature value has been updated to 0.41, and the weight coefficient for the path planning feature value has been updated to 0.325. Based on these weight coefficients, the first round of semantic correction is performed: taking the automated guided vehicle with vehicle number "AGV-001" as an example, the navigation data recorded by the vehicle in the data lake is: real-time speed 2.5 meters per second, real-time steering angle 15 degrees due north, and the planned path node sequence containing 10 coordinate points. During semantic correction, firstly, the real-time speed feature value of 2.5 meters per second is weighted according to the vehicle speed field weight coefficient of 0.3125 in the multi-source label mapping rule; the real-time steering angle of 15 degrees is weighted according to the steering angle field weight coefficient of 0.41; simultaneously, the number of 10 path nodes is weighted according to the path planning field weight coefficient of 0.325, forming weighted feature values:

[0129] Weighted velocity characteristic value = real-time velocity characteristic value × velocity weighting coefficient = 2.5 m / s × 0.3125 = 0.78125;

[0130] The weighted steering angle characteristic value = real-time steering angle × angle weighting coefficient = 15 degrees × 0.41 = 6.15;

[0131] The weighted path planning characteristic value = number of path nodes × path weight coefficient = 10 × 0.325 = 3.25.

[0132] The above calculations yield semantically corrected vehicle navigation trajectory information data, reflecting the results of field-level weight allocation. This ensures the consistency, reliability, and accuracy of the semantic representation of vehicle navigation data during the migration process from the data lake to the data warehouse.

[0133] The berth scheduling log data includes multiple fields such as berth number, current berth occupancy status, equipment number occupying the berth, and berth occupancy duration. Based on the alignment strategy, the constraint rules have been expanded to allow a wider range of fluctuations. In the original rules, the allowed fluctuation range for berth occupancy status frequency was ±20% of the historical baseline value. After feature offset expansion, this range has been increased to ±40% of the historical baseline value. A second round of semantic correction is then performed based on the expanded allowable fluctuation range constraint rules.

[0134] Taking berth number "B-12" as an example, the occupancy status code collected in the current data window is "1", meaning it is occupied. The actual frequency of this occurrence is 80 times, the historical baseline frequency is 60 times, and the extended allowable fluctuation range is 36 to 84 times (i.e., 60 times ± 40%). Therefore, the actual frequency of 80 times is verified through constraint rules. Since 80 times is within the extended allowable fluctuation range of 36 to 84 times, the berth occupancy status feature value passes the second round of semantic correction.

[0135] The same constraint rules are also applied to the verification of device association feature values. For example, if the device with device number "CRANE-03" has a historical baseline frequency of 50 times, the extended allowable fluctuation range is 30 to 70 times, and the actual frequency in the current data window is 68 times, then the device association feature value will also pass the second round of semantic correction.

[0136] Through constraint rule verification, the berth scheduling log data in the berth status coding table achieves more accurate semantic alignment, ensuring the accuracy and consistency of berth occupancy status information during the migration process between the data lake and the data warehouse.

[0137] Real-time vehicle coordinate data includes the latitude and longitude coordinates of the vehicle's real-time location. Initially, this data was collected using the WGS84 coordinate system. To maintain consistency with the coordinate system used by the port (such as the UTM coordinate system), coordinate compensation calculations are required. The updated standardized conversion protocol includes the latest coordinate compensation parameters; for example, the latitude compensation parameter is 0.000016 degrees, and the longitude compensation parameter is 0.000024 degrees. A third round of semantic correction is performed on the real-time vehicle coordinate data: Taking the real-time vehicle coordinate data of vehicle number "TRUCK-005" as an example, the initially collected real-time coordinates are 30.22222 degrees North latitude and 120.33333 degrees East longitude. During the coordinate compensation process, coordinate adjustment calculations are performed using the latest compensation parameters in the standardized conversion protocol.

[0138] Compensated North latitude coordinates = Original North latitude coordinates + North latitude compensation parameter = 30.22222 degrees + 0.000016 degrees = 30.222236 degrees;

[0139] Compensated east longitude coordinates = original east longitude coordinates + east longitude compensation parameter = 120.33333 degrees + 0.000024 degrees = 120.333354 degrees.

[0140] Through the above coordinate compensation calculations, real-time geographic coordinate information after the third round of semantic correction is generated, resulting in higher data accuracy and ensuring accurate alignment of real-time vehicle coordinate data during the migration process between the data lake and the data warehouse.

[0141] The vehicle navigation trajectory information completed in the first round of semantic correction, the berth allocation status information completed in the second round of semantic correction, and the real-time geographic coordinate information completed in the third round of semantic correction are merged one by one to finally generate lake-warehouse fusion data for port traffic management applications. Example 2

[0142] The difference between Embodiment 2 and Embodiment 1 is that this embodiment introduces an integrated data lake warehouse management system based on port traffic.

[0143] Figure 2 A schematic diagram of a data lake warehouse integrated management system based on port traffic is provided. The data lake warehouse integrated management system based on port traffic includes:

[0144] Data acquisition module: Collects multi-source heterogeneous traffic data, unifies timestamps and standardizes formats to obtain pre-processed multi-source heterogeneous traffic data;

[0145] Semantic extraction module: Based on predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on preprocessed heterogeneous traffic data to generate a semantic feature set;

[0146] Mapping rule module: Constructs a semantic mapping rule set based on semantic feature set, and generates initial mapping rules through ontology alignment, multi-source label mapping and field caliber mapping;

[0147] Drift analysis module: Performs historical semantic drift cumulative deviation analysis on the semantic feature set and generates semantic drift correction parameters;

[0148] Alignment mechanism module: Combines initial mapping rules and semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism and generate alignment strategies;

[0149] Fusion Output Module: Based on the alignment strategy, semantic correction and fusion of multi-source heterogeneous traffic data in the data lake and data warehouse switching process are performed, and semantically aligned lake warehouse fused data is output.

[0150] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0151] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0152] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0153] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0155] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0156] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0157] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0158] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0159] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data lake warehouse integrated management method based on port traffic, characterized in that, Includes the following steps: S1: Collect multi-source heterogeneous traffic data, and unify the timestamps and standardize the format to obtain pre-processed multi-source heterogeneous traffic data; the multi-source heterogeneous traffic data includes vehicle navigation data, parking space scheduling log data and real-time vehicle coordinate data; S2: Based on the predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on the preprocessed heterogeneous traffic data to generate a semantic feature set; S3: Construct a semantic mapping rule set based on the semantic feature set, and generate initial mapping rules through ontology alignment, multi-source label mapping and field caliber mapping; S4: Perform historical semantic drift cumulative deviation analysis on the semantic feature set to generate semantic drift correction parameters; The vehicle speed feature value, steering angle feature value, and path planning feature value in the semantic feature set are divided into time slices, and the feature mean drift of historical data is calculated based on a predefined time window. A status coding distribution analysis was performed on the berth occupancy status feature value, operation timeliness feature value and equipment association feature value in the semantic feature set, and the status feature offset rate was calculated based on the historical frequency distribution. The spatial offset calculation rule is applied to the coordinate point sequence feature value and the movement trajectory feature value in the semantic feature set to calculate the spatial position drift vector based on the historical geographic coordinate reference point sequence. The mean drift of vehicle speed features, mean drift of steering angle features, mean drift of path planning features, offset rate of parking space status features, offset rate of equipment association features, and spatial position drift vector are summarized to generate semantic drift correction parameters. S5: Combine the initial mapping rules with semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism and generate an alignment strategy; Based on the mean drift of vehicle speed features, mean drift of steering angle features, and mean drift of path planning features in the semantic drift correction parameters, the weight allocation coefficients in the multi-source label mapping rules are adjusted. Based on the berth status feature offset rate and equipment association feature offset rate in the semantic drift correction parameters, the allowable floating range of the status code in the extended constraint rules is determined. Update the coordinate compensation parameters in the standardized transformation protocol based on the spatial position drift vector in the semantic drift correction parameters; Based on the multi-source label mapping rules after adjusting the weight allocation coefficients, the constraint rules after expanding the allowed floating range, and the standardized transformation protocol after updating the coordinate compensation parameters, an alignment strategy corresponding to the dynamic semantic alignment and correction mechanism is generated. S6: Based on the alignment strategy, perform semantic correction and fusion on multi-source heterogeneous traffic data in the data lake and data warehouse switching process, and output semantically aligned lake warehouse fused data.

2. The integrated data lake warehouse management method based on port traffic according to claim 1, characterized in that, S1, specifically: Vehicle navigation data is collected from the automated guided vehicle navigation system, parking space scheduling log data is collected from the parking space scheduling system, and real-time vehicle coordinate data is collected from the vehicle positioning system. Timestamp alignment is performed on vehicle navigation data, parking space scheduling log data, and real-time vehicle coordinate data respectively. Reconstruct the fields of the vehicle navigation data to generate a navigation data table; Convert the data type of the berth scheduling log data to generate a berth status code table; Perform coordinate system transformation on real-time vehicle coordinate data to generate a standard geographic coordinate sequence; Based on navigation data tables, berth status coding tables, and standard geographic coordinate sequences, preprocessed multi-source heterogeneous traffic data is generated.

3. The integrated data lake warehouse management method based on port traffic according to claim 2, characterized in that, S2, specifically: Based on the port traffic concept set in the predefined business ontology library, the semantics of the fields in the navigation data table are parsed to extract vehicle speed feature values, steering angle feature values ​​and path planning feature values; Based on the attribute relationship tree in the predefined business ontology library, the field semantics of the berth status coding table are parsed to extract berth occupancy status feature values, operation timeliness feature values ​​and equipment association feature values; The spatial location parsing rules in the predefined semantic meta-model are invoked to parse the field semantics of the standard geographic coordinate sequence and extract the feature values ​​of the coordinate point sequence and the movement trajectory feature values. Integrate all feature values ​​to generate a semantic feature set.

4. The integrated data lake warehouse management method based on port traffic according to claim 3, characterized in that, S3, specifically: Ontology alignment is performed on vehicle speed feature values, steering angle feature values, and path planning feature values ​​in the semantic feature set. The attribute definitions of vehicle trajectory concepts in the predefined business ontology library are matched and multi-source label mapping rules are generated. Field mapping is performed on the berth occupancy status feature value, operation timeliness feature value and equipment association feature value in the semantic feature set, and constraint rules between feature values ​​and data warehouse fields are established based on the attribute relationship tree of the predefined business ontology library; The spatial relationship parsing rules of the predefined semantic meta-model are invoked on the coordinate point sequence feature values ​​and movement trajectory feature values ​​in the semantic feature set to generate a standardized conversion protocol for the geographic coordinate field; An initial mapping rule set is generated based on multi-source label mapping rules, constraint rules, and standardized conversion protocols.

5. The integrated data lake warehouse management method based on port traffic according to claim 4, characterized in that, S6, specifically: Multi-source heterogeneous traffic data is extracted from the data lake. Based on the multi-source label mapping rules in the alignment strategy, field-level weights are assigned to the vehicle navigation data, and the first round of semantic correction is performed. The constraint rules in the alignment strategy are invoked to perform status encoding and allowable floating range verification on the berth scheduling log data, and a second round of semantic correction is performed. The standardized transformation protocol in the alignment strategy is applied to perform coordinate compensation calculation on real-time vehicle coordinate data, and a third round of semantic correction is executed. By integrating the vehicle navigation data after the first round of semantic correction, the parking space scheduling log data after the second round of semantic correction, and the real-time vehicle coordinate data after the third round of semantic correction, semantically aligned lake-warehouse fusion data is generated.

6. A data lake warehouse integrated management system based on port traffic, used to implement the data lake warehouse integrated management method based on port traffic as described in any one of claims 1-5, characterized in that, include: Data acquisition module: Collects multi-source heterogeneous traffic data, unifies timestamps and standardizes formats to obtain pre-processed multi-source heterogeneous traffic data; Semantic extraction module: Based on predefined business ontology and semantic meta-model, semantic feature extraction and hierarchical relationship analysis are performed on preprocessed heterogeneous traffic data to generate a semantic feature set; Mapping rule module: Constructs a semantic mapping rule set based on semantic feature set, and generates initial mapping rules through ontology alignment, multi-source label mapping and field caliber mapping; Drift analysis module: Performs historical semantic drift cumulative deviation analysis on the semantic feature set and generates semantic drift correction parameters; Alignment mechanism module: Combines initial mapping rules and semantic drift correction parameters to construct a dynamic semantic alignment and correction mechanism and generate alignment strategies; Fusion Output Module: Based on the alignment strategy, semantic correction and fusion of multi-source heterogeneous traffic data in the data lake and data warehouse switching process are performed, and semantically aligned lake warehouse fused data is output.

Images