An AI knowledge base construction method and system for a smart seaport

Abstract

The application discloses an AI knowledge base construction method and system for a smart sea port, and belongs to the cross field of smart port technology and artificial intelligence. First, multi-source heterogeneous data of the sea port is collected, and semantic analysis is performed on the multi-source heterogeneous data; then, the multi-source heterogeneous data of the sea port after the semantic analysis is subjected to format conversion to obtain a static port knowledge base; secondly, real-time change information is detected through an existing monitoring system of the smart sea port, two update strategies of a trigger event and a multi-level update cycle are set; finally, the detected change information is subjected to update strategy processing to generate update data, the update data is subjected to repeated semantic analysis and format conversion, the update data in the Markdown format is written into the port knowledge base, and an AI knowledge base is finally generated. The AI knowledge base construction system comprises a port multi-modal data acquisition module, a semantic analysis module, a format conversion module and a knowledge base automatic update module. The application can break through the limitation of a traditional static port knowledge base, and realize real-time synchronization of the knowledge base and a port operation state.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of smart port technology and artificial intelligence, and relates to a method and system for constructing an AI knowledge base for smart seaports. Background Technology

[0002] The existing port knowledge base relies primarily on manual experience summaries and static knowledge storage. The manual update mechanism is outdated and cannot adapt to real-time dynamic changes in knowledge, leading to outdated information and inefficient decision-making. Simultaneously, the inconsistent data formats and interface standards among various subsystems in port operations (such as ship management, equipment monitoring, and logistics scheduling) create data silos that are difficult to integrate, resulting in low efficiency in multi-source data interaction and fusion. In the context of smart seaports, this problem is amplified dramatically: the demands of ultra-large-scale operations, multimodal transport collaboration, and bottlenecks in equipment cluster energy efficiency management render traditional static knowledge bases completely inadequate to meet the highly complex and real-time operational requirements.

[0003] Chinese invention patent CN119271827A proposes a method for constructing an AI knowledge base based on large models and RAG technology. This method builds a general AI knowledge base system that supports retrieval-enhanced generation and enables the storage, retrieval, and intelligent question answering of internal documents. However, while this patent has strong domain generality, it lacks domain adaptability and cannot handle specific entities in the port sector. In port-specific scenarios, its knowledge retrieval accuracy is insufficient, and it lacks deep integration of structured and unstructured data, thus limiting the practicality of the knowledge base to some extent. Chinese invention patent CN118585656B proposes a method for constructing a knowledge base for cybersecurity, capable of meeting the needs of rapid updates to cybersecurity knowledge. However, the knowledge base constructed by this patent has a single data source, cannot handle multi-source heterogeneous data, has a narrow coverage, and is difficult to adapt to the complex scenarios of smart seaports.

[0004] In conclusion, there is an urgent need to develop a method for constructing an AI knowledge base for smart seaports, in order to break down information barriers and achieve efficient collaboration. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an AI knowledge base construction method and system for smart seaports, which can achieve deep integration and intelligent processing of multi-source heterogeneous data of seaports, and by introducing a dynamic priority update mechanism, enables the knowledge base to be updated in real time, thereby providing accurate and efficient knowledge service support for the operation of smart seaports.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for constructing an AI knowledge base for smart seaports includes the following steps:

[0008] The first step is to collect heterogeneous data from multiple sources at the seaport. Details are as follows:

[0009] Step 1.1: Collect relevant data on smart seaports from public data sources using web crawling tools or API interfaces, and initially classify them into structured data and unstructured data.

[0010] The structured data includes port IoT equipment clusters, multimodal transport scheduling systems, automatic identification systems for ships, equipment sensor data, logistics databases, etc.; the unstructured data includes professional documents such as policies and regulations related to smart seaports, professional technical papers, and related web content.

[0011] Step 1.2: Data cleaning of the structured data. During data cleaning, missing data may be found in the original structured data, including both numerical and categorical missing data. Numerical missing data is filled using a sliding window, while categorical missing data is filled using high-frequency items from historical records. Missing data that cannot be filled triggers manual review, and outliers are calculated and removed using statistical methods or by setting thresholds.

[0012] Step 1.3 involves cleaning the unstructured data. This includes removing textual noise such as advertisements from web pages, correcting OCR recognition errors, standardizing encoding, time formats, and technical terms, automatically correcting non-port terminology, and using grammar checking tools to correct spelling errors.

[0013] Step 1.4: Merge the structured and unstructured data after data cleaning in Steps 1.2 and 1.3, delete duplicate data, and obtain multi-source heterogeneous data of the seaport.

[0014] The second step is semantic parsing. Semantic parsing is performed on the multi-source heterogeneous data of the seaport obtained in the first step. The details are as follows:

[0015] Step 2.1: The collected heterogeneous data from various sources in the seaport includes data in three formats: text, images, and tables. Generative pre-trained transformation model (GPT) or bidirectional encoder representation method (BERT) open large model are used to process the data in each format separately: semantic understanding is performed on text data; image recognition is performed on image data, followed by text recognition using an optical character recognition (OCR) engine, and the recognized text content is used as a description of the image and inserted into the corresponding position in the document; table layout analysis is performed on table data. This facilitates subsequent format conversion of the different data formats.

[0016] Step 2.2 uses the BERT-CRF (BERT-based Sequence Labeling with Conditional Random Fields) open-source model to perform domain entity recognition on the multi-source heterogeneous seaport data processed in Step 2.1, identifying port-specific entities such as "ship name," "berth number," "container number," and "dangerous goods category." Then, using Relation Extraction based on end-to-end language generation (REBEL), the relationships between entities are constructed. Steps 2.1 and 2.2 result in preliminary semantic parsing of the multi-source heterogeneous seaport data.

[0017] Step 2.3, Rule Engine Enhancement: A rule engine with format understanding and domain adaptation capabilities is employed to enhance the accuracy of the multi-source heterogeneous port data obtained after the preliminary semantic parsing in Steps 2.1 and 2.2. Format understanding refers to identifying specific document formats (such as dates, numbers, and citations) using regular expressions, addressing potential format heterogeneity issues in the multi-source heterogeneous port data after preliminary semantic parsing. Domain adaptation involves building a domain rule base by loading rule templates from documents in different domains, including academic papers, technical reports, and contracts. When processing specific document types, the corresponding domain rule set is automatically activated, thereby improving the accuracy of the multi-source heterogeneous port data after preliminary semantic parsing. Finally, the semantically parsed multi-source heterogeneous port data is obtained.

[0018] The third step involves converting the heterogeneous, multi-source port data obtained from the semantic parsing in the second step into Markdown syntax, resulting in a static port knowledge base; the details are as follows:

[0019] Step 3.1: The semantically parsed multi-source heterogeneous data of the seaport includes data in three formats: text, images, and tables. The text data includes titles and paragraphs, with titles having different levels. The different levels of titles are converted to Markdown heading syntax, preserving the level relationships; paragraphs are converted to Markdown paragraph syntax, preserving the original line breaks and indentation; a deep learning-based table recognition model is used to parse the structure of the table data, and then it is converted to Markdown syntax to solve the problem of complex tables spanning rows and columns, preserving structural information to the maximum extent; image data is saved as a local file or uploaded to the cloud, generating image links, and then the descriptions of the corresponding images obtained in Step 2.1 are inserted after the image links.

[0020] Step 3.2, Style Standardization Mapping: Visual style information, including fonts, colors, and alignment, is extracted from the original heterogeneous port data source documents collected in Step 1 using Cascading Style Sheets (CSS). This modifies the visual style information of the port's heterogeneous data processed in Step 3.1. The final result is a static port knowledge base.

[0021] The fourth step involves real-time detection of changes by integrating with the existing monitoring system of the smart seaport, and setting up two update strategies: triggered events and multi-level update cycles. The detected changes are processed by the update strategies to generate updated data. Details are as follows:

[0022] Step 4.1: For the existing monitoring system of the smart seaport, the monitoring equipment is diverse and comprehensive, capable of real-time sensing changes in information such as ships, equipment, and weather. Upon detecting a change, its trigger type is determined. The trigger types include event-triggered and periodic-triggered changes. Event-triggered changes refer to triggering events generated by the port operation system, such as ship berthing, equipment fault alarms, and extreme weather warnings. Periodic-triggered changes refer to information detected through multi-level update cycles, including hourly high-timeliness data (such as ship dynamics), daily updated routine data (such as equipment manuals), and monthly updated basic data (such as port maps).

[0023] Step 4.2: Set the trigger event update strategy and the multi-level update cycle update strategy to obtain updated data. The trigger event update strategy prioritizes allocating computing resources for event-triggered change information, calculates data priority weights for this information, and determines the processing method and order for each change information based on the calculation results. The multi-level update cycle update strategy, for event-triggered change information, includes this information in the regular task queue, smoothly allocates resources according to its preset plan, calculates data priority weights, and determines the processing method and order for each change information based on the calculation results.

[0024] Furthermore, the preset plans are the hourly inspection, daily update, and monthly update mentioned in step 4.1.

[0025] Furthermore, the data priority weight calculation refers to calculating the data priority of the change information detected in step 4.1 using a dynamic weighted priority algorithm based on multi-dimensional factors. The core of this dynamic weighted priority algorithm lies in expressing the priority weight PW as a linear combination function of multiple key influencing factors. Its calculation formula is as follows:

[0026] PW=α·T+β·O+γ·C+δ·R (1)

[0027] Wherein, α is the normalized weight coefficient of the timeliness factor, β is the normalized weight coefficient of the operational criticality factor, γ is the normalized weight coefficient of the change impact factor, and δ is the normalized weight coefficient of the request frequency factor; these four coefficients are used to adjust the relative importance of different factors in the overall priority assessment, and can be set through expert experience or machine learning optimization according to the specific port's operational characteristics; T represents the timeliness factor, used to measure the time sensitivity of data. This factor is calculated based on the validity period of the data itself or the last update time, and is usually negatively correlated with the time interval; O represents the operational criticality factor. This factor is used to measure the importance of the port operations associated with the data. Data directly related to core operations (such as quay crane status and berth plans) has a high O value, while auxiliary or supporting data (such as historical reports and annual summaries) has a lower O value. This factor is usually predefined by domain experts according to business rules. C represents the change impact factor, used to measure the magnitude of change in the updated data compared to the old version. R represents the request frequency factor, used to measure the frequency with which the data is queried or accessed by users on the terminal. Frequently accessed knowledge items indicate high demand and are therefore assigned a higher R value to ensure they remain up-to-date. Specifically:

[0028] The formula for calculating the timeliness factor T is as follows:

[0029] T=e (-λ·Δt) (2)

[0030] Where λ is the attenuation coefficient, which is set according to the data type; Δt is the interval between the current time and the last data update time or the effective deadline (in hours).

[0031] The formula for calculating the impact factor C of the change is as follows:

[0032] For numerical data, the relative rate of change is calculated using the following formula:

[0033] C = |new value - old value| / old value (3)

[0034] For text-based data, the cosine similarity Sim between the old and new texts is calculated based on word frequency statistics:

[0035] C = 1 - Sim (4)

[0036] The formula for calculating the request frequency factor R is as follows:

[0037] R=min(1,log(1+N) / log(1+N max (5)

[0038] Where N represents the number of times this data has been queried within a certain period of time; N max This represents the maximum number of data queries within that time period.

[0039] Furthermore, the processing method and processing order of each change information are determined by the calculated priority weight PW value and the preset update threshold, thereby ensuring that the update resources of the knowledge base are always given priority to the information most valuable to the real-time operation of the port, as follows: if the priority weight PW value calculated by formula (1) is ≥ the update threshold, then update data is generated; if the priority weight PW value calculated by formula (1) is < the update threshold, then it is discarded or downgraded.

[0040] The update threshold is a dynamically floating baseline, adjusted based on the following factors: the threshold is raised during peak business periods to process only the most critical information; the threshold is lowered during idle periods to process more routine information; and the threshold is significantly lowered in emergency situations to ensure that all changes trigger updates and generate new data. The specific threshold is determined based on the actual situation.

[0041] The fifth step involves repeating the semantic parsing and format conversion steps of the second and third steps on the updated data generated in the fourth step, converting it into Markdown format updated data, and writing it into the static port knowledge base obtained in the third step, ultimately generating an AI knowledge base for smart seaports.

[0042] An AI knowledge base construction system for smart seaports is disclosed. The system implements the AI ​​knowledge base construction method and includes a port multimodal data acquisition module, a semantic parsing module, a format conversion module, and a knowledge base automatic update module. Specifically:

[0043] The port multimodal data acquisition module is connected to the semantic parsing module and is used to collect multi-source heterogeneous data of the seaport in real time, corresponding to the first step in the AI ​​knowledge base construction method for smart seaports. This port multimodal data acquisition module includes a data acquisition unit, a structured data processing unit, and an unstructured data processing unit. The data acquisition unit collects multi-source heterogeneous data of the smart seaport from public data sources through web crawling tools or API interfaces; the structured data processing unit cleans the structured data; and the unstructured data processing unit cleans the unstructured data.

[0044] The semantic parsing module, connected to the format conversion module, is used to perform semantic parsing on the collected multi-source heterogeneous data from the seaport. This corresponds to the second step in the AI ​​knowledge base construction method for smart seaports, and includes a surface parsing unit, an entity recognition unit, and a rule engine enhancement unit. The surface parsing unit performs semantic understanding, image recognition, and table layout analysis on different formats of data, such as text, images, and tables, from the collected multi-source heterogeneous data from the seaport. The entity recognition unit performs domain entity recognition on the collected multi-source heterogeneous data from the seaport. The rule engine enhancement unit enhances the accuracy of semantic parsing by utilizing format rules and domain adaptation.

[0045] The format conversion module is used to convert semantically parsed multi-source heterogeneous data of the seaport into Markdown syntax, corresponding to the third step of the AI ​​knowledge base construction method for smart seaports. It includes a conversion unit and a style preservation unit. The conversion unit converts headings, paragraphs, tables, and images into Markdown syntax respectively; the style preservation unit preserves the original style of the document using CSS styles or Markdown extended syntax.

[0046] The automatic knowledge base update module is connected to the semantic parsing module and repeats semantic parsing and format conversion sequentially. This module sets update strategies and generates updated data, enabling the static port knowledge base to adapt to dynamic changes in port operations and possess continuous automatic update capabilities. This makes it an AI knowledge base for smart seaports, corresponding to the fifth step in the AI ​​knowledge base construction method for smart seaports. It includes a trigger condition detection unit, a data priority weight calculation unit, and an update unit. The trigger condition detection unit sets strategies such as trigger events and multi-level update cycles. After detecting a smart seaport update trigger condition, it determines whether the trigger type is event-triggered or periodic-triggered. The data priority weight calculation unit uses a dynamic weighted priority algorithm based on multi-dimensional factors to calculate data priority. The update unit intelligently determines the priority and processing method of each piece of data by combining preset update thresholds.

[0047] The beneficial effects of this invention are as follows:

[0048] This invention utilizes an open large model combined with OCR, entity recognition, and relation extraction technologies to achieve deep semantic analysis of structured and unstructured data, and enhances the accuracy of semantic analysis through a rule engine; it breaks down data silos through format unification; and it overcomes the limitations of traditional static port knowledge bases by setting trigger events, multi-level update cycles, and developing a dynamic weighted priority algorithm based on multi-dimensional factors, thereby achieving real-time synchronization between the knowledge base and the port's operational status. Attached Figure Description

[0049] Figure 1This is a flowchart of the AI ​​knowledge base construction method for smart seaports according to the present invention.

[0050] Figure 2 This is a flowchart illustrating the process of collecting multi-source heterogeneous data from seaports in the knowledge base construction method.

[0051] Figure 3 A flowchart for semantic parsing in the knowledge base construction method.

[0052] Figure 4 A flowchart for format conversion in the knowledge base construction method.

[0053] Figure 5 This is a flowchart illustrating the detection of change information and the setting of update strategies in the knowledge base construction method.

[0054] Figure 6 This is a schematic diagram of the AI ​​knowledge base construction system for smart seaports according to the present invention. Detailed Implementation

[0055] The technical solution of the present invention will be further explained below with reference to specific implementation examples.

[0056] A method for constructing an AI knowledge base for smart seaports includes the following implementation steps:

[0057] The first step is to collect heterogeneous data from multiple sources at the seaport. For example... Figure 2 As shown, the details are as follows:

[0058] Step 1.1: For public data sources, configure web crawler tools, set crawling rules and parsers for the target website's data structure to collect unstructured data such as relevant policies and regulations, industry reports, and web page text. For internal business systems, obtain structured data by calling the application programming interfaces (APIs) provided by each system, such as the Automatic Identification System (AIS). For database systems, after obtaining authorization, access logistics databases, equipment management databases, etc., through database connection interfaces to execute query statements to extract large amounts of historical or real-time structured data. The collected data is initially divided into structured and unstructured data.

[0059] Taking the Shanghai Yangshan Port Automated Container Terminal as an example, the structured data includes: port IoT equipment cluster data such as the location and operation status of quay cranes, AGVs, and rail-mounted gantry cranes, as well as the operating status data of lighting, gates, and monitoring equipment; AIS data such as the dynamic data of the latitude and longitude of all ships in the port and nearby waters, and static data such as basic ship data; equipment sensor data such as environmental sensor data, such as meteorological data such as wind speed, wind direction, and temperature in the port area, and marine data such as waves and seawater current speed; and logistics database data such as container numbers, dimensions, cargo manifest numbers, ship operation data, and yard storage plans.

[0060] The unstructured data includes: policies and regulations related to smart seaports, such as relevant conventions of the International Maritime Organization, basic national laws, national standards and industry norms related to ports, policy documents promoting the intelligent upgrading of Yangshan Port, and relevant policies or plans promulgated by Shanghai or the Yangtze River Delta region; and professional technical papers, such as Chinese and English papers on various research directions of smart seaports.

[0061] Step 1.2 involves data cleaning of the structured data. During data cleaning, missing data may be found in the original structured data. This missing data includes both numerical and categorical missing data. For numerical missing data, such as when processing the load weight data of Yangshan Port quay crane spreaders, if several data points are missing consecutively within a certain time period due to signal interference, a window is formed by taking five data points before and after the missing point, and the median is calculated for filling. For categorical missing data, such as the missing "ship type" field for some ships in AIS data, a query of the historical database reveals that 90% of the historical records for ships with this MMSI code are "Container-Ship," so it is filled with "Container Ship." Missing data that cannot be filled triggers manual review. For example, if some fields in sensor-uploaded data are missing, and the load weight is far beyond the reasonable range, it is identified as an outlier and removed.

[0062] Step 1.3 involves data cleaning of unstructured data. For example, when crawling a maritime news website's report on "Yangshan Port's Automation Upgrade," the advertisements on both sides of the page and the "Recommended Articles" module at the bottom are removed; when scanning a document, "Level II Response" is misidentified as "Level 11 Response," which is corrected by establishing a port OCR error mapping table; non-port terminology is automatically corrected, and grammar checking tools are used to correct spelling errors.

[0063] Step 1.4 merges the structured and unstructured data after the data cleaning processes in Steps 1.2 and 1.3, removing duplicate data. For example, news articles about "Yangshan Port throughput" collected from two different data sources. After similarity calculation, it is found that the two articles are highly similar in content, differing only in publication time and source. Therefore, the official version with more complete information is retained, and the other version is marked for deletion. Finally, multi-source heterogeneous data of the seaport is obtained.

[0064] The second step is semantic parsing. Semantic parsing is performed on the multi-source heterogeneous data of the seaport collected in the first step, such as... Figure 3 As shown. Specifically:

[0065] Step 2.1: The collected heterogeneous data from various sources in the seaport includes data in three formats: text, images, and tables. Taking a port operation report as an example, the GPT or BERT open large model is used to process the data in each format separately: semantic understanding is performed on the text data; image recognition is performed on the image data, and then an optical character recognition (OCR) engine is used for text recognition. The recognized text content is used as a description of the image and inserted into the corresponding position in the document. For example, an image of a quay crane operation process is inserted into the corresponding position in the document as "Queey crane operation process: After the ship berths, the spreader lowers to grab the container" (quay crane operation process.png); table layout analysis is performed on the table data. This facilitates subsequent format conversion of the different data formats.

[0066] Step 2.2 uses the BERT-CRF open-source model to perform domain entity recognition on the multi-source heterogeneous port data processed in Step 2.1, specifically identifying port-specific entities such as "ship name," "berth number," "container number," and "dangerous goods category," for example: "COSCO Shipping Venus," "berth A103," and "container number CMAU0012345." Then, using REBEL technology, the relationships between the "entities" identified in Step 2.2 are constructed, for example, "COSCO Shipping Venus" is docked at "berth A103." After Steps 2.1 and 2.2, the multi-source heterogeneous port data is obtained after preliminary semantic parsing.

[0067] Step 2.3 employs a rule engine with format understanding and domain adaptation capabilities to enhance the accuracy of the multi-source heterogeneous data of the seaport after preliminary semantic parsing. Format understanding refers to identifying specific document formats using regular expressions and converting them into Markdown syntax, resolving potential format heterogeneity issues in the multi-source heterogeneous data of the seaport after preliminary semantic parsing; for example, using regular expressions to match dates, "2024.4.20" is standardized to "2024-05-20" and converted to Markdown syntax. Domain adaptation involves building a domain rule base by loading rule templates from documents in different domains, including academic papers, technical reports, and contracts. When processing specific document types, the corresponding domain rule set is automatically activated, thereby improving the accuracy of the multi-source heterogeneous data of the seaport after preliminary semantic parsing; for example, a "Chapter-Summary-Conclusion" template is loaded for technical reports; and a "Party A-Party B-Terms" template is loaded for loading and unloading contracts. Finally, the semantically parsed multi-source heterogeneous data of the seaport is obtained.

[0068] The third step involves converting the heterogeneous, multi-source port data obtained from the semantic parsing in the second step into Markdown syntax, resulting in a static port knowledge base, such as... Figure 4 As shown. Specifically:

[0069] Step 3.1: The semantically parsed multi-source heterogeneous data of the seaport includes data in three formats: text, images, and tables. The text data includes titles and paragraphs, and the titles include different levels. The different levels of titles are converted to Markdown heading syntax, preserving the hierarchical relationships; for example, "1. Port Operation Specifications" → #Port Operation Specifications, "1.1 Safety Requirements" → ##Safety Requirements. Paragraphs are converted to Markdown paragraph syntax, preserving the original line breaks and indentation. A deep learning-based table recognition model is used to parse the structure of table-formatted data (e.g., Yangshan Port ship berthing schedule), and then converted to Markdown syntax to solve the problem of complex tables spanning rows and columns, preserving structural information to the maximum extent. Image-formatted data is saved as a local file or uploaded to the cloud, generating image links, and then the descriptions of the corresponding images obtained in Step 2.1 are inserted after the image links. For example, the "A certain quay crane operation process.png" mentioned in Step 2.1 is uploaded to Alibaba Cloud OSS, and finally a link with a description is generated: https: / / oss.example.com / a certain quay crane operation process.png — Quay crane operation process: After the ship berths, the spreader descends to grab the container.

[0070] Step 3.2, Style Standardization Mapping: Visual style information, including font, color, and alignment, is extracted from the original heterogeneous data source documents of the seaport collected in Step 1 using Cascading Style Sheets (CSS). The visual style information of the heterogeneous seaport data processed in Step 3.1 is then modified. Taking the red warning text in a policy document as an example, the original style of the document is restored and preserved using CSS styles:<spanstyle="color:red;"> Open flame operations are strictly prohibited.

[0071] The data obtained after the third step forms a static port knowledge base.

[0072] The fourth step involves real-time detection of changes by integrating with the existing monitoring system of the smart seaport, and setting up two update strategies: triggered events and multi-level update cycles. The detected changes are processed by the update strategies to generate updated data, such as... Figure 5 As shown. Specifically:

[0073] Step 4.1: For the existing monitoring system of the smart seaport, the monitoring equipment is diverse and comprehensive, capable of real-time sensing changes in information such as ships, equipment, and weather. Upon detecting a change, its trigger type is determined. The trigger types include event-triggered and periodic-triggered information. Event-triggered changes refer to triggering events generated by the port operation system, such as a real-time message from the AIS system stating that "a certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule." Periodic-triggered changes refer to information detected through setting multi-level update cycles, including hourly high-timeliness data (such as ship dynamics), daily updated routine data (such as equipment manuals), and monthly updated basic data (such as port maps), such as a "monthly updated general port introduction article."

[0074] Step 4.2 sets up two update strategies: trigger event and multi-level update cycle, to obtain updated data. The trigger event update strategy prioritizes allocating computing resources to calculate the data priority weight for event-triggered change information, and determines the processing method and order for each change based on the calculation results. The multi-level update cycle update strategy, also for event-triggered change information, includes it in a regular task queue, smoothly allocates resources to calculate the data priority weight according to a preset plan, and determines the processing method and order for each change based on the calculation results. The preset plans are hourly inspection, daily update, and monthly update, as mentioned in Step 4.1.

[0075] In this embodiment, the data priority weight calculation refers to calculating the data priority of the change information detected in step 4.1 using a dynamic weighted priority algorithm based on multi-dimensional factors. The core of this dynamic weighted priority algorithm lies in expressing the priority weight PW value as a linear combination function of multiple key influencing factors. Its calculation formula is as follows:

[0076] PW=α·T+β·O+γ·C+δ·R (1)

[0077] Wherein, α is the normalized weight coefficient of the timeliness factor, β is the normalized weight coefficient of the operational criticality factor, γ is the normalized weight coefficient of the change impact factor, and δ is the normalized weight coefficient of the request frequency factor; these four coefficients are used to adjust the relative importance of different factors in the overall priority assessment, and can be set through expert experience or machine learning optimization according to the specific port's operational characteristics; T represents the timeliness factor, used to measure the time sensitivity of the data. This factor is calculated based on the validity period of the data itself or the last update time, and is usually negatively correlated with the time interval; O represents the operational criticality factor, used to measure the importance of the port operation links associated with the data; data directly related to core operations (such as quay crane status, berth plans) have high O values. Auxiliary or supporting data (such as historical reports and annual summaries) have lower O values. This factor is usually predefined by domain experts according to business rules. For example, a real-time message from the AIS system mentioned in step 4.1, "A certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule," is preset to O=1.0 by experts. Auxiliary or supporting data, such as a "monthly updated general port introduction article," has a lower O value, which is preset to O=0.1 by experts. C represents the change impact factor, used to measure the magnitude of change in the updated data content compared to the old version. R represents the request frequency factor, used to measure the popularity of the data being queried or accessed by users on the terminal. Frequently accessed knowledge items indicate high demand, therefore a higher R value is assigned to ensure they remain up-to-date. Specifically:

[0078] The formula for calculating the timeliness factor T is as follows:

[0079] T=e (-λ·Δt) (2)

[0080] Where λ is the attenuation coefficient, which can be set according to the data type; Δt is the interval between the current time and the last update time or effective deadline of the data (in hours).

[0081] For example, for a real-time message from the AIS system mentioned in step 4.1 stating that "a certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule," let λ=0.5 and Δt=2h; for a "monthly updated general port introduction article," let λ=0.05 and Δt=1 / mout=720h. Then the T values ​​are 0.37 and 2.32E-16, respectively.

[0082] The formula for calculating the impact factor C of the change is as follows:

[0083] For numerical data, the relative rate of change is calculated using the following formula:

[0084] C = |new value - old value| / old value (3)

[0085] For example, a real-time message from the AIS system stating that "a certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule" (assuming the original plan was to wait 10 hours), C=0.2;

[0086] For text-based data, the cosine similarity Sim between the old and new texts is calculated based on word frequency statistics:

[0087] C = 1 - Sim (4)

[0088] For example, a "monthly updated general port introduction article" has a high text similarity, with Sim=0.9 and C=0.1.

[0089] The formula for calculating the request frequency factor R is as follows:

[0090] R=min(1,log(1+N) / log(1+N max (5)

[0091] Where N represents the number of times this data has been queried within a certain period of time; N max This represents the maximum number of data queries within that time period.

[0092] For example, a real-time message from the AIS system stating that "a certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule" will be frequently queried by the scheduling and operations departments, with N=100, Nmax=500, and R≈0.8; while a "monthly updated general port introduction article" will be queried by almost no business, with N=1 and R≈0.1.

[0093] In this embodiment, the processing method and order of each changed information are determined by combining the calculated priority weight PW value and the preset update threshold, so as to ensure that the knowledge base update resources are always preferentially allocated to the information most valuable to the real-time operation of the port, as follows: if the priority weight PW value calculated by formula (1) is less than the update threshold, it is discarded or downgraded; if the priority weight PW value calculated by formula (1) is greater than or equal to the update threshold, updated data is generated. The update threshold is a dynamically floating baseline that can be dynamically adjusted according to the following factors: the threshold is raised during peak business periods to process only the most critical information; the threshold is lowered during idle periods to process more routine information; and the threshold is significantly reduced in emergency situations to ensure that all changed information can trigger updates and generate updated data. For example, if the threshold is set to 0.4, and the normalized weight coefficients of each factor are tentatively set to α=0.4, β=0.3, γ=0.2, δ=0.1, then a real-time message from the AIS system stating "A certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule" has a PW=0.57>0.4 and will immediately generate updated data; a message stating "A general port introduction article updated monthly" has a PW=0.03<0.4 and will be downgraded and processed when there are no other update tasks or discarded directly.

[0094] The fifth step involves repeating the semantic parsing and format conversion steps of the second and third steps on the updated data generated in the fourth step, converting it into Markdown format updated data, and writing it into the static port knowledge base obtained in the third step, ultimately generating an AI knowledge base for smart seaports.

[0095] An AI knowledge base construction system for smart seaports is proposed. This system implements an AI knowledge base construction method, including a port multimodal data acquisition module, a semantic parsing module, a format conversion module, and a knowledge base automatic update module. Figure 6 As shown, specifically:

[0096] The port multimodal data acquisition module is connected to the semantic parsing module and is used to collect multi-source heterogeneous data of the seaport in real time, corresponding to the first step in the AI ​​knowledge base construction method for smart seaports. This port multimodal data acquisition module includes a data acquisition unit, a structured data processing unit, and an unstructured data processing unit. The data acquisition unit collects multi-source heterogeneous data of the smart seaport from public data sources through web crawling tools or API interfaces; the structured data processing unit cleans the structured data; and the unstructured data processing unit cleans the unstructured data.

[0097] Taking the Shanghai Yangshan Port Automated Container Terminal as an example, this module outputs data such as the location and operation status of quay cranes, AGVs, and rail-mounted gantry cranes, as well as the operating status data of lighting, gates, and monitoring equipment. It also outputs dynamic data such as the latitude and longitude positions of all ships in and around the port, and static data such as basic ship data. Structured data includes container numbers, dimensions, cargo manifest numbers, ship operation data, and yard storage plans. Unstructured data includes relevant policies and regulations for smart seaports, such as conventions of the International Maritime Organization, basic national laws, national standards and industry norms related to ports, policy documents promoting the intelligent upgrading of Yangshan Port, relevant policies or plans issued by Shanghai or the Yangtze River Delta region, and professional technical papers, such as Chinese and English papers on various research directions related to smart seaports.

[0098] The semantic parsing module, connected to the format conversion module, is used to perform semantic parsing on the collected multi-source heterogeneous data from the seaport. This corresponds to the second step in the AI ​​knowledge base construction method for smart seaports, and includes a surface parsing unit, an entity recognition unit, and a rule engine enhancement unit. The surface parsing unit performs semantic understanding, image recognition, and table layout analysis on different formats of data, such as text, images, and tables, from the collected multi-source heterogeneous data from the seaport. The entity recognition unit performs domain entity recognition on the collected multi-source heterogeneous data from the seaport. The rule engine enhancement unit enhances the accuracy of semantic parsing by utilizing format rules and domain adaptation.

[0099] Based on the above example, all the data output by the port multimodal data acquisition module is input into this module, and after semantic parsing, it is input into the format conversion module.

[0100] The format conversion module is used to convert semantically parsed multi-source heterogeneous data of the seaport into Markdown syntax, corresponding to the third step of the AI ​​knowledge base construction method for smart seaports. It includes a conversion unit and a style preservation unit. The conversion unit converts headings, paragraphs, tables, and images into Markdown syntax respectively; the style preservation unit preserves the original style of the document using CSS styles or Markdown extended syntax.

[0101] Based on the above example, all the data output by the semantic parsing module is input into this module, and after being converted into Markdown syntax, all the data forms a static port knowledge base.

[0102] The automatic knowledge base update module is connected to the semantic parsing module and repeats semantic parsing and format conversion sequentially. This module sets update strategies and generates updated data, enabling the static port knowledge base to adapt to dynamic changes in port operations and possess continuous automatic update capabilities. This makes it an AI knowledge base for smart seaports, corresponding to the fifth step in the AI ​​knowledge base construction method for smart seaports. It includes a trigger condition detection unit, a data priority weight calculation unit, and an update unit. The trigger condition detection unit sets strategies such as trigger events and multi-level update cycles. After detecting a smart seaport update trigger condition, it determines whether the trigger type is event-triggered or periodic-triggered. The data priority weight calculation unit uses a dynamic weighted priority algorithm based on multi-dimensional factors to calculate data priority. The update unit intelligently determines the priority and processing method of each piece of data by combining preset update thresholds.

[0103] Based on the example of the fifth step of the AI ​​knowledge base construction method for smart seaports, this module outputs an updated data: a real-time message from the AIS system that "a certain ultra-large container ship is expected to arrive at the port 2 hours ahead of schedule" (PW=0.57>0.4, updated data is generated immediately).

[0104] The above embodiments are merely illustrative of the implementation methods of the present invention, but should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the protection scope of the present invention.

Claims

1. A method for constructing an AI knowledge base for smart seaports, characterized in that, The method for constructing the AI ​​knowledge base includes the following steps: The first step is to collect heterogeneous data from multiple sources in the seaport; The second step is to perform semantic analysis on the multi-source heterogeneous data of the seaport obtained in the first step. The third step is to convert the heterogeneous port data from multiple sources after semantic parsing in the second step into Markdown syntax to obtain a static port knowledge base. The fourth step involves real-time detection of changes by linking with the existing monitoring system of the smart seaport, and setting two update strategies: trigger events and multi-level update cycles. The detected changes are then processed by the update strategies to generate updated data. The fourth step is specifically as follows: Step 4.1: For the existing monitoring system of the smart seaport, when a change is detected, determine its trigger type; the trigger type includes event triggering and periodic triggering. Step 4.2: Set the trigger event update strategy and multi-level update cycle update strategy to obtain the updated data; The trigger event update strategy prioritizes allocating computing resources for change information triggered by events, calculates data priority weights for the information, and determines the processing method and order for each change information based on the calculation results. The multi-level update cycle update strategy includes change information triggered by events in the regular task queue, allocates resources smoothly according to its preset plan, calculates data priority weights, and determines the processing method and order for each change information based on the calculation results. The preset plan refers to the hourly inspection, daily update, and monthly update mentioned in step 4.1; The data priority weight calculation refers to the calculation of the data priority of the change information detected in step 4.1 using a dynamic weighted priority algorithm based on multi-dimensional factors. The calculation formula is as follows: PW=α·T+β·O+γ·C+δ·R(1); Where α is the normalized weight coefficient of the timeliness factor, β is the normalized weight coefficient of the operational criticality factor, γ is the normalized weight coefficient of the change impact factor, and δ is the normalized weight coefficient of the request frequency factor; T represents the timeliness factor; O represents the operational criticality factor; C represents the change impact factor; and R represents the request frequency factor. The processing method and processing order of each change information are determined by the calculated priority weight PW value and the preset update threshold: if the priority weight PW value calculated by formula (1) is greater than or equal to the update threshold, then update data is generated; if the priority weight PW value calculated by formula (1) is less than the update threshold, then it is discarded or downgraded; the update threshold is a dynamically floating baseline that is dynamically adjusted according to the actual situation. The fifth step involves repeating the semantic parsing and format conversion steps of the second and third steps on the updated data generated in the fourth step, converting it into Markdown format updated data, and writing it into the static port knowledge base obtained in the third step, ultimately generating an AI knowledge base for smart seaports.

2. The method for constructing an AI knowledge base for smart seaports according to claim 1, characterized in that, The first step is specifically as follows: Step 1.1: Collect relevant data on smart seaports from public data sources and initially divide them into structured data and unstructured data; The structured data includes port IoT equipment clusters, multimodal transport scheduling systems, automatic identification systems for ships, equipment sensor data, and logistics databases; the unstructured data includes professional documents such as smart seaport-related policies and regulations, professional technical papers, and related web page content. Step 1.2: Perform data cleaning on the structured data; during the data cleaning process, there may be missing data, including missing numerical data and missing categorical data. For missing numerical data, a sliding window is used to fill the missing data; for missing categorical data, high-frequency items are filled using historical records; outliers are calculated and removed. Step 1.3: Clean the unstructured data; Step 1.4: Merge the structured and unstructured data after data cleaning in Steps 1.2 and 1.3, delete duplicate data, and obtain multi-source heterogeneous data of the seaport.

3. The method for constructing an AI knowledge base for smart seaports according to claim 2, characterized in that, The second step is specifically as follows: Step 2.1: The collected heterogeneous data from multiple sources in the seaport includes data in three formats: text, image, and table. Generative pre-trained transformation model (GPT) or bidirectional encoder representation method (BERT) open large model are used to process the data in each format separately: semantic understanding is performed on text data; image recognition is performed on image data, and then an optical character recognition (OCR) engine is used for text recognition, with the recognized text content used as a description of the image and inserted into the corresponding position in the document; table layout analysis is performed on table data to facilitate subsequent format conversion of different data formats. Step 2.2: Use the open-source sequence labeling model based on BERT and conditional random fields to perform domain entity recognition on the multi-source heterogeneous data of the seaport processed in Step 2.1; use the relation extraction technology based on end-to-end language generation to construct the association relationship between entities; after Steps 2.1 and 2.2, the multi-source heterogeneous data of the seaport after preliminary semantic parsing is obtained; Step 2.3, Rule Engine Enhancement: A rule engine with format understanding and domain adaptation capabilities is used to process the data to obtain semantically parsed multi-source heterogeneous data of the seaport.

4. The method for constructing an AI knowledge base for smart seaports according to claim 3, characterized in that, In step 2.3: the format understanding means that the document format is identified by regular expressions to solve the format heterogeneity problem of multi-source heterogeneous data of the seaport after preliminary semantic parsing; the domain adaptation means that a domain rule library is constructed by loading rule templates of documents from different domains to process documents of corresponding types.

5. The method for constructing an AI knowledge base for smart seaports according to claim 3, characterized in that, The third step is specifically as follows: Step 3.1: The semantically parsed multi-source heterogeneous data of the seaport includes data in text format, image format, and table format; The text-formatted data includes headings and paragraphs. The headings are divided into different levels. The headings of different levels are converted into Markdown heading syntax while preserving the level relationship. The paragraphs are converted into Markdown paragraph syntax while preserving the original line breaks and indentation. A deep learning-based table recognition model is used to parse the structure of the table-formatted data and then convert it into Markdown syntax. The image-formatted data is saved as a local file or uploaded to the cloud to generate an image link. The description of the image obtained in step 2.1 is then inserted after the image link. Step 3.2, Style Standardization Mapping: Use Cascading Style Sheets (CSS) to extract the visual style information, including font, color, and alignment, from the multi-source heterogeneous data source documents of the seaport collected in Step 1; modify the visual style information of the multi-source heterogeneous data of the seaport after processing in Step 3.1; finally, a static port knowledge base is obtained.

6. The method for constructing an AI knowledge base for smart seaports according to claim 1, characterized in that, In step 4.1, the event-triggered change information refers to the triggering events generated by the port operation system; the period-triggered change information refers to the information detected by setting multiple update cycles, including high-timeliness data from hourly inspections, routine data updated daily, and basic data updated monthly.

7. The method for constructing an AI knowledge base for smart seaports according to claim 1, characterized in that, In step 4.1, in formula (1): The formula for calculating the timeliness factor T is as follows: T=e (-λ·Δt) (2); Where λ is the attenuation coefficient; Δt is the interval between the current time and the last data update time or the effective deadline. The formula for calculating the impact factor C of the change is as follows: For numerical data, the relative rate of change is calculated using the following formula: C = |new value - old value| / old value (3); For text-based data, the cosine similarity Sim between the old and new texts is calculated based on word frequency statistics: C = 1 - Sim (4); The formula for calculating the request frequency factor R is as follows: R=min(1,log(1+N) / log(1+N max ))(5); Where N is the number of times this data has been queried within a certain period of time; N max This represents the maximum number of data queries within that time period.

8. An AI knowledge base construction system for smart seaports, characterized in that, The AI ​​knowledge base construction method according to any one of claims 1-7 is implemented through the AI ​​knowledge base construction system. The AI ​​knowledge base construction system includes a port multimodal data acquisition module, a semantic parsing module, a format conversion module, and a knowledge base automatic update module. Specifically: The port multimodal data acquisition module is connected to the semantic parsing module and is used to collect multi-source heterogeneous data of the seaport in real time; it includes a data acquisition unit, a structured data processing unit, and an unstructured data processing unit; the data acquisition unit is used to collect multi-source heterogeneous data of the smart seaport; the structured data processing unit performs data cleaning on the structured data; and the unstructured data processing unit performs data cleaning on the unstructured data. The semantic parsing module is connected to the format conversion module and is used to perform semantic parsing on the collected multi-source heterogeneous data of the seaport. It includes a surface parsing unit, an entity recognition unit, and a rule engine enhancement unit. The surface parsing unit performs semantic understanding, image recognition, and table layout analysis on the data in the collected multi-source heterogeneous data of the seaport. The entity recognition unit performs domain entity recognition on the collected multi-source heterogeneous data of the seaport. The rule engine enhancement unit enhances the accuracy of semantic parsing by using format rules and domain adaptation. The format conversion module is used to convert semantically parsed multi-source heterogeneous data of the seaport into Markdown syntax, and includes a conversion unit and a style preservation unit. The knowledge base automatic update module is connected to the semantic parsing module and repeats semantic parsing and format conversion in sequence; the knowledge base automatic update module is used to set update strategies and generate update data; it includes a trigger condition detection unit, a data priority weight calculation unit, and an update unit; The trigger condition detection unit is used to set the update strategy. After detecting the smart port update trigger condition, it determines whether the trigger type is event-triggered or periodic-triggered. The data priority weight calculation unit is used to calculate the data priority. The update unit is used to intelligently determine the priority and processing method of each piece of data by combining the preset update threshold.

Images