Intelligent storage sharing method and system for container port data

Abstract

The invention provides an intelligent storage sharing method and system for container port data, and the method comprises the steps: collecting data from each link and a data source of a container port based on a preset strategy, and dividing the collected data into Internet of Things data from equipment facilities and application system interconnection data; determining data dimensions of the Internet of Things data and the application system interconnection data; based on the data dimension, the Internet of Things data and the application system interconnection data are stored in a time sequence database or a relational database according to the storage principle; receiving a data subscription request of a user; when the data reading request is the first request and the authority information is qualified, completely sharing the total data set of the time sequence database and the relational database to the user side; and when the data subscription request is not the first request, the data of the time sequence database and the relational database are shared to the user side in an incremental pushing mode, and through intelligent classified data storage and sharing, the intelligent level is improved, and the management efficiency of container port data is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to an intelligent storage and sharing method and system for container port data. Background Art

[0002] With the booming global trade, container ports, as key hubs for international trade, have seen a sustained growth in their business volume. Ports need to handle a large number of ship inbound and outbound operations, container loading and unloading, and cargo transportation. Under such large-scale business operations, the information exchange between various departments within the port and between the port and external related companies has become extremely complex and frequent. Shipping companies need to keep abreast of the dynamics of ships at the port, including berthing time, loading and unloading progress, etc., in order to reasonably arrange subsequent route plans. Freight forwarding companies need to understand the location and status of the containers of the goods they represent to ensure that the goods can be delivered on time. At the same time, the port's own operation and management departments, such as the dispatch center and yard management department, also need to share data to optimize resource allocation and improve the port's operational efficiency.

[0003] Today's ports are undergoing digital transformation. By comprehensively interconnecting the data generated by various equipment and facilities in the port with a variety of application systems covering production, safety, health, environment, and fire protection, and using rigorous industry regulations and forward-looking port planning as guidance, we strive to create a data access platform that meets port standards.

[0004] Therefore, how to realize the intelligent storage and sharing of a large amount of container port data has become a technical problem that technical personnel in this field need to solve urgently. Summary of the invention

[0005] The present invention provides a method and system for intelligent storage and sharing of container port data, which are used to solve the defects of low management efficiency and poor intelligent storage and sharing level of a large amount of container port data in the prior art.

[0006] In a first aspect, the present invention provides a method for intelligent storage and sharing of container port data, comprising: Based on a preset strategy, data is collected from various links and data sources of the container port, and based on the differences in the forms of the various links and data sources, the collected data is divided into IoT data from equipment and facilities and application system interconnection data; Determine the data dimensions of the IoT data and the application system interconnection data, wherein the data dimensions include a time dimension, a business dimension, a flow dimension, a structure dimension, and a function dimension; Based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, the IoT data and the application system interconnection data are stored in a time series database or a relational database respectively using storage principles; In terms of storage, determine the data stability in the time series database or relational database, and determine the backup period based on the data stability; In terms of function, the validity of data in the time series database or relational database is determined, and the retention period is determined based on the validity of the data; Receive a data subscription request from a user, where the data subscription request includes a first request and a non-first request; When the data access request is the first request and the authority information is qualified, the full data set of the time series database and the relational database is fully shared with the user end; When the data subscription request is not the first request, the data of the time series database and the relational database are shared to the user end in an incremental push mode.

[0007] According to the present invention, a method for intelligent storage and sharing of container port data is provided, wherein data is collected from various links and data sources of the container port based on a preset strategy, including: For data with high timeliness requirements, it is obtained from various links and data sources of container ports through real-time collection; For data with low timeliness requirements but huge data volume, it is acquired from various links and data sources of container ports through batch collection; For one-time data, it is obtained from various links and data sources of the container port through initial collection.

[0008] According to an intelligent storage and sharing method for container port data provided by the present invention, the collected data is divided into IoT data from equipment and facilities and application system interconnection data based on the differences in the forms of the various links and data sources, including: When the various links and data sources of the container port are ship unloaders, ship loaders and stackers / reclaimers, the corresponding data are determined to be IoT data derived from the equipment and facilities; When the various links and data sources of the container port are a generation management system, a video system, a sprinkler system, a terminal shore power system, a lighting system and an energy management system, the corresponding data is determined to be application system interconnection data.

[0009] According to a method for intelligent storage and sharing of container port data provided by the present invention, the step of determining the data dimensions of the IoT data and the application system interconnection data comprises: When the IoT data and application system interconnection data include cache data, real-time data and historical data, it is determined as a time dimension; When the IoT data and application system interconnection data include ship data, cargo data, equipment data, facility data and energy consumption data, it is determined as a business dimension; When the IoT data and application system interconnection data include write data, control data and read data, it is determined as a flow direction dimension; When the IoT data and application system interconnection data include structured data, semi-structured data and unstructured data, it is determined as a structural dimension; When the IoT data and application system interconnection data include business flow basic data, job start and stop execution data and decision data, it is determined as an action dimension.

[0010] According to an intelligent storage and sharing method for container port data provided by the present invention, the storage principle is used to store the IoT data and the application system interconnection data in a time series database or a relational database, including: Based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, identify corresponding data characteristics and usage functions, terminal business operation data and IoT terminal dynamic monitoring data; Based on the data characteristics and usage functions, classification and table design are carried out. Based on the terminal business operation data, tables are created according to business modules. Based on the dynamic monitoring data of the Internet of Things terminals, tables are created according to subject functions, so as to be stored in a time series database or a relational database respectively.

[0011] According to the intelligent storage and sharing method of container port data provided by the present invention, after storing in a time series database or a relational database respectively, it also includes: Network isolation and access control, device authentication mechanism, data transmission encryption, authority management and storage encryption security management are performed on the data in the time series database and the relational database.

[0012] According to the intelligent storage and sharing method of container port data provided by the present invention, the backup period is determined according to the stability of the data, including: real-time backup of data with an update frequency reaching a preset number of times, daily scheduled backup of general business data, and weekly scheduled backup of stable data; The retention period is determined based on the validity of the data, including: one month for short-term reference data, six months for quarterly business data, and one year for annual business data.

[0013] According to a method for intelligent storage and sharing of container port data provided by the present invention, before collecting data from various links and data sources of the container port, the method includes: Initialize the reader according to the command line parameters, establish a connection between the OPC reader client and the specified server IP and port, and create a new thread to start data processing; When the new thread is executed, the protocol and configuration information are obtained and stored in the dictionary, the request is sent to the server and the log is recorded, when there is new point information, the point mapping is updated, the point data is traversed first, the ParseNumber method is used to parse and store it in the pointdic dictionary, and then the subscription information is traversed and updated, and a list of subscription item strings is stored. The OPC DA server instance and TDSGROUP group are obtained or created, and the group is activated when the data changes. The ValuesChanged method is used to update the valid value to the dictionary, add the subscription item to the group, process the addition result and output the error message, and finally calculate the data length according to the data type.

[0014] According to an intelligent storage and sharing method for container port data provided by the present invention, the interface services supported by the data subscription request include: WebService, Web API, custom TCP / IP and message queue KAFKA / MSMQ / MQTT.

[0015] In a second aspect, the present invention provides an intelligent storage and sharing system for container port data, comprising: A collection module, for collecting data from various links and data sources of the container port based on a preset strategy, and dividing the collected data into IoT data from equipment and facilities and application system interconnection data based on differences in the forms of the various links and data sources; A storage module is used to determine the data dimensions of the IoT data and the application system interconnection data, wherein the data dimensions include a time dimension, a business dimension, a flow dimension, a structure dimension, and an action dimension; based on the time dimension, the business dimension, the flow dimension, the structure dimension, and the action dimension, the IoT data and the application system interconnection data are respectively stored in a time series database or a relational database using a storage principle; A sharing module is used to receive a user's data subscription request, wherein the data subscription request includes a first request and a non-first request; when the data subscription request is a first request and the authority information is qualified, the full data set of the time series database and the relational database are completely shared with the user end; when the data subscription request is a non-first request, the data of the time series database and the relational database are shared with the user end in an incremental push mode.

[0016] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the intelligent storage and sharing method for container port data as described above is implemented.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described intelligent storage and sharing methods for container port data.

[0018] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-mentioned methods for intelligent storage and sharing of container port data.

[0019] The intelligent storage and sharing method and system of container port data provided by the present invention collect data from various links and data sources of the container port based on a preset strategy, and divide the collected data into Internet of Things data and application system interconnection data from equipment and facilities based on the form differences of various links and data sources; determine the data dimensions of the Internet of Things data and the application system interconnection data, and the data dimensions include time dimension, business dimension, flow dimension, structure dimension and function dimension; based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, use the storage principle to store the Internet of Things data and the application system interconnection data in a time series database or a relational database respectively; receive a user's data subscription request, and the data subscription request includes a first request and a non-first request; when the data reading request is the first request and the authority information is qualified, the full data set of the time series database and the relational database is completely pushed to the user end to realize data sharing; when the data subscription request is not the first request, the data of the time series database and the relational database is pushed to the user end in an incremental push mode to realize data sharing, and through intelligent classified data storage and sharing, the intelligence level is improved while effectively improving the management efficiency of container port data. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a schematic diagram of the process of the intelligent storage and sharing method of container port data provided by this embodiment; Figure 2 is a schematic diagram of the structure of the intelligent storage and sharing system for container port data provided by this embodiment; Figure 3 It is a schematic diagram of the structure of the electronic device provided in this embodiment. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Figure 1 It is a flow chart of the intelligent storage and sharing method of container port data provided in this embodiment.

[0024] like Figure 1 As shown, the intelligent storage and sharing method for container port data provided by the embodiment of the present invention can be executed by a port data platform and mainly includes the following steps: 101. Based on the preset strategy, data is collected from various links and data sources of the container port, and based on the differences in the forms of each link and data source, the collected data is divided into IoT data from equipment and facilities and application system interconnection data.

[0025] In a specific implementation process, the port data platform collects data from various links and data sources of container ports through various sensors, equipment and systems based on preset strategies, which can meet users' dynamic understanding of logistics information of containers and transport vehicles, meet the regulatory requirements of management agencies on containers and transport vehicles, and meet the simple operation of container business processes by actual operating departments such as terminals and yards.

[0026] Due to the high sampling frequency and large amount of data collected in port container inspection, the data collection system is required to have strong data transmission and processing capabilities. Therefore, the preset strategy for data collection is mainly divided into three types: real-time collection, batch collection and initialization collection.

[0027] Real-time collection: For data with high timeliness requirements, real-time collection is used to obtain data from various links and data sources of container ports. Through instant acquisition, rapid collection, and rapid storage, it can be shared and applied at the fastest speed.

[0028] Batch collection: For data with low timeliness requirements but large data volume, batch collection is used to obtain data from various links and data sources of container ports. It will aggregate a large amount of data according to certain rules or time intervals and then perform unified collection operations. It can not only reduce the burden of the system during business peak periods, but also meet the needs of subsequent data analysis for large amounts of data. It is defined that when the timeliness requirement is greater than the preset threshold, the timeliness requirement is high, otherwise it is low, and when the data volume exceeds the data volume threshold, the data volume is large.

[0029] Initialization collection: For one-time data, it is obtained from various links and data sources of container ports through initialization collection. Usually it is when the system is first built or after a major upgrade or reconstruction. It will import all the prepared relevant data into the system at one time to ensure that the system has basic business data support during the initialization stage.

[0030] After completing the data collection, the collected data is divided into IoT data from equipment and facilities and application system interconnection data based on the differences in the forms of each link and data source, including: when the links and data sources of the container port are ship unloaders, ship loaders and stackers, the corresponding data is determined to be IoT data from equipment and facilities; when the links and data sources of the container port are generation management systems, video systems, sprinkler systems, terminal shore power systems, lighting systems and energy management systems, the corresponding data is determined to be application system interconnection data.

[0031] Among them, the ship unloader includes: monitoring the collectable data of lifting, opening and closing, trolley, car, pitching, operating table, and power supply. The ship loader includes: monitoring the collectable data of trolley, boom, rotation, power supply, and operating table. The stacker-reclaimer includes: monitoring the collectable data of trolley, pitching, rotation, power supply, and operating table.

[0032] The data collected by the generation management system include: ship schedule data, operation plan, inventory information, information bulletin, and report. The data collected by the video system include: electronic fence (warning area), safety monitoring (reflective clothing, safety helmet), and alarm data. The data collected by the sprinkler system include: sprinkler equipment name, number, type, location, status and other data. The data collected by the terminal shore power system include: shore power equipment name, status, charging amount and other information. The data collected by the lighting system include: lighting equipment location, status and other information. The data collected by the energy management system include: power meter measurement data of the power plant power supply system.

[0033] 102. Determine the data dimensions of IoT data and application system interconnection data. The data dimensions include time dimension, business dimension, flow dimension, structure dimension and function dimension.

[0034] Cache the collected data to provide efficient, core and effective data for data sharing. Classify and store the collected data in a time series database or relational database to form a complete data record. The collected data can be accurately replayed when needed.

[0035] First, since the collected data covers the port facilities and equipment, the data dimensions of IoT data and application system interconnection data are determined, including: When IoT data and application system interconnection data include cached data, real-time data and historical data, the time dimension is determined. Cache (temporary) data is used for temporary calculations; real-time data instantly reflects current port activities; result (historical) data is a record of past business activities, which can be used for analysis and trend research.

[0036] When IoT data and application system interconnected data include ship data, cargo data, equipment data, facility data and energy consumption data, they are determined as business dimensions. There are ship data, such as arrival and departure time, type, etc.; cargo data, such as type, quantity, etc.; equipment data, covering operating status, maintenance records; facility data, involving the use of terminals, yards, etc.; energy consumption data, recording energy consumption, which are of great significance to the management of their respective businesses.

[0037] When IoT data and application system interconnection data include write data, control data and read data, it is determined as the flow dimension. It is divided into write data, which is input from the outside; control data, which is used to regulate port operations; and read data, which is obtained from the storage system for use.

[0038] When IoT data and application system interconnection data include structured data, semi-structured data and unstructured data, it is determined as a structural dimension. There is structured data, which is stored in a relational database or a time series database in a table for easy query and analysis; semi-structured data has a flexible format, such as XML and JSON format; unstructured data, such as videos and images, requires special technical processing.

[0039] When IoT data and application system interconnection data include business flow basic data, operation start and stop execution data, and decision data, they are determined as action dimensions. The first is business flow basic data, such as basic information of cargo and ships, to ensure the connection of business links; the second is operation start and stop execution data, which is related to the operation; the third is decision data, which is obtained by analyzing historical, abnormal, and alarm data.

[0040] 103. Based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, the storage principle is used to store the IoT data and the application system interconnection data in the time series database or relational database respectively.

[0041] After determining the data dimension, it is necessary to store data intelligently. First, clarify the requirements for data storage. Ensure the material basis for data storage through high-performance, stable and reliable servers and storage devices. According to the actual operation needs of port business, comprehensively and carefully plan the storage capacity to ensure that the historical data accumulation and real-time business data inflow can be properly stored to avoid data loss or confusion. Ensure that the data maintains a high degree of consistency and integrity throughout the process, strictly follow the unified data specifications and logical rules, eliminate redundancy and errors, and provide strong guarantees for data application and analysis. Plan a reasonable index structure and scientific partitioning strategy, optimize the storage and query efficiency of data, accurately locate the target data, and improve the ability to respond quickly to data. Establish a regular data backup mechanism, and fully back up important data according to the predetermined backup cycle and strategy for possible hardware failures, human errors, natural disasters and other unexpected situations. Ensure that data can be quickly restored from backups when it encounters unexpected losses, minimize the risk of data loss, and ensure the continuity and stability of port business.

[0042] Then, based on the data dimensions, the storage principles are used to store the IoT data and the application system interconnection data in a time series database or a relational database, including: based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, identifying the corresponding data characteristics and usage functions, terminal business operation data and IoT terminal dynamic monitoring data.

[0043] Based on the data characteristics and usage functions, we carried out classification and table design, took the basic data elements of the port area as the basis, sorted according to the "one number and one source", and constructed basic data including six categories of ships, vehicles, containers (cargo), equipment, facilities, and personnel.

[0044] Based on the terminal business operation data, tables are created according to business modules to closely link data with business processes, facilitating query analysis and decision support.

[0045] Based on the dynamic monitoring data of the Internet of Things terminals, tables are created according to thematic functions to present the monitoring information clearly and enhance the control over the real-time status of the port area.

[0046] Establish data lineage relationships, accurately track the location changes and evolution of data over time, and form key data tracks.

[0047] Therefore, they are stored in time series databases or relational databases respectively to achieve classified storage, so as to build a complete and orderly data record archive, which is convenient for accurate tracing and review of previously collected data when needed. For example, video, audio and log data, due to their unstructured characteristics, are stored in the form of files for easy management and quick retrieval; real-time monitoring data, due to its high requirements for read and write speed and flexibility, chooses non-relational database storage; and business processing result data, due to its strong structure and correlation, is stored in a relational database to support complex query and correlation analysis operations. Data can also be divided into different storage areas based on date or data attributes. This measure can significantly reduce the amount of data involved in a single query, greatly improve query efficiency, speed up data retrieval, and enhance system responsiveness.

[0048] In today’s digital environment, data is extremely vulnerable to cyber attacks and identity theft, which may seriously damage equipment and harm user rights. Therefore, data security needs to be ensured in the process from data transmission to storage to prevent future problems.

[0049] Perform network isolation and access control on the data in the time series database and relational database: strictly limit the use of data to the local area network, isolate it from the public network, and reduce the risk of network attacks. If communication with the public network is required, set inbound and outbound rules, specify IP to open specific ports, and build firewalls to reduce the possibility of attacks and ensure data security.

[0050] Device authentication mechanism: In the security authentication phase, device authentication is adopted. When the device connects to the platform, the platform verifies the device's certificate information. Data can only be transmitted after authentication is passed to ensure access security.

[0051] Data transmission encryption: Data transmission supports the TLS handshake protocol to ensure the authenticity and integrity of collected data and control instructions, and prevent tampering and forgery. It also uses MD5, AES, DES and asymmetric encryption to encrypt transmitted data to prevent sensitive data leakage.

[0052] Permission management: The system strictly controls device and user permissions. Any operation requires authentication to prevent unauthorized access and maintain data security and system stability.

[0053] Security management of storage encryption: Use database tools to encrypt stored data, enhance data confidentiality, and ensure data security throughout its life cycle.

[0054] After completing the intelligent storage of container port data, formulate strategies based on the storage method and data function. In terms of storage, determine the data stability in the time series database or relational database, and determine the backup cycle based on the data stability, including: real-time backup of data that has been updated at a preset frequency, daily scheduled backup of general business data, and weekly scheduled backup of stable data, balancing resources and security.

[0055] In terms of function, it determines the validity of data in a time series database or relational database, and determines the retention period based on the validity of the data, including: one month for short-term reference data, six months for quarterly business data, and one year for annual business data. Data of great significance is permanently preserved. In this way, it can ensure the safe backup and recovery of different types of data, meet the needs of data retention at all stages of the business, and ensure that the enterprise's operational decisions have sufficient and appropriate data support.

[0056] 104. Receive a data subscription request from a user, where the data subscription request includes a first request and a non-first request.

[0057] The monitoring, display and statistical functions of the port data platform are of great significance. Through monitoring, the operating status of each module can be intuitively presented to help managers detect problems in a timely manner. The statistical function deeply analyzes and summarizes data, which can provide strong support for decision-making and facilitate enterprises to optimize resource allocation and business strategies.

[0058] During the interface design process, data exchange and interface calls must strictly follow a unified interface model architecture. On the one hand, this interface model must closely follow the established unified data standards and interface specification guidelines of the project, and effectively achieve the various functions specified in the interface specification; on the other hand, it must be carefully designed from multiple key dimensions, covering the rigor of data management, data integrity protection measures, interface security protection strategies, interface access efficiency optimization, overall performance stability, and system scalability expansion directions, etc., to fully shape high-quality, high-performance, and well-adaptable interface specifications.

[0059] In the business scenario of data sharing through interfaces, a one-time password authentication mechanism is adopted. By generating a one-time password, the scope of visitor permissions is accurately and strictly limited, ensuring strict identity authentication and authorization control for each data sharing, effectively protecting the rights and interests of both data owners and users.

[0060] In addition, in the communication between the system platform and the interface integration system, various encryption methods such as link encryption, network encryption or application encryption will be fully adopted. Through these encryption methods, it is possible to effectively prevent irrelevant personnel and applications from stealing key business information through network monitoring, and to ensure the confidentiality, integrity and security of business information during transmission from the source, effectively resist external threats, and ensure the efficient and stable operation of the system.

[0061] 105. When the data access request is the first request and the permission information is qualified, the full data set of the time series database and relational database will be fully shared with the user end.

[0062] After receiving the user's data subscription request, when the data access request is the first request and the permission information is qualified, the full data set of the time series database and relational database will be fully shared to the user side to ensure that the user can obtain comprehensive and complete basic data information and provide sufficient data sources for their subsequent business development or data analysis.

[0063] 106. When the data subscription request is not the first request, the data of the time series database and the relational database are shared to the user end in an incremental push mode.

[0064] When the data subscription request is not the first request, the data of the time series database and relational database are shared to the user end in incremental push mode. Only when the data actually changes will the newly added or updated data be accurately pushed to the subscribed user. This efficient push mechanism greatly reduces unnecessary data transmission, not only optimizes the utilization efficiency of network resources, but also enables users to grasp the dynamic changes of data in a timely and accurate manner, always maintain the timeliness and accuracy of their data information, and strongly support users' various decisions and business operations based on real-time data changes.

[0065] The interface services supported by data subscription requests include: WebService, Web API, custom TCP / IP, and message queues KAFKA / MSMQ / MQTT.

[0066] Furthermore, in this embodiment, the PLC and the industrial computer establish a communication channel with the help of a network cable, and a special acquisition program is run on the industrial computer. The program is responsible for obtaining point data from the PLC, parsing and processing these data according to the standard protocol, and then synchronously storing the parsed data in the time series database. After that, data docking is achieved with other servers in the local area network through a wired or wireless network cable connection, thereby completing the entire data transmission and sharing process.

[0067] Initialization includes: initializing the reader according to the command line parameters, establishing a connection between the OPC reader client and the specified server IP and port, and creating a new thread to start data processing; The loop processing includes: when executing a new thread, obtaining the protocol and configuration information and storing it in the dictionary, communicating with the server to send a request and record the log, updating the point mapping when there is new point information, traversing the point data first, parsing it with the ParseNumber method and storing it in the pointdic dictionary, then traversing and updating the subscription information, storing the subscription item string list, obtaining or creating the OPC DA server instance and TDSGROUP group, activating the group when the data changes, using the ValuesChanged method to update the valid value to the dictionary, adding the subscription item to the group, processing the addition result and outputting the error message, and finally calculating the data length according to the data type.

[0068] The port data platform is a key support for port construction. It has the characteristics of comprehensive perception, extensive interconnection and high sharing. It provides Internet of Things services for production safety supervision and equipment control, builds a basic platform for port intelligence, and realizes full-cycle digitalization. As a complete data management method, the port data platform includes three core modules. The data collection module collects data from various links and data sources of the port to ensure its completeness and accuracy; the storage management module uses a scientific architecture to properly store, organize and optimize data to ensure safety, efficiency and availability; the data sharing module uses standard interfaces and mechanisms to break data silos, realize the circulation and sharing of data between internal departments and with the outside, and tap the value of data, providing strong support for port smart operations and decision-making, and promoting the development of ports to a higher level.

[0069] In terms of methods, we deeply analyzed the characteristics of container port data, created an efficient classification, coding and indexing system, optimized storage, overcame traditional defects, improved storage efficiency and quality, and achieved rapid data location. At the same time, we formulated strict data sharing protocols and authority management strategies to promote port data circulation and collaboration, break down information barriers, provide accurate and timely data support for operational decisions, and help the intelligent development of ports while ensuring data security and privacy.

[0070] In terms of system implementation, the developed system has stable and reliable performance. It has powerful data collection capabilities and can collect various data from various port operation processes and equipment in real time and accurately, ensuring the integrity and timeliness of the data. The flexible storage architecture can not only support the long-term storage of massive data, but also quickly optimize strategies according to the frequency and importance of data use to reduce costs. The system provides a variety of convenient and fast data sharing interfaces such as WebService and Web API, which fully meet the data acquisition needs of different users and application systems, greatly facilitate the access and acquisition of data by partners inside and outside the port, and provide solid and strong technical support for the efficient operation and sustainable development of the port.

[0071] According to actual application, this method and system have improved the operational efficiency and management level of container ports. In key businesses such as ship dispatching and cargo loading and unloading, accurate decisions and collaborative operations can be made with the help of efficiently stored and shared data, which has successfully reduced the time ships spend in port, improved the utilization efficiency of the yard, and strengthened the competitiveness and economic benefits of the port, creating more economic value and social benefits.

[0072] Based on the same general inventive concept, the present invention also protects an intelligent storage and sharing system for container port data. The intelligent storage and sharing system for container port data described below and the intelligent storage and sharing method for container port data described above can refer to each other.

[0073] Figure 2 It is a schematic diagram of the structure of the intelligent storage and sharing system for container port data provided in this embodiment.

[0074] like Figure 2 As shown, this embodiment provides an intelligent storage and sharing system for container port data, including: The collection module 201 is used to collect data from various links and data sources of the container port based on a preset strategy, and divide the collected data into IoT data from equipment and facilities and application system interconnection data based on the differences in the forms of each link and data source; The storage module 202 is used to determine the data dimensions of the Internet of Things data and the application system interconnection data, which include the time dimension, business dimension, flow dimension, structure dimension and role dimension; based on the time dimension, business dimension, flow dimension, structure dimension and role dimension, the Internet of Things data and the application system interconnection data are respectively stored in a time series database or a relational database using storage principles; in terms of storage, the data stability of the time series database or the relational database is determined, and the backup period is determined according to the data stability; in terms of role, the data validity of the time series database or the relational database is determined, and the preservation period is determined according to the data validity; The sharing module 203 is used to receive the user's data subscription request, which includes the first request and the non-first request; when the data subscription request is the first request and the permission information is qualified, the full data set of the time series database and the relational database will be fully shared with the user end; when the data subscription request is not the first request, the data of the time series database and the relational database will be shared with the user end in an incremental push mode.

[0075] Figure 3 It is a schematic diagram of the structure of the electronic device provided in this embodiment.

[0076] like Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330 and a communications bus 340, wherein the processor 310, the communications interface 320 and the memory 330 communicate with each other through the communications bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the intelligent storage and sharing method of container port data, which includes: based on a preset strategy, collecting data from various links and data sources of the container port, and based on the form differences of the various links and data sources, dividing the collected data into Internet of Things data and application system interconnection data derived from equipment and facilities; determining the data dimensions of the Internet of Things data and application system interconnection data, the data dimensions including time dimension, business dimension, flow dimension, structure dimension and function dimension; based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, using the storage principle to store the Internet of Things data and application system interconnection data in a time series database respectively. or relational database; in terms of storage, determine the data stability in the time series database or relational database, and determine the backup period based on the data stability; in terms of function, determine the data validity in the time series database or relational database, and determine the retention period based on the data validity; receive the user's data subscription request, the data subscription request includes the first request and the non-first request; when the data subscription request is the first request and the authority information is qualified, the full data set of the time series database and the relational database is fully shared with the user end; when the data subscription request is not the first request, the data of the time series database and the relational database is shared with the user end in incremental push mode.

[0077] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the intelligent storage and sharing method of container port data provided by the above methods, the method comprising: based on a preset strategy, collecting data from various links and data sources of the container port, and based on the form differences of the various links and data sources, dividing the collected data into Internet of Things data and application system interconnection data derived from equipment and facilities; determining the data dimensions of the Internet of Things data and application system interconnection data, the data dimensions including time dimension, business dimension, flow dimension, structure dimension and action dimension; based on the time dimension, business dimension, flow dimension, structure dimension and action dimension , using the storage principle to store the IoT data and the application system interconnection data in a time series database or a relational database respectively; in terms of storage, determine the data stability in the time series database or the relational database, and determine the backup period based on the data stability; in terms of function, determine the data validity in the time series database or the relational database, and determine the preservation period based on the data validity; receive the user's data subscription request, the data subscription request includes the first request and the non-first request; when the data subscription request is the first request and the authority information is qualified, the full data set of the time series database and the relational database is fully shared with the user end; when the data subscription request is not the first request, the data of the time series database and the relational database is shared with the user end in an incremental push mode.

[0079] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent storage and sharing method for container port data provided by the above methods, the method comprising: based on a preset strategy, collecting data from various links and data sources of the container port, and based on the form differences of the various links and data sources, dividing the collected data into Internet of Things data and application system interconnection data derived from equipment and facilities; determining the data dimensions of the Internet of Things data and application system interconnection data, the data dimensions including time dimension, business dimension, flow dimension, structure dimension and function dimension; based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, using storage principles to respectively store the Internet of Things data and application system interconnection data; The data and application system interconnection data are stored in a time series database or a relational database; in terms of storage, the data stability in the time series database or the relational database is determined, and the backup period is determined based on the data stability; in terms of function, the data validity in the time series database or the relational database is determined, and the retention period is determined based on the data validity; the user's data subscription request is received, and the data subscription request includes the first request and the non-first request; when the data subscription request is the first request and the authority information is qualified, the full data set of the time series database and the relational database is fully shared with the user end; when the data subscription request is not the first request, the data of the time series database and the relational database are shared with the user end in incremental push mode.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent storage and sharing method for container port data, characterized in that: include: Based on a preset strategy, data is collected from various links and data sources of the container port, and based on the differences in the forms of the various links and data sources, the collected data is divided into IoT data from equipment and facilities and application system interconnection data; Determine the data dimensions of the IoT data and the application system interconnection data, wherein the data dimensions include a time dimension, a business dimension, a flow dimension, a structure dimension, and a function dimension; Based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, the IoT data and the application system interconnection data are stored in a time series database or a relational database respectively using storage principles; In terms of storage, determine the data stability in the time series database or relational database, and determine the backup period based on the data stability; In terms of function, the validity of data in the time series database or relational database is determined, and the retention period is determined based on the validity of the data; Receive a data subscription request from a user, where the data subscription request includes a first request and a non-first request; When the data access request is the first request and the authority information is qualified, the full data set of the time series database and the relational database is fully shared with the user end; When the data subscription request is not the first request, the data of the time series database and the relational database are shared to the user end in an incremental push mode.

2. The intelligent storage and sharing method for container port data according to claim 1 is characterized in that: Based on the preset strategy, data is collected from various links and data sources of the container port, including: For data with high timeliness requirements, it is obtained from various links and data sources of container ports through real-time collection; For data with low timeliness requirements but huge data volume, it is acquired from various links and data sources of container ports through batch collection; For one-time data, it is obtained from various links and data sources of the container port through initial collection.

3. The intelligent storage and sharing method for container port data according to claim 1 is characterized in that: Based on the differences in the forms of the various links and data sources, the collected data is divided into IoT data from equipment and facilities and application system interconnection data, including: When the various links and data sources of the container port are ship unloaders, ship loaders and stackers / reclaimers, the corresponding data are determined to be IoT data derived from the equipment and facilities; When the various links and data sources of the container port are a generation management system, a video system, a sprinkler system, a terminal shore power system, a lighting system and an energy management system, the corresponding data is determined to be application system interconnection data.

4. The intelligent storage and sharing method for container port data according to claim 1 is characterized in that: The determining of the data dimensions of the IoT data and the application system interconnection data includes: When the IoT data and application system interconnection data include cache data, real-time data and historical data, it is determined as a time dimension; When the IoT data and application system interconnection data include ship data, cargo data, equipment data, facility data and energy consumption data, it is determined as a business dimension; When the IoT data and application system interconnection data include write data, control data and read data, it is determined as a flow dimension; When the IoT data and application system interconnection data include structured data, semi-structured data and unstructured data, it is determined as a structural dimension; When the IoT data and application system interconnection data include business flow basic data, job start and stop execution data and decision data, it is determined as an action dimension.

5. The intelligent storage and sharing method for container port data according to claim 4 is characterized in that: The storage principle is used to store the IoT data and the application system interconnection data in a time series database or a relational database, including: Based on the time dimension, business dimension, flow dimension, structure dimension and function dimension, identify corresponding data characteristics and usage functions, terminal business operation data and IoT terminal dynamic monitoring data; Based on the data characteristics and usage functions, classification and table design are carried out. Based on the terminal business operation data, tables are created according to business modules. Based on the dynamic monitoring data of the Internet of Things terminals, tables are created according to subject functions, so as to be stored in a time series database or a relational database respectively.

6. The intelligent storage and sharing method for container port data according to claim 5 is characterized in that: After storing in the time series database or relational database respectively, the following further comprises: Network isolation and access control, device authentication mechanism, data transmission encryption, authority management and storage encryption security management are performed on the data in the time series database and the relational database.

7. The intelligent storage and sharing method for container port data according to claim 1 is characterized in that: Determining the backup cycle according to the data stability includes: backing up the data with a preset update frequency in real time, backing up the general business data on a daily basis, and backing up the stable data on a weekly basis; The retention period is determined based on the validity of the data, including: one month for short-term reference data, six months for quarterly business data, and one year for annual business data.

8. The intelligent storage and sharing method for container port data according to claim 1 is characterized in that: Before collecting data from various links and data sources of container ports, it also includes: Initialize the reader according to the command line parameters, establish a connection between the OPC reader client and the specified server IP and port, and create a new thread to start data processing; When the new thread is executed, the protocol and configuration information are obtained and stored in the dictionary, the request is sent to the server and the log is recorded, when there is new point information, the point mapping is updated, the point data is traversed first, the ParseNumber method is used to parse and store it in the pointdic dictionary, and then the subscription information is traversed and updated, and a list of subscription item strings is stored. The OPC DA server instance and TDSGROUP group are obtained or created, and the group is activated when the data changes. The ValuesChanged method is used to update the valid value to the dictionary, add the subscription item to the group, process the addition result and output the error message, and finally calculate the data length according to the data type.

9. The intelligent storage and sharing method for container port data according to any one of claims 1 to 8, characterized in that: The interface services supported by the data subscription request include: WebService, Web API, custom TCP / IP and message queue KAFKA / MSMQ / MQTT.

10. An intelligent storage and sharing system for container port data, characterized in that: include: A collection module, for collecting data from various links and data sources of the container port based on a preset strategy, and dividing the collected data into IoT data from equipment and facilities and application system interconnection data based on differences in the forms of the various links and data sources; A storage module is used to determine the data dimensions of the IoT data and the application system interconnection data, wherein the data dimensions include a time dimension, a business dimension, a flow dimension, a structure dimension, and an action dimension; based on the time dimension, the business dimension, the flow dimension, the structure dimension, and the action dimension, the IoT data and the application system interconnection data are respectively stored in a time series database or a relational database using a storage principle; In terms of storage, the stability of the data in the time series database or relational database is determined, and the backup period is determined based on the data stability; in terms of function, the validity of the data in the time series database or relational database is determined, and the preservation period is determined based on the data validity; A sharing module is used to receive a user's data subscription request, wherein the data subscription request includes a first request and a non-first request; when the data subscription request is a first request and the authority information is qualified, the full data set of the time series database and the relational database are completely shared with the user end; when the data subscription request is a non-first request, the data of the time series database and the relational database are shared with the user end in an incremental push mode.

Images