Data management method for port hoisting equipment based on big data

Abstract

The invention relates to a data management method for port hoisting equipment based on big data, which is characterized in that a structured prefix code is added to each piece of data at an equipment data acquisition end for source coding, and the code comprises a data type, an equipment identifier and specific variable information; after the data is uploaded, the central platform automatically collects the data to a corresponding processing assembly line according to a data type prefix, carries out structuralization, standardization and vectorization processing, and carries out classified storage; furthermore, the basic data are calculated through a functional module, and intelligent data such as service life and advanced fault diagnosis are derived. According to the invention, through automatic collection and accurate positioning of coding driving data, the data query and analysis efficiency is improved from several hours to a minute level, the system computing power and bandwidth pressure are significantly reduced, and standardized and intelligent full-life-cycle management of port hoisting equipment data is realized.

Description

Technical Field

[0001] This invention relates to the field of data management technology for port crane equipment, and specifically to a data management method for port crane equipment based on big data. Background Technology

[0002] Port lifting equipment, such as quay container cranes and gantry cranes, is the core equipment of the port logistics system. Its safe, reliable, and efficient operation directly affects the overall operational efficiency and economic benefits of the port. With the development of industrial internet and big data technologies, predictive maintenance and intelligent management through the collection, analysis, and utilization of equipment operation data have become an inevitable trend in the industry.

[0003] However, current port equipment data management practices commonly suffer from the following problems, which severely restrict the extraction of data value: Data heterogeneity and lack of standardization: Data from port crane equipment comes from diverse sources, including programmable logic controllers (PLCs), various sensors, instruments, and images. These data have different formats, lack naming conventions, and have inconsistent units, making data interoperability and integration extremely difficult.

[0004] Inefficient data processing and querying: Existing systems typically store and process hundreds of millions of metrics indiscriminately. When querying specific metrics or performing complex calculations (such as calculating remaining equipment lifespan), the system must perform a global scan of massive amounts of data, resulting in query cycles lasting several hours, slow data processing, and a lack of real-time performance. This not only increases the system's computing power burden but also significantly raises operating costs.

[0005] The system architecture is rigid and unable to support big data applications: Currently, most processing methods still adopt a "single device, single server" or simple data address matching mode, failing to form a distributed, scalable big data processing architecture. Data classification is massive, processing resources are high, speed is slow, and subsequent data tracking and querying are cumbersome.

[0006] While some data warehouse platforms exist in the industry, such as the MiCRANE data warehouse specifically designed for crane lifecycle management, providing a foundation for data storage and visualization, these platforms themselves do not address the issue of source standardization before data is entered into the warehouse. If the raw data itself is disorganized and lacks effective organization and classification, even the most powerful data platform will face significant challenges in data preprocessing, real-time querying, and efficient computation, making it difficult to achieve its intended performance. Summary of the Invention

[0007] In view of the problems existing in the prior art, the purpose of this invention is to provide a data management method for port crane equipment based on big data. By introducing structured prefix coding, the data is fundamentally classified, collected and identified, so as to improve the management efficiency of port crane equipment.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A data management method for port crane equipment based on big data includes the following steps: Data acquisition and source coding steps: The edge data acquisition controller deployed on the port crane equipment reads the first type of data from the equipment and adds a prefix code to each data entry. The prefix code is in the format of [data type prefix][equipment identifier prefix][symbol identification], forming a complete encoded data packet. The data type prefix is ​​used to distinguish the type of the first type of data, the equipment identifier prefix is ​​used to identify the dock and equipment from which the data originates, and the symbol identification is used to identify the component and variable corresponding to the data. Intelligent collection, processing, and storage steps: The data packet is transmitted to the central data processing platform; the central data processing platform automatically distributes the data packet to the corresponding data processing pipeline according to the data type prefix; in the data processing pipeline, the data is sequentially processed by structuring, standardizing, and vectorizing to form standard data records; according to the data type prefix, the standard data records are stored in the corresponding first type database in the MiCRANE data warehouse; Data extraction and intelligent derivation steps: In response to the triggering of the functional module, extract the associated first type of data from the first type of database; perform calculations based on the extracted data to generate second type of data; add a prefix code to the second type of data, the format of which is also [data type prefix][device identifier prefix][symbol recognition]; store the second type of data in the corresponding second type of database in the MiCRANE data warehouse; Application invocation steps: In response to a data retrieval request from the terminal or external system, retrieve the data from the corresponding database in the MiCRANE data warehouse according to the prefix code of the requested data and return it.

[0009] In the data acquisition and source coding steps, the first type of data includes real-time reading data, status data, fault data, heartbeat data, and bypass data; wherein, the data type prefix of the real-time reading data is R_D_, the data type prefix of the status data is R_S_, the data type prefix of the fault data is R_A_, the data type prefix of the heartbeat data is R_H_, and the data type prefix of the bypass data is R_P_.

[0010] In the intelligent collection, processing and storage steps, fine-grained sharding or indexing is performed within the first type of database based on the dock code, equipment code and component code to achieve hierarchical storage.

[0011] In the data extraction and intelligent derivation steps, the functional modules include at least one of a lifespan calculation module, an online fault diagnosis module, and a periodic statistics module; wherein, the second type of data generated by the lifespan calculation module is lifespan data, and its data type prefix is ​​R_E_; the second type of data generated by the online fault diagnosis module is fault diagnosis data, and its data type prefix is ​​R_F_; the second type of data generated by the periodic statistics module includes daily average data and monthly average data, and their data type prefixes are R_C_ and R_M_, respectively.

[0012] In the intelligent collection, processing and storage steps, the structured processing refers to converting the data into a unified data model that includes device ID, component code, variable name, unit, value and timestamp.

[0013] By adopting the above scheme, the present invention stores data in layers according to type and device through source encoding and prefix-based intelligent aggregation, which reduces the scope of data query and complex calculation from scanning "the entire database" to retrieving "a specific subset of devices in a specific database", shortening the processing time from several hours to minutes or even seconds, and greatly improving real-time performance.

[0014] The prefix-based encoding system of this invention makes it very easy to add new data types, devices, or analysis modules. It only requires defining a new prefix and the corresponding processing logic, resulting in excellent system compatibility and future scalability.

[0015] This invention covers the entire data pipeline from equipment-side data acquisition, standardized processing, intelligent analysis to final application invocation, realizing standardized management and value mining of port crane equipment's full lifecycle data, and powerfully promoting the integration, informatization and intelligentization of port equipment management. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0017] This invention discloses a data management method for port crane equipment based on big data, which includes the following steps: Step 1: Data Acquisition and Source Coding.

[0018] The edge data acquisition controller deployed on the port's lifting equipment reads the first type of data from the equipment and encodes it by adding a prefix code to the front of the first type of data. The format of the prefix code is: [data type prefix][equipment identifier prefix] [symbol recognition].

[0019] The data type prefix is ​​the primary label of the data, determining its type and subsequent flow. The equipment identification prefix consists of "dock code + equipment code." This prefix indicates which crane at which dock the data originated from. The symbol identification consists of "component code + variable name," precisely answering "which parameter of which component."

[0020] The first type of data consists of raw or simply converted data directly generated by the lifting equipment. During data collection, different data type prefixes are assigned based on the data's attributes. In this embodiment, the first type of data includes the following data types: Real-time reading data: prefixed with "R_D_", which is analog data collected from data sensors.

[0021] Status data: prefixed with "R_S_", indicating the binary operating status of the device (e.g., start / stop, on / off).

[0022] Fault data: prefixed with "R_S_", which are hard fault codes or alarm bits directly generated inside the crane equipment controller.

[0023] Heartbeat data: prefixed with "R_H_", is a communication link maintenance signal sent periodically by the device or gateway.

[0024] Bypass data: Prefixed with "R_P_", data that bypasses the warning signal after a warning message is received is called bypass data.

[0025] The data collection and source coding are as follows: Edge data acquisition controllers poll the device's PLC, sensors, and instruments at a fixed frequency (e.g., 1Hz).

[0026] To improve efficiency, a change reporting mechanism is usually adopted, which means that only data whose values ​​have changed is read and reported.

[0027] For each piece of data that needs to be reported, the edge data acquisition controller generates a corresponding complete code in real time based on its data attributes (whether it is a real number or a status), device affiliation, and physical meaning.

[0028] For example, the edge data acquisition control center collects data from the following locations: Terminal: Container Terminal 1; Equipment: No. 1 shore-side crane; Component: Main hoisting mechanism brake; Data: Brake pad wear, braking status, brake malfunction.

[0029] Suppose that the edge data acquisition controller reads the following series of raw data related to the main hoisting mechanism brake within one scan cycle and encodes it immediately.

[0030] The brake opening stroke belongs to real-time reading data, and its complete code is R_D_C59101_STS001_BrkStroke.

[0031] Braking status data belongs to status data, and its complete code is R_S_C59101_STS001_BrkRelese.

[0032] Braking fault data is a type of fault data, and its complete code is R_A_C59101_STS001_BrkFault.

[0033] C59101 represents Container Terminal 1, and STS001 represents Quay Crane 1.

[0034] Step 2: Transmit the data packet with complete encoding to the central data processing platform; the central data processing platform performs intelligent collection, data processing and storage.

[0035] (1) Intelligent collection; The central data processing platform reads the data type prefix of the data packet. Based on the data type prefix, it automatically distributes the data packet to the corresponding dedicated data processing pipeline. R_D_→ Real-time reading data processing pipeline; R_S_→State data processing pipeline; R_A_→ Fault Data Processing Pipeline.

[0036] (2) Data processing; Different data types undergo the following core processing sequentially within a designated pipeline to form standardized data records: Data structuring: Transform into a unified model that includes device ID, component code, variable name, unit, value, and timestamp.

[0037] Data standardization: unify units and state values ​​(such as 0 / 1).

[0038] Data vectorization: Sort data of the same type according to timestamps, which facilitates rapid response to calculations, analysis, retrieval and other operations on different data on the same timeline.

[0039] (3) Data storage; Based on data type encoding, the processed data is stored in the corresponding first-type database of the MiCRANE data warehouse. Within the first-type database, fine-grained sharding or indexing is performed according to dock code, equipment code, and component code to achieve hierarchical storage. Target data includes real-time reading database, status database, fault database, heartbeat database, and bypass database.

[0040] Step 3: Data extraction and intelligent derivation.

[0041] The central data processing platform is equipped with functional modules, including a lifespan calculation module, an online fault diagnosis module, and a periodic statistics module.

[0042] When a functional module is triggered, it extracts relevant first-class data from the corresponding first-class database based on the prefix code and performs calculations to obtain second-class data.

[0043] Encode the second type of data: Add a prefix code to the front of the second type of data; and store it in the corresponding second type database. The format of the prefix code is the same as that described in step 1, which is: [data type prefix] [device identifier prefix] [symbol recognition].

[0044] The second type of data calculated by the lifespan calculation module is lifespan data, with the data type prefix R_E_. The second type of data calculated by the online fault diagnosis module is fault diagnosis data, with the data type prefix R_F_. The second type of data calculated by the periodic statistics module includes daily average data and monthly average data. The prefix code for daily average data is R_C_, and the prefix code for monthly average data is R_M_.

[0045] Step 4, Application Invocation: In response to a data retrieval request from the terminal or external system, the application retrieves and returns data from the corresponding database in the MiCRANE data warehouse based on the prefix code of the requested data.

[0046] A terminal or external system initiates a data request, which includes the complete prefix code of the target data. First, based on the data type prefix (e.g., R_D_, R_A_), the request is routed to the corresponding dedicated database (e.g., real-time database, fault database) in the MiCRANE data warehouse. Next, using the device identifier prefix and symbol recognition in the code, the data record is accurately located in the hierarchical index already established within the database. The system returns the query results and supports various application scenarios. For example, it can dynamically display R_D_ and R_S_ data in a real-time monitoring dashboard; retrieve R_A_, R_F_, and related historical data during fault diagnosis; query R_E_ lifespan and R_C_ / R_M_ statistical trends in health assessments; and retrieve relevant sequence data based on a time range during historical playback.

[0047] In summary, this invention, through source encoding and prefix-based intelligent aggregation, stores data hierarchically by type and device, thereby narrowing the scope of data query and complex calculation from scanning the "entire database" to retrieving a "specific subset of devices in a specific database," reducing processing time from hours to minutes or even seconds, and significantly improving real-time performance.

[0048] Distributed storage architecture distributes the load, and combined with edge computing capabilities, it reduces unnecessary raw data uploads, saves network bandwidth, greatly reduces the computing power pressure on the central server, and lowers the overall operating cost.

[0049] Maximizing Data Quality and Value: Through standardized data processing workflows (structuring, standardization, and vectorization), high-quality, uniformly formatted data assets are produced. Simultaneously, through intelligent derivation modules, raw data is transformed into high-value insights such as lifespan, health status, and advanced fault diagnosis, directly supporting precise maintenance and intelligent decision-making.

[0050] The prefix-based encoding system of this invention makes it very easy to add new data types, devices, or analysis modules. It only requires defining a new prefix and the corresponding processing logic, resulting in excellent system compatibility and future scalability.

[0051] This invention covers the entire data pipeline from equipment-side data acquisition, standardized processing, intelligent analysis to final application invocation, realizing standardized management and value mining of port crane equipment's full lifecycle data, and powerfully promoting the integration, informatization and intelligentization of port equipment management.

[0052] The above description is merely an embodiment of the present invention and does not constitute any limitation on the technical scope of the present invention. Therefore, any minor modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A data management method for port crane equipment based on big data, characterized in that, Includes the following steps: Data acquisition and source coding steps: The edge data acquisition controller deployed on the port crane equipment reads the first type of data from the equipment and adds a prefix code to each data entry. The prefix code is in the format of [data type prefix][equipment identifier prefix][symbol identification], forming a complete encoded data packet. The data type prefix is ​​used to distinguish the type of the first type of data, the equipment identifier prefix is ​​used to identify the dock and equipment from which the data originates, and the symbol identification is used to identify the component and variable corresponding to the data. Intelligent collection, processing, and storage steps: The data packet is transmitted to the central data processing platform; the central data processing platform automatically distributes the data packet to the corresponding data processing pipeline according to the data type prefix; in the data processing pipeline, the data is sequentially processed by structuring, standardizing, and vectorizing to form standard data records; according to the data type prefix, the standard data records are stored in the corresponding first type database in the MiCRANE data warehouse; Data extraction and intelligent derivation steps: In response to the triggering of the functional module, extract the associated first type of data from the first type of database; perform calculations based on the extracted data to generate second type of data; add a prefix code to the second type of data, the format of which is also [data type prefix][device identifier prefix][symbol recognition]; store the second type of data in the corresponding second type of database in the MiCRANE data warehouse; Application invocation steps: In response to a data retrieval request from the terminal or external system, retrieve the data from the corresponding database in the MiCRANE data warehouse according to the prefix code of the requested data and return it.

2. The data management method for port crane equipment based on big data according to claim 1, characterized in that, In the data acquisition and source coding steps, the first type of data includes real-time reading data, status data, fault data, heartbeat data, and bypass data; wherein, the data type prefix of the real-time reading data is R_D_, the data type prefix of the status data is R_S_, the data type prefix of the fault data is R_A_, the data type prefix of the heartbeat data is R_H_, and the data type prefix of the bypass data is R_P_.

3. The data management method for port crane equipment based on big data according to claim 1, characterized in that, In the intelligent collection, processing and storage steps, fine-grained sharding or indexing is performed within the first type of database based on the dock code, equipment code and component code to achieve hierarchical storage.

4. The data management method for port crane equipment based on big data according to claim 1, characterized in that, In the data extraction and intelligent derivation steps, the functional modules include at least one of a lifespan calculation module, an online fault diagnosis module, and a periodic statistics module; wherein, the second type of data generated by the lifespan calculation module is lifespan data, and its data type prefix is ​​R_E_; the second type of data generated by the online fault diagnosis module is fault diagnosis data, and its data type prefix is ​​R_F_; the second type of data generated by the periodic statistics module includes daily average data and monthly average data, and their data type prefixes are R_C_ and R_M_, respectively.

5. The data management method for port crane equipment based on big data according to claim 1, characterized in that, In the intelligent collection, processing and storage steps, the structured processing refers to converting the data into a unified data model that includes device ID, component code, variable name, unit, value and timestamp.

Images