A smart port crane operation monitoring method and system

Abstract

The application discloses a kind of wisdom port crane operation monitoring method and system, it is related to crane system technical field, system includes data acquisition unit, edge computing node, cloud processor and early warning monitoring unit, through multi-source data depth fusion and efficient management, establish dynamic twinborn data pool architecture, through multi-modal coding realizes the space-time alignment of cross data type, improves feature fusion accuracy;Through intelligent analysis dynamic twinborn data pool and real-time optimization to crane operation path, to dynamically adjust crane operation path trajectory, improve work efficiency;Through digital twin model and integrate multi-modal physical engine, assess crane operation physical state deviation, and real-time correction operation plan, improve operation accuracy;Through depth analysis video stream and carry out virtual-actual contrast, monitoring model predicts video deviation, to improve simulation accuracy;Through establishing closed-loop early warning mechanism, realize whole-process digital management and control, promote port intelligent upgrading.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of crane systems, in particular to a smart port crane operation monitoring method and system. BACKGROUND

[0002] ‌The technical means and system functions of smart port crane operation monitoring include: through digital twin technology, Internet of Things technology, big data and artificial intelligence technology, real-time data of the crane is collected and analyzed to realize health state monitoring and operation standardization of the equipment; through sensors, various operating parameters and state information of the crane are collected to monitor the operation of the equipment in real time; through cameras and high-resolution monitors, the operation process of the crane is monitored in real time to ensure safe operation and good equipment state.

[0003] However, the current crane operation monitoring technology has the problems of data island fragmentation, insufficient real-time decision-making, and difficulty in multi-dimensional comprehensive risk early warning monitoring; due to independent storage of multi-source data such as sensors, video streams, and equipment logs, there is a lack of unified data center support, resulting in information fragmentation, inability to process high-concurrency data streams in real time, difficulty in adapting to dynamic operation scenarios and complex mechanical interactions, resulting in insufficient precision of digital twin video streams, low virtual-real comparison reliability, slow response to sudden changes, and lag in real-time decision-making processing, and lack of multi-dimensional comprehensive risk assessment, resulting in insufficient port crane operation efficiency and safety.

[0004] In view of the above technical defects, a solution is proposed. SUMMARY

[0005] The purpose of the present application is to solve the problems of data island fragmentation, insufficient real-time decision-making, and difficulty in multi-dimensional comprehensive risk early warning monitoring in the existing crane operation monitoring technology.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] A smart port crane operation monitoring method, comprising the following steps:

[0008] S1, real-time collection of multi-source heterogeneous data of crane operation and construction of dynamic twin data pool: through the data center of the port digital management and control platform, real-time collection of multi-source heterogeneous data of crane operation;

[0009] S2, intelligent analysis of dynamic twin data pool and real-time optimization of crane operation path: through multi-source heterogeneous data analysis of port production operation deviation, efficiency index score of crane operation is obtained, and crane dynamic adjustment strategy is output;

[0010] S3, build a digital twin model and embed a multi-modal physical engine: by synchronously simulating the crane operation mechanics interaction process in the real environment, real-time detection of the physical parameters of the crane grab, and analysis of the physical assessment report of the crane operation, the operation plan is corrected in real time and the warning instruction is pushed to the surrounding system;

[0011] S4, deep analysis of video stream and virtual-real comparison: through the monitoring video stream of the port crane operation and the simulation video stream of the digital twin model, the key action features of the crane are extracted and compared in time and space alignment, and the video deviation score of the crane operation is generated;

[0012] S5, establish a hierarchical warning mechanism for abnormal automatic processing: through the threshold comparison of the efficiency index score, the physical assessment report and the deviation risk score of the crane operation, the device warning log is output, and the abnormal event traceability report of the port crane is automatically generated for closed-loop processing.

[0013] Further, the multi-source heterogeneous data includes sensor data, device log data and video stream data;

[0014] Through the space-time alignment algorithm, the physical coordinates of the crane, container, transport vehicle and port infrastructure in the real environment are uniformly mapped to the world coordinate system, and a high-precision dynamic digital twin model is generated;

[0015] Through the wavelet transform model, the original data is cleaned and denoised, missing values are filled and standardized, and a sliding window is set for dynamic adaptive standardization, so as to unify the time stamp and space coordinate system, the specific process is as follows:

[0016] Mark the current time as t; Mark the sliding window length as W, the sliding window length is the number of data points contained in the window; Mark the index variable of the time as i, which is used to traverse all data points in the sliding window; Mark the original data value at the i-th time as ; Thus, the local mean of the sliding window data corresponding to time t is obtained ;

[0017] Further, the local standard deviation of the sliding window data corresponding to time t is obtained ;

[0018] Through the local mean and local standard deviation of the sliding window data corresponding to time t, the original data value at time t is standardized, and the standardized data value is marked as ;

[0019] Through distributed storage technology, the data types are divided into real-time stream data layer, historical data layer and metadata layer, and a dynamic twin data pool architecture is constructed; ​​

[0020] By defining the timestamp synchronization function T(ts, tv) with the historical time nodes of the device log data, the sensor, video stream and log are multi-source data spatio-temporal alignment.

[0021] Further, the specific process of S2 is as follows:

[0022] By extracting the indicators from the device log data, including the task completion time of the crane, the empty running rate, the unit cargo energy consumption, the emergency stop frequency and the load overrun rate; the number of indicators of the device log data is marked as n0;

[0023] The reference value of any indicator p of the device log data is marked as Dp0, and the actual value of the indicator p at time t is marked as Dpt. The deviation amplitude value of the indicator p is obtained by the difference between the reference value and the actual value of the indicator p ; the deviation amplitude values of the n0 indicators are measured and the maximum-minimum value is normalized to obtain the standardized value Sp of the indicator p; and the information entropy Hp of the indicator p is calculated to obtain the weight coefficient of the indicator p.

[0024] The deviation amplitude value of the indicator p and the weight coefficient are weighted and fused to calculate the efficiency indicator score SC1 of the crane operation.

[0025] Further, the specific process of S3 is as follows:

[0026] The physical parameters include crane grabber acceleration, load stress, container displacement deviation and vehicle positioning error, and the number of indicators of the physical parameters is marked as n1;

[0027] The reference value of any indicator r of the physical parameters is marked as Dr0, and the actual value of the indicator r at time t is marked as Drt. The deviation amplitude value of the indicator r is obtained by the difference between the reference value and the actual value of the indicator r , and the standardized value Sr, information entropy Hr and weight coefficient of the indicator r are further obtained.

[0028] The deviation amplitude value of the indicator r and the weight coefficient are weighted and fused to calculate the physical action score SC2 of the crane operation.

[0029] Further, the specific process of S4 is as follows:

[0030] By frame-level analysis of the monitoring video stream, the key action features of the crane are extracted and compared with the spatio-temporal alignment of the digital twin simulation video stream;

[0031] The prediction category of the digital twin model video stream on the key action feature of the crane is divided into n2, and the prediction category probability distribution of the digital twin model is ; The real category label of the monitoring video stream of the crane operation is encoded as ;

[0032] The prediction box of the digital twin simulation video stream is marked as B: , wherein the center point coordinates of the prediction box are (b1, b2), the length is b3, and the width is b4;

[0033] The real box of the monitoring video stream of the crane operation is marked as C: , wherein the center point coordinates of the real box are (c1, c2), the length is c3, and the width is c4;

[0034] The cross-entropy of the virtual-real comparison of the crane action feature is calculated by spatiotemporal alignment and complete intersection union CIoU, and the target detection loss Ld is obtained;

[0035] The position coordinate sequence of the target point in consecutive m frames is marked as E, so as to calculate the change difference of adjacent frames. The displacement of any pair of adjacent frames in the x direction and the y direction is marked as , , so as to obtain the tracking consistency loss Lt;

[0036] The velocity and acceleration of any pair of adjacent frames in the x direction and the y direction are calculated by the position coordinates of the target point in consecutive m frames and the time interval of adjacent frames, so as to obtain the kinematics rationality loss Lm;

[0037] The target detection loss Ld, the tracking consistency loss Lt and the kinematics rationality loss Lm are weighted and fused, so as to generate the video deviation score SC3 of the crane operation.

[0038] Further, the specific process of S5 is as follows:

[0039] A hierarchical warning mechanism is established, and the threshold value of the efficiency index score of the crane operation, the physical assessment report and the deviation risk score is compared, and the device warning log is output;

[0040] The threshold value Q1 of the efficiency index score SC1 of the crane operation is set, and when the efficiency index score SC1 is lower than the threshold value Q1, the crane path trajectory is re-planned, the crane operation path corresponding to the maximum value of the efficiency index score SC1 is taken as the optimal path strategy, and it is marked as the crane dynamic adjustment strategy and output;

[0041] A threshold Q2 of a physical action score SC2 of the crane operation is set, the physical action state of the crane operation is evaluated through threshold comparison, and a corresponding early warning instruction is generated, so as to prompt the surrounding system to perform corresponding crane operation management;

[0042] A threshold Q3 of a video deviation score SC3 of the crane operation is set, the virtual-real deviation degree of the crane is evaluated through threshold comparison, and a corresponding early warning prompt is given to the background technical personnel;

[0043] Further, an abnormal event traceability report of the port crane is automatically generated: the historical operation data and the equipment warning log of the port data center are integrated into the abnormal event traceability report, and are pushed to the preventive equipment collaborative management platform for closed-loop processing.

[0044] A smart port crane operation monitoring system, comprising a data acquisition unit, an edge computing node, a cloud processor and an early warning monitoring unit, wherein the data acquisition unit, the edge computing node, the cloud processor and the early warning monitoring unit are communicatively connected, and the system applies the smart port crane operation monitoring method described above;

[0045] The data acquisition unit is used to acquire real-time multi-source heterogeneous data of the crane operation and construct a dynamic twin data pool; the data acquisition unit and the data center of the port digital management and control platform are communicatively connected to acquire real-time multi-source heterogeneous data of the crane operation;

[0046] The edge computing node is used to preprocess the multi-source heterogeneous data, and through a space-time alignment algorithm, the physical coordinates of the crane, the container, the transport vehicle and the port infrastructure in the real environment are uniformly mapped to the world coordinate system to generate a high-precision dynamic digital twin model;

[0047] The cloud processor is used to analyze the multi-source heterogeneous data: the cloud processor includes a path adjustment module, an operation control module and a video comparison module; the path adjustment module is used to intelligently analyze the dynamic twin data pool and optimize the crane operation path in real time; the operation control module is used to construct a digital twin model and embed a multi-modal physical engine; the video comparison module is used to deeply analyze the video stream and perform virtual-real comparison;

[0048] The early warning monitoring unit is used to establish a hierarchical early warning mechanism to automatically handle abnormalities.

[0049] As described above, due to the adoption of the above technical solutions, the present application has the following advantages:

[0050] The application solves the problems of sensor data noise and dynamic load fluctuation by deep fusion of multi-source data and efficient management, establishes a dynamic twin data pool architecture, uses a sliding window adaptive standardization to improve data cleaning efficiency, realizes spatio-temporal alignment across data types through multi-modal encoding, and improves feature fusion accuracy;

[0051] The application dynamically adjusts the crane operation path trajectory by intelligently analyzing the dynamic twin data pool and optimizing the crane operation path in real time, thereby improving work efficiency; through the digital twin model and the integration of multi-modal physical engine, the physical state deviation of the crane operation is evaluated, and the operation plan is corrected in real time to improve operation accuracy; through deep analysis of video streams and virtual-real comparison, the simulation accuracy is improved by monitoring model prediction video deviation;

[0052] The scheme solves the problems of data island, decision lag and incomplete risk warning in traditional port crane operation through multi-source data fusion, high-precision digital twin, intelligent optimization algorithm and closed-loop early warning mechanism, realizes full-process digital management and control, and promotes the intelligent upgrading of the port. BRIEF DESCRIPTION OF DRAWINGS

[0053] Fig. 1 The connection schematic diagram of the system module of the application is shown;

[0054] Fig. 2 The method flowchart of the application is shown. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0056] Embodiment 1

[0057] As shown in Figs. 1-2 A smart port crane operation monitoring system, comprising a data acquisition unit, an edge computing node, a cloud processor and an early warning monitoring unit, wherein the data acquisition unit, the edge computing node, the cloud processor and the early warning monitoring unit are in communication connection;

[0058] The data acquisition unit is used for real-time acquisition of multi-source heterogeneous data of crane operation and construction of a dynamic twin data pool; the data acquisition unit and the data center of the port digital management and control platform are in communication connection, so as to real-time acquisition of multi-source heterogeneous data of crane operation;

[0059] The edge computing node is used for preprocessing multi-source heterogeneous data, and through a space-time alignment algorithm, physical coordinates of cranes, containers, transport vehicles and port infrastructure in a real environment are uniformly mapped to a world coordinate system to generate a high-precision dynamic digital twin model.

[0060] The cloud processor is used for analyzing multi-source heterogeneous data: the cloud processor includes a path adjustment module, a job control module and a video comparison module; the path adjustment module is used for intelligently analyzing a dynamic twin data pool and performing real-time optimization on a crane operation path; the job control module is used for constructing a digital twin model and embedding a multi-modal physical engine; the video comparison module is used for deeply analyzing a video stream and performing virtual-real comparison;

[0061] The early warning monitoring unit is used for establishing a hierarchical early warning mechanism to automatically process exceptions.

[0062] The working steps are as follows:

[0063] S1, real-time collection of multi-source heterogeneous data of crane operation and construction of a dynamic twin data pool: through a data center of a port digital management and control platform, multi-source heterogeneous data of crane operation is collected in real time;

[0064] Business databases of various business systems are synchronized to an intermediate library, data is extracted from the intermediate library by the data center, analyzed data is returned to each business system, and each business system is modified according to the feedback result. The list of business systems involved includes industry and finance integration systems, equipment and energy systems, comprehensive pipe network systems, production business integration systems, container business systems, roll-on / roll-off logistics production systems, production cost accounting systems, safety comprehensive management, logistics information networks, Internet of Things collection, etc. The port production, operation, equipment facilities, safety and environmental protection links are connected, data sharing, business cooperation and fine management of port operation are realized.

[0065] S11, the multi-source heterogeneous data includes sensor data, device log data and video stream data;

[0066] The sensor data includes coordinate parameters, angle parameters and weight parameters:

[0067] The sensor includes a position sensor, an inclination sensor and a load sensor, the position sensor includes a GPS and a laser radar; wherein, the coordinate parameters (x, y, z) are collected by the position sensor, the angle parameters (θ, φ) are collected by the inclination sensor, and the weight parameters F are collected by the load sensor;

[0068] The device log data includes a crane running speed v, a motor current I and an operation time T;

[0069] The video stream data includes crane visual parameters collected by a camera, including a gripper posture and a cargo state;

[0070] S12 uses a spatiotemporal alignment algorithm to uniformly map the physical coordinates of cranes, containers, transport vehicles and port infrastructure in the real environment to the world coordinate system, generating a high-precision dynamic digital twin model.

[0071] The original data is cleaned, denoised, missing values ​​are filled, and standardized using a wavelet transform model. A sliding window is set for dynamic adaptive standardization, thereby unifying the timestamps and spatial coordinate system. The specific process is as follows:

[0072] Let the current time be t; let the sliding window length be W, which is the number of data points contained in the window; let the index variable of time be i, used to traverse all data points in the sliding window; let the original data value at time i be... This allows us to obtain the local mean of the sliding window data at time t. : ;

[0073] Then, obtain the local standard deviation of the sliding window data at time t. :

[0074] ;

[0075] Local mean of the sliding window data at time t and local standard deviation The original data values ​​at time t are standardized, and the standardized data values ​​are labeled as follows: : ;

[0076] in, Smoothing factor and Greater than 0, used to avoid The denominator is 0, thereby eliminating the impact of sudden abnormal values ​​and adapting to dynamic load changes during crane operation;

[0077] S13 uses distributed storage technology to divide data types into a real-time streaming data layer, a historical data layer, and a metadata layer, thus constructing a dynamic twin data pool architecture.

[0078] The preprocessed multi-source heterogeneous data is encoded using a deep learning model and then mapped to a unified vector space. Sensor data is encoded using an LSTM (Long Short-Term Memory) network model, video stream data is encoded using a CNN (Convolutional Neural Network) model, and device log data is encoded using a Transformer self-attention mechanism neural network model.

[0079] S14, redefining the timestamp synchronization function T(ts, tv) to align the sensor data timestamp ts with the video stream data timestamp tv, and then aligning the sensor, video stream and log through the historical time nodes of the device log data to perform multi-source data space-time alignment;

[0080] Through dynamic adaptive processing and multi-modal deep fusion, the precision and efficiency of port crane data acquisition and twin modeling are significantly improved, so that the mathematical model has both theoretical rigor and engineering practicality.

[0081] S2, intelligently analyzing the dynamic twin data pool and real-time optimizing the crane operation path: analyzing the port production operation deviation through multi-source heterogeneous data, obtaining the efficiency index score of the crane operation, and outputting the crane dynamic adjustment strategy, the specific process is as follows:

[0082] Intelligently analyzing the dynamic twin data pool and real-time optimizing the crane operation path;

[0083] Analyzing the port production operation deviation through multi-source heterogeneous data to obtain the efficiency index score of the crane operation;

[0084] S21, extracting the task completion time, empty running rate, unit cargo energy consumption, emergency stop times and load overrun rate of the crane through the device log; marking the number of device log data indicators as n0;

[0085] Marking the reference value of any indicator p of the device log data as Dp0, and marking the actual value of the indicator p at time t as Dpt, obtaining the deviation amplitude value of the indicator p through the difference between the reference value and the actual value of the indicator p : ;

[0086] S22, calculating the deviation amplitude value of n0 indicators and performing maximum-minimum value normalization to obtain the standardized value Sp of the indicator p: , wherein, and are the maximum and minimum values of the n0 deviation amplitude values ;

[0087] S23, calculating the information entropy Hp of the indicator p to obtain the weight coefficient of the indicator p;

[0088] , wherein, ; Hq is the information entropy Hq of any indicator q in the n0 indicators, and P is the sequence set of the n0 indicators; the smaller the information entropy Hp of the indicator p, the higher the discrimination of the indicator p, and the larger the weight coefficient ;

[0089] S24, the deviation amplitude value of the index p and the weight coefficient weighted fusion, calculate the efficiency index score SC1 of the crane operation: .

[0090] S3, build a digital twin model and embed a multi-modal physical engine: thereby synchronously simulate the crane operation mechanics interaction process in the real environment, real-time detect the physical parameters of the crane grab, and analyze and generate a physical evaluation report of the crane operation, so as to real-time correct the operation plan and push the warning instruction to the surrounding system, the specific process is as follows:

[0091] Build a digital twin model and embed a multi-modal physical engine, synchronously simulate the crane operation mechanics interaction process in the real environment, real-time detect the physical parameters of the crane grab through sensors, the physical parameters include crane grab acceleration, load stress, container displacement deviation and vehicle positioning error, and analyze and calculate the physical action score SC2 of the crane operation: ;

[0092] Wherein, the number of indexes marked physical parameters is n1; the reference value of any index r marked physical parameter is Dr0, the actual value of index r at time t is marked as Drt, the deviation amplitude value of index r is obtained through the difference value between the reference value and the actual value of index r , and then the standardized value Sr, information entropy Hr and weight coefficient of index r are obtained, thereby obtaining the physical action score SC2 of the crane operation.

[0093] S4, deeply analyze the video stream and compare virtual and real: through the monitoring video stream of the port crane operation and the simulation video stream of the digital twin model, extract the key action features of the crane and compare them in time and space, generate the video deviation score of the crane operation, the specific process is as follows:

[0094] Compare and analyze the monitoring video stream of the port crane operation and the simulation video stream of the digital twin model;

[0095] Through frame-level analysis of the monitoring video stream, extract the key action features of the crane and compare them with the digital twin simulation video stream in time and space;

[0096] S41, divide the prediction categories of the digital twin model video stream for the key action features of the crane into n2, then the prediction category probability distribution of the digital twin model is : , and ;

[0097] Encode the real category label of the monitoring video stream of the crane operation as : Define the true category position y as 1, and y as 0 otherwise;

[0098] The prediction bounding box of the digital twin analog video stream is labeled B: The center point of the prediction box is (b1, b2), with a length of b3 and a width of b4.

[0099] Mark the ground truth frame of the monitoring video stream of the crane operation as C: The center point of the real bounding box is (c1, c2), the length is c3, and the width is c4.

[0100] S42, calculate the cross-entropy of the virtual-real comparison of crane motion features through spatiotemporal alignment comparison. By comparing the complete intersection and union (CIoU) of the two objects, the target detection loss Ld is obtained.

[0101] S4201, Cross-entropy ;

[0102] When predicting probability With real labels The closer they are, the smaller the cross-entropy value; when Then the correct category is predicted. Cross-entropy The value is 0; if the prediction is incorrect, then... Approaching 0 Negative numbers, especially the higher their absolute values, will increase the cross-entropy. The larger;

[0103] S4202, Complete Intersection and Union (CIoU): ;

[0104] Where IoU is the intersection-union ratio and ;

[0105] The distance between the center points of the two boxes and ;

[0106] The length of the diagonal of the smallest bounding rectangle containing the two boxes;

[0107] ;

[0108] The aspect ratio is a consistency measure factor. ,when The higher the value, the greater the inconsistency; for The weighting coefficients, ;

[0109] S4203, target detection loss Ld is obtained: ;

[0110] S43, the position coordinate sequence of the target point in the continuous m frames is marked as E, the change difference of adjacent frames is calculated, and the displacement of any pair of adjacent frames in the x direction and the y direction is marked as 、 ;

[0111] ;

[0112] The tracking consistency loss Lt is obtained: ;

[0113] When the trajectory of the target point is smoother, the position change of adjacent frames is smaller, and the tracking consistency loss Lt is smaller;

[0114] S44, the speed and acceleration of any pair of adjacent frames in the x direction and the y direction are calculated through the position coordinates of the target point in the continuous m frames and the time interval of adjacent frames;

[0115] The calculated speed and acceleration in the x direction are marked as Vx1 and Ax1 respectively, and the calculated speed and acceleration in the y direction are marked as Vy1 and Ay1 respectively;

[0116] The reasonable speed and acceleration in the x direction are marked as Vx2 and Ax2 respectively, and the reasonable speed and acceleration in the y direction are marked as Vy2 and Ay2 respectively, and the specific values are pre-set through experiments and experience;

[0117] The kinematic rationality loss Lm is obtained:

[0118] ;

[0119] S45, the target detection loss Ld, the tracking consistency loss Lt and the kinematic rationality loss Lm are weighted and fused to generate the video deviation score SC3 of the crane operation:

[0120] ;

[0121] Wherein, 、 、 The target detection loss Ld, the tracking consistency loss Lt and the kinematic rationality loss Lm are weight factor coefficients respectively, and the weight factor coefficients are pre-set and obtained through a large amount of experimental data calculation.

[0122] S5, establish a hierarchical early warning mechanism for automatic processing of abnormalities: through threshold comparison of efficiency index score of crane operation, physical evaluation report and deviation risk score, output device warning log, thereby automatically generating abnormal event traceability report of port crane for closed loop processing, the specific process is as follows:

[0123] S51, set the threshold Q1 of the efficiency index score SC1 of the crane operation, when the efficiency index score SC1 is lower than the threshold Q1, then re-plan the crane path trajectory, take the crane operation path corresponding to the maximum value of the efficiency index score SC1 as the optimal path strategy, and mark it as the crane dynamic adjustment strategy and output, the crane dynamic adjustment strategy includes dynamically adjusting the crane grabbing sequence, moving trajectory and collaborative operation strategy;

[0124] S52, set the threshold Q2 of the physical action score SC2 of the crane operation, evaluate the physical action state of the crane operation through threshold comparison, and generate corresponding warning instructions, so as to prompt the surrounding system to perform corresponding crane operation management; when the physical action score SC2 is lower than the threshold Q2, then re-plan the crane grabbing path, take the crane operation strategy corresponding to the maximum value of the efficiency index score SC1 as the optimal operation plan, and integrate to generate the physical evaluation report of the crane operation, so as to correct the operation plan in real time and push the warning instruction to the surrounding system;

[0125] S53, set the threshold Q3 of the video deviation score SC3 of the crane operation, evaluate the virtual-real deviation degree of the crane through threshold comparison, when the video deviation score SC3 is higher, the simulation prediction effect of the digital twin model is poor, then corresponding warning prompt is given to the background technical personnel;

[0126] Further automatically generate abnormal event traceability report of port crane: integrate the historical operation data and device warning log of port data center into abnormal event traceability report, and push to preventive equipment collaborative management platform for closed loop processing;

[0127] Realize the whole process digital monitoring and intelligent control of intelligent port crane operation, and improve the port operation efficiency and safety.

[0128] As described above, through multi-source data deep fusion and efficient management, the dynamic twin data pool architecture is established, the sliding window adaptive standardization is adopted, the sensor data noise and dynamic load fluctuation problem is solved, the data cleaning efficiency is improved, the cross-data type space-time alignment is realized through multi-modal coding, and the feature fusion accuracy is improved;

[0129] The application dynamically adjusts the crane operation path trajectory and improves work efficiency by intelligently analyzing dynamic twin data pools and optimizing the crane operation path in real time; the application improves operation accuracy by integrating a multi-modal physical engine with a digital twin model to evaluate crane operation physical state deviation and correct operation plans in real time; and the application improves simulation accuracy by deeply analyzing video streams and comparing virtual and real deviations to monitor model prediction video deviation.

[0130] The scheme solves the problems of data islands, decision lag and incomplete risk warning in traditional port crane operation through multi-source data fusion, high-precision digital twin, intelligent optimization algorithm and closed-loop early warning mechanism, realizes full-process digital management and control, and promotes the intelligent upgrading of ports.

[0131] The above embodiments can be realized by software, hardware, firmware or any combination thereof, in whole or in part. When realized by software, the above embodiments can be realized in the form of a computer program product in whole or in part. Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution.

[0132] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment according to actual needs.

[0133] The above data processing is a dimensionless numerical calculation, and the size of the interval and threshold is set for easy comparison. The size of the threshold depends on the number of sample data and the base number set by the person skilled in the art for each group of sample data. As long as it does not affect the proportional relationship between the parameters and the quantized values, the preset parameters are set by the person skilled in the art according to the actual situation.

[0134] The above is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can make equivalent replacements or changes to the technical solutions and inventive concepts of the application within the technical scope disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. A method for monitoring the operation of intelligent port cranes, characterized in that, Includes the following steps: S1, Real-time collection of multi-source heterogeneous data on crane operations and construction of a dynamic twin data pool: Real-time collection of multi-source heterogeneous data on crane operations through the data platform of the port digital management and control platform; S2 intelligently analyzes the dynamic twin data pool and optimizes the crane operation path in real time: it analyzes port production operation deviations through multi-source heterogeneous data analysis, obtains the efficiency index score of crane operation, and outputs the dynamic adjustment strategy of crane. S3 constructs a digital twin model and embeds a multimodal physics engine: by synchronously simulating the mechanical interaction process of crane operation in a real environment, it detects the physical parameters of the crane gripper in real time, analyzes and generates a physical assessment report of crane operation, thereby correcting the operation plan in real time and pushing early warning instructions to the surrounding system. S4, Deeply analyze video streams and perform virtual-real comparison: By comparing the monitoring video stream of port crane operations with the simulated video stream of the digital twin model, extract key crane action features and perform spatiotemporal alignment comparison to generate a video deviation score for crane operations. S5, establish a graded early warning mechanism for automatic anomaly handling: compare thresholds by using crane operation efficiency index scores, physical assessment reports and video deviation scores, output equipment early warning logs, and automatically generate port crane abnormal event tracing reports for closed-loop processing; The specific process of S2 is as follows: Indicators are extracted from equipment log data, including crane task completion time, empty run rate, energy consumption per unit of cargo, number of emergency stops, and overload rate; the number of indicators in the equipment log data is marked as n0. The baseline value of any indicator p in the device log data is Dp0. The actual value of indicator p at time t is marked as Dpt. The deviation magnitude of indicator p is obtained by the difference between the baseline value and the actual value of indicator p. By calculating the deviation range of n0 indicators Then perform maximum-minimum standardization to obtain the standardized value Sp of index p; Then, by calculating the information entropy Hp of indicator p, the weight coefficient of indicator p can be obtained. ; The deviation magnitude value of index p and weighting coefficients We perform weighted fusion to calculate the efficiency index score SC1 for crane operation; The specific process of S3 is as follows: The physical parameters include crane gripper acceleration, load stress, container displacement deviation, and vehicle positioning error. The number of indicators marking the physical parameters is n1. The baseline value of any physical parameter index r is Dr0, and the actual value of index r at time t is labeled Drt. The deviation magnitude of index r is obtained by the difference between the baseline value and the actual value of index r. This allows us to obtain the standardized value Sr of the indicator r, the information entropy Hr, and the weight coefficients. ; The deviation magnitude value of index r and weighting coefficients A weighted fusion method is used to calculate the physical motion score SC2 for crane operations.

2. The intelligent port crane operation monitoring method according to claim 1, characterized in that: Multi-source heterogeneous data includes sensor data, device log data, and video stream data; The spatiotemporal alignment algorithm maps the physical coordinates of cranes, containers, transport vehicles and port infrastructure in the real environment to the world coordinate system, generating a high-precision dynamic digital twin model. The original data is cleaned, denoised, missing values ​​are filled, and standardized using a wavelet transform model. A sliding window is set for dynamic adaptive standardization, thereby unifying the timestamps and spatial coordinate system. The specific process is as follows: Let the current time be t; let the sliding window length be W, which is the number of data points contained in the window; let the index variable of time be i, used to traverse all data points in the sliding window; let the original data value at time i be... This allows us to obtain the local mean of the sliding window data at time t. ; Then, obtain the local standard deviation of the sliding window data at time t. ; Local mean of the sliding window data at time t and local standard deviation The original data values ​​at time t are standardized, and the standardized data values ​​are labeled as follows: ; By using distributed storage technology, data types are divided into a real-time streaming data layer, a historical data layer, and a metadata layer, thus constructing a dynamic twin data pool architecture. By defining a timestamp synchronization function T(ts, tv) and the historical time nodes of device log data, multi-source data spatiotemporal alignment of sensors, video streams, and logs is achieved.

3. The intelligent port crane operation monitoring method according to claim 2, characterized in that: The specific process of S4 is as follows: By performing frame-level analysis on the monitoring video stream, key motion features of the crane are extracted and compared with the digital twin analog video stream in a spatiotemporal alignment. If the predicted categories of key crane motion features from the digital twin model video stream are divided into n² categories, then the probability distribution of the predicted categories of the digital twin model is as follows: Encode the real category label of the monitoring video stream of crane operation as... ; The prediction bounding box of the digital twin analog video stream is labeled B: The center point of the prediction box is (b1, b2), with a length of b3 and a width of b4. Mark the ground truth frame of the monitoring video stream of the crane operation as C: The center point of the real bounding box is (c1, c2), the length is c3, and the width is c4. Cross-entropy of crane motion features (real-virtual comparison) is calculated by spatiotemporal alignment comparison. By comparing the complete intersection and union (CIoU) of the two objects, the target detection loss Ld is obtained. The position coordinate sequence of the target point in m consecutive frames is denoted as E, and the difference in change between adjacent frames is calculated. The displacements in the x and y directions of any pair of adjacent frames are respectively denoted as E. , Thus, the tracking consistency loss Lt is obtained; By using the position coordinates of the target point in m consecutive frames and the time interval between adjacent frames, the velocity and acceleration in the x and y directions of any pair of adjacent frames are calculated, thereby obtaining the kinematic rationality loss Lm. The target detection loss Ld, tracking consistency loss Lt, and kinematic rationality loss Lm are weighted and fused to generate a video deviation score SC3 for crane operation.

4. The intelligent port crane operation monitoring method according to claim 3, characterized in that: The specific process of S5 is as follows: Establish a tiered early warning mechanism, compare thresholds using crane operation efficiency index scores, physical assessment reports, and video deviation scores, and output equipment early warning logs; Set a threshold Q1 for the efficiency index score SC1 of crane operation. When the efficiency index score SC1 is lower than the threshold Q1, the crane path trajectory is replanned. The crane operation path corresponding to the maximum value of the efficiency index score SC1 is the optimal path strategy, and it is marked as the crane dynamic adjustment strategy and output. Set a threshold Q2 for the physical motion score SC2 of crane operation. Evaluate the physical motion status of crane operation by comparing the threshold and generate corresponding early warning instructions to prompt the surrounding system to perform corresponding crane operation management. Set a threshold Q3 for the video deviation score SC3 of crane operation. By comparing the threshold, evaluate the degree of virtual and real deviation of the crane, and thus provide corresponding early warning prompts to the back-end technicians. This automatically generates anomaly reports for port cranes: historical operation data and equipment warning logs from the port data platform are integrated into anomaly reports and pushed to the preventive equipment collaborative management platform for closed-loop processing.

Images