Port portal crane operation state recognition method

Abstract

The application discloses a port portal crane operation state recognition method and relates to the technical field of port equipment monitoring, which comprises the following steps: deploying a sensor group containing a camera, a PLC data acquisition module, a laser radar, a millimeter wave radar and an angle sensor at a key position of a port portal crane; collecting image data of the portal crane operation through the camera; and acquiring operation control data of the portal crane through the PLC data acquisition module; the application realizes the effects of comprehensive acquisition of portal crane operation state information and improvement of real-time performance and reliability of recognition by deploying multiple source sensors and adopting a unique data fusion algorithm; the effects of accurate recognition of the portal crane operation state under different light and weather conditions are realized by collecting environmental parameters in real time and automatically adjusting image preprocessing and recognition algorithm parameters; the effects of real-time feedback of the portal crane operation state and timely warning of abnormal conditions are realized by establishing a real-time monitoring and abnormal warning mechanism, and the effect of ensuring operation safety is realized.

Description

Technical Field

[0001] This invention relates to the field of port equipment monitoring technology, specifically a method for identifying the operational status of port gantry cranes. Background Technology

[0002] Ports are vital hubs connecting land and sea transportation, undertaking crucial tasks such as cargo loading, unloading, storage, and transshipment. They are indispensable nodes in the global supply chain, and their operational efficiency and safety directly impact the overall rhythm of logistics and transportation. Port gantry cranes, also known as port portal cranes, are core equipment in port cargo handling operations. They are primarily used for lifting, moving, and lowering containers, bulk cargo, and other goods, and are widely applied in cargo handling scenarios at the wharf. They are key equipment for ensuring the orderly operation of port activities.

[0003] Port gantry crane operation status identification refers to the process of using technical means to perceive and judge the operating status, load, and any abnormalities of the gantry crane during operation in real time. Accurate port gantry crane operation status identification provides port managers with real-time data support for gantry crane operation. On the one hand, it helps managers to rationally allocate gantry crane resources, optimize operational processes, and improve the overall efficiency of port operations; on the other hand, it can promptly detect abnormalities in gantry crane operation, prevent the escalation of equipment failures, and ensure the safety of personnel and equipment, thus having significant practical implications.

[0004] However, existing port gantry crane operation status recognition technologies still have certain shortcomings. Existing technologies cannot fully acquire multi-dimensional information during gantry crane operations, resulting in limited perception of the operation status and difficulty in accurately reflecting the actual operating conditions of the gantry crane. They also have poor adaptability to the complex port environment. When faced with changes in lighting or severe weather, the image preprocessing and recognition algorithm parameters cannot be flexibly adjusted, leading to a significant drop in recognition accuracy. Existing technologies only achieve status recognition functions and lack supporting real-time monitoring and early warning mechanisms. They cannot promptly alert staff when the gantry crane exhibits abnormal operating conditions, posing safety hazards. Therefore, developing a port gantry crane operation status recognition method is of great significance. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a method for identifying the operational status of port gantry cranes. By deploying multi-source sensors and employing a unique data fusion algorithm, it can achieve comprehensive acquisition and identification of gantry crane operational status information, improving real-time performance and reliability. By collecting environmental parameters in real time and automatically adjusting image preprocessing and recognition algorithm parameters, it can accurately identify the operational status of gantry cranes under different lighting and weather conditions. By establishing a real-time monitoring and anomaly early warning mechanism, it can achieve real-time feedback on the operational status of gantry cranes and timely warning of abnormal situations, ensuring operational safety.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for identifying the operating status of a port gantry crane, the method comprising the following steps:

[0007] S1. Multi-source sensor data acquisition: Deploy sensor groups including cameras, PLC data acquisition modules, lidar, millimeter-wave radar and angle sensors at key locations of the port gantry crane. The cameras acquire image data of the gantry crane operation, the PLC data acquisition module acquires the gantry crane operation control data, the lidar and millimeter-wave radar acquire spatial position and distance data of the gantry crane operation area, and the angle sensors acquire angle change data of the gantry crane components, thus simultaneously acquiring multi-dimensional gantry crane operation status information.

[0008] S2. Multi-source data fusion processing: A data fusion algorithm is adopted. First, the collected multi-source sensor data is synchronized in time based on timestamp alignment technology. Then, spatial matching of multi-source sensor data is achieved through spatial coordinate transformation, and different types of sensor data are associated with the same spatial coordinate system. Finally, the multi-source data after time synchronization and spatial matching is fused and analyzed.

[0009] S3. Environmental Adaptive Adjustment: Real-time collection of light intensity and weather conditions parameters of the port operation environment; based on the collected light intensity and weather conditions parameters, automatically adjust the image preprocessing parameters and the gantry crane operation status recognition algorithm parameters.

[0010] S4. Real-time monitoring and anomaly warning: Based on the multi-source data after fusion processing and the adjusted recognition algorithm, the gantry crane operation status is identified and fed back in real time. The normal threshold range of the gantry crane operation status is preset. The real-time identified gantry crane operation status data is compared with the normal threshold range. If the real-time data exceeds the normal threshold range, the warning mechanism is triggered.

[0011] Furthermore, in step S1, the camera is installed below the main beam of the gantry crane and on the side of the gantry crane column. The PLC data acquisition module establishes a communication connection with the gantry crane control system. The lidar and millimeter-wave radar are installed at the bottom of the gantry crane trolley. The angle sensor is installed at the hinge point of the gantry crane boom and the rotation axis of the hook. The sensors at different locations collect image data of the gantry crane hook working area, image data of the gantry crane boom moving area, operation control data of the gantry crane motor speed and brake status, spatial position and distance data of the hook and goods and the gantry crane and surrounding equipment, and angle change data of the gantry crane boom pitch angle and hook rotation angle. To clarify the priority order of different sensor data during processing, a sensor data acquisition priority coefficient is defined. Through formula Calculate the priority of data collected by each sensor, where For the first Historical data recognition accuracy of sensor-like devices For the first Current signal stability of the sensor For the first The criticality of sensor data for identifying operational status , , The priority coefficient is determined by iterative optimization using gradient descent based on the gantry crane's operational data over the past 12 months. This ensures that the priority coefficient accurately reflects the importance of sensor data in the current operational scenario, and sensor data with higher priority coefficients are prioritized during data processing.

[0012] Furthermore, in step S2, when the data fusion algorithm performs time synchronization, the sampling time of the PLC data acquisition module is used as the reference time. Each sensor records its data acquisition time through a timestamp module. The time difference between each sensor's data acquisition time and the reference time is calculated and corrected to ensure that the data acquired by all sensors correspond to the same operating time of the gantry crane. After completing time synchronization and spatial matching, the data is then processed using the formula... The fused data values ​​are calculated to achieve effective integration of multi-source data. For the first Raw data from sensors after time synchronization and spatial matching. For the first The fusion weights of sensor-like data The importance of sensor data is determined by the analytic hierarchy process. First, a matrix is ​​constructed to judge the importance of sensor data. Then, the rationality of the matrix is ​​verified by consistency check. Finally, the fusion weight of each sensor data is calculated to ensure that the fused data can integrate the advantages of each sensor.

[0013] Furthermore, during the spatial coordinate transformation in step S2, a spatial rectangular coordinate system is established with the intersection of the center line of the gantry crane track and the gantry crane column as the origin. The installation coordinate parameters of the camera, lidar, millimeter-wave radar, and angle sensor in this spatial rectangular coordinate system are measured in advance. The relative distance data collected by the lidar and millimeter-wave radar are converted into absolute coordinate data in the spatial rectangular coordinate system. The angle data collected by the angle sensor is converted into absolute coordinate data of the gantry crane boom endpoint and hook endpoint in the spatial rectangular coordinate system through trigonometric function calculation, so that all sensor data are unified into the same spatial rectangular coordinate system.

[0014] Furthermore, in step S3, environmental parameters are collected by installing a light sensor, a temperature and humidity sensor, and a rain and snow sensor on the top of the door operator. The light sensor collects the ambient light intensity value, the temperature and humidity sensor collects the ambient temperature and humidity data and judges the foggy weather situation by combining the humidity change trend, and the rain and snow sensor detects the rain and snow status. Each sensor transmits the collected environmental parameters to the door operator data processing module in real time.

[0015] Furthermore, the method for automatically adjusting the image preprocessing parameters in step S3 is as follows:

[0016] When the light intensity value collected by the light sensor is lower than the preset lower limit, the brightness gain parameter and exposure time parameter of the image preprocessing are adjusted to increase the image brightness gain and extend the exposure time.

[0017] When the light intensity value is higher than the preset upper limit, the image brightness gain is reduced and the exposure time is shortened, while the image contrast parameter is adjusted to enhance the image contrast.

[0018] When the rain and snow sensor detects rain or snow, it activates the image morphological filtering algorithm.

[0019] When the temperature and humidity sensor determines that it is in foggy weather, the dark channel prior algorithm is used for image processing.

[0020] Furthermore, the method for automatically adjusting the state recognition algorithm parameters in step S3 is as follows:

[0021] When the environmental parameters collected by the light sensor, temperature and humidity sensor, and rain and snow sensor are within the preset normal range, conventional feature extraction weights are used.

[0022] When environmental parameters exceed the preset normal range, the feature extraction weights in the recognition algorithm need to be adjusted. Increase the extraction weights of the gantry crane structure outline and hook shape geometric features, decrease the extraction weights of color features, and adjust the threshold parameters of feature matching in the recognition algorithm.

[0023] To accurately calculate the adjusted feature extraction weights, the formula is used. Calculate the adjusted first Class feature extraction weights ,in For the first time under normal conditions Class feature extraction weights, For the first The anti-interference coefficient of class features in the current environment. This is the weighting adjustment factor. The accuracy deviation of this type of feature recognition under the same historical conditions is determined by statistical analysis. Through multiple environmental simulation experiments, combined with the target value of recognition accuracy, the adjusted weights are determined to ensure that they can adapt to the current environment.

[0024] Furthermore, the early warning mechanism in step S4 includes:

[0025] An audible and visual alarm is installed in the gantry crane control room. When the real-time gantry crane operation status data exceeds the normal threshold range, the audible and visual alarm is activated and emits an audible alarm signal and a flashing light signal.

[0026] A wireless communication module is integrated into the gantry crane data processing module. The wireless communication module sends abnormal operation status information, abnormal occurrence time data, gantry crane number information, and abnormal location data to the port management terminal and the staff's mobile terminal.

[0027] Compared with existing technologies, this port gantry crane operation status identification method has the following advantages:

[0028] This invention addresses the problems of incomplete information and difficulty in synchronizing and matching multi-source data caused by relying on a single sensor in existing technologies by deploying multi-source sensors and employing a unique data fusion algorithm. It achieves comprehensive acquisition and recognition of gantry crane operation status information, improving real-time performance and reliability. By collecting environmental parameters in real time and automatically adjusting image preprocessing and recognition algorithm parameters, it solves the problem of poor adaptability to complex environments in existing technologies, achieving accurate recognition of gantry crane operation status under different lighting and weather conditions. By establishing a real-time monitoring and anomaly early warning mechanism, it solves the problem of lack of anomaly alerts in existing technologies, enabling real-time feedback of gantry crane operation status and timely warning of abnormal situations, thus ensuring operational safety.

[0029] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0031] Figure 1 A flowchart for a method to identify the operational status of a port gantry crane;

[0032] Figure 2 A flowchart of a method for identifying the operational status of port gantry cranes. Detailed Implementation

[0033] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0034] The port gantry crane operation status identification method provided by this invention comprises four core steps, forming a complete gantry crane operation status identification and early warning system. (See also...) Figure 1 and Figure 2 The specific details are as follows:

[0035] Firstly, multi-source sensor data acquisition is implemented. Sensor arrays consisting of cameras, PLC data acquisition modules, LiDAR, millimeter-wave radar, and angle sensors are deployed at key locations on the port gantry crane. Specifically, cameras are installed below the main beam and on the side of the columns to collect operational image data; the PLC data acquisition module communicates with the gantry crane control system to acquire operational control data; the LiDAR and millimeter-wave radar are mounted on the bottom of the gantry crane trolley to collect spatial position and distance data; and the angle sensors are located at the boom hinge point and hook rotation axis to collect angle change data. Simultaneously, sensor data acquisition priorities are defined, and higher-priority data is processed first.

[0036] Secondly, multi-source data fusion processing is carried out. A data fusion algorithm is adopted. First, the sampling time of PLC data is used as the benchmark to realize the time synchronization of multi-source data through timestamp alignment. Then, a spatial rectangular coordinate system is established with the intersection of the center line of the gantry crane track and the column as the origin to complete the spatial matching of multi-source data. Finally, the synchronized and matched multi-source data is integrated.

[0037] Next is the environmental adaptive adjustment. Environmental parameters are collected by the light, temperature, humidity, rain and snow sensors on the top of the door machine. Based on this, the image preprocessing parameters are automatically adjusted. For example, when the lighting is abnormal, the brightness gain, exposure time and contrast are adjusted. Morphological filtering is used on rainy or snowy days and dark channel prior algorithm is used on foggy days. At the same time, the state recognition algorithm parameters are adjusted. When the environment is abnormal, the feature extraction weight and feature matching threshold are changed.

[0038] Finally, there is real-time monitoring and anomaly early warning. Based on fused data and adjusted algorithms, the system identifies and provides feedback on the gantry crane's operating status in real time, presets normal threshold ranges, and triggers an early warning when data exceeds the limits. The alarm is triggered by the audible and visual alarm in the control room, and the abnormal information is sent to the port management terminal and the staff's mobile terminal via the wireless communication module.

[0039] Example 1

[0040] This embodiment applies to a scenario involving the operational status recognition of a gantry crane at a coastal container port. This port handles a large daily container throughput, and the gantry cranes need to operate continuously under varying lighting and complex weather conditions. Furthermore, precise control of the crane's operational status is crucial to prevent delays in cargo loading / unloading or safety accidents due to equipment malfunctions. Based on this scenario, see [link to relevant documentation]. Figure 1 and Figure 2 The port gantry crane operation status identification method of the present invention enables comprehensive, real-time, and accurate identification and early warning of anomalies in the gantry crane operation status, ensuring efficient and safe port operations. The specific steps are as follows:

[0041] In the multi-source sensor data acquisition phase, sensor arrays are first deployed at key locations on the port gantry crane. Cameras are installed below the main beam and on the side of the crane's columns. The camera below the main beam collects image data of the crane hook's operating area, while the camera on the side of the columns collects image data of the crane boom's movement area. The PLC data acquisition module establishes a communication connection with the crane control system to obtain operational control data such as the crane motor speed and brake status. LiDAR and millimeter-wave radar are installed at the bottom of the crane trolley to collect spatial position and distance data between the hook and cargo, and between the crane and surrounding equipment. Angle sensors are installed at the crane boom hinge point and the hook rotation axis to collect data on changes in the crane boom pitch angle and hook rotation angle.

[0042] To clarify the priority order of different sensor data during processing, a sensor data acquisition priority coefficient is defined. Through formula Calculate the priority of data collected by each sensor. For the first Historical data recognition accuracy of sensor-like devices For the first Current signal stability of the sensor For the first The criticality of sensor data for identifying operational status , , The weighting coefficient is determined iteratively using gradient descent based on the gantry crane's operational data from the past 12 months. During actual data processing, sensor data with higher priority coefficients are prioritized to ensure that information more critical to operational status identification and more reliable data is processed first.

[0043] The process then moves to the multi-source data fusion processing stage, employing a data fusion algorithm. First, time synchronization is performed, using the sampling time of the PLC data acquisition module as the baseline time. Each sensor records its data acquisition time using its own timestamp module. The time difference between each sensor's data acquisition time and the baseline time is calculated and corrected to ensure that the data collected by all sensors corresponds to the same operating time of the gantry crane, avoiding identification errors caused by data time asynchrony.

[0044] Next, a spatial coordinate transformation is performed, establishing a spatial rectangular coordinate system with the intersection of the gantry crane's main track centerline and the gantry crane's column as the origin. The installation coordinate parameters of the camera, LiDAR, millimeter-wave radar, and angle sensor in this spatial rectangular coordinate system are pre-measured, and the relative distance data collected by the LiDAR and millimeter-wave radar are converted into absolute coordinate data in the spatial rectangular coordinate system. Trigonometric function calculations are then used to convert the angle data collected by the angle sensor into absolute coordinate data of the gantry crane boom endpoint and hook endpoint in the spatial rectangular coordinate system, achieving uniformity of all sensor data within the same spatial coordinate system.

[0045] After completing time synchronization and spatial matching, the formula is used. Calculate the merged data values. For the first Raw data from sensors after time synchronization and spatial matching. For the first The fusion weights of sensor-like data The importance of sensor data is determined using the analytic hierarchy process (AHP). First, a sensor data importance judgment matrix is ​​constructed. Then, the rationality of the matrix is ​​verified through a consistency check. Finally, the fusion weight of each sensor data is calculated, thereby achieving effective integration of multi-source data and improving data reliability by combining the advantages of each sensor.

[0046] During the environmental adaptive adjustment phase, environmental parameters are collected by light sensors, temperature and humidity sensors, and rain and snow sensors installed on the top of the door operator. The light sensor collects the ambient light intensity value in real time, the temperature and humidity sensor collects the ambient temperature and humidity data and combines the humidity change trend to determine whether it is a foggy day, and the rain and snow sensor detects whether it is raining or snowing. Each sensor transmits the collected environmental parameters to the door operator's data processing module in real time.

[0047] The data processing module automatically adjusts image preprocessing parameters based on different environmental parameters: when the light intensity value is lower than the preset lower limit, the brightness gain and exposure time parameters of the image preprocessing are adjusted to increase the image brightness gain and extend the exposure time; when the light intensity value is higher than the preset upper limit, the image brightness gain is reduced and the exposure time is shortened, while the image contrast parameter is adjusted to enhance the image contrast; when the rain and snow sensor detects rain or snow, the image morphological filtering algorithm is activated; when the temperature and humidity sensor determines that it is a foggy day, the dark channel prior algorithm is used for image processing.

[0048] Meanwhile, the data processing module will also adjust the parameters of the gantry crane operation status recognition algorithm: if the environmental parameters are within the preset normal range, the conventional feature extraction weights will be used; if the environmental parameters exceed the preset normal range, the feature extraction weights in the recognition algorithm will be adjusted, increasing the extraction weights of the gantry crane structure outline and hook shape geometric features, decreasing the extraction weights of color features, and adjusting the threshold parameters of feature matching in the recognition algorithm.

[0049] To accurately calculate the adjusted feature extraction weights, the formula is used. Calculate the adjusted first Class feature extraction weights .in For the first time under normal conditions Class feature extraction weights, For the first The anti-interference coefficient of class features in the current environment. The accuracy deviation of this type of feature recognition under the same historical conditions is determined by statistical analysis. This is the weighting adjustment factor. Through multiple environmental simulation experiments, combined with the target value of recognition accuracy, the adjusted weights are determined to ensure that they can adapt to the current environment.

[0050] Finally, the real-time monitoring and anomaly early warning stage begins. Based on the fused multi-source data and adjusted recognition algorithms, the gantry crane's operating status is identified in real time, and the results are fed back, allowing port management personnel to monitor the gantry crane's operation in real time. A pre-set normal threshold range for the gantry crane's operating status is used to compare the real-time identified gantry crane operating status data with this normal threshold range. If the real-time data exceeds the normal threshold range, an early warning mechanism is triggered.

[0051] The early warning mechanism includes two aspects: First, an audible and visual alarm is installed in the gantry crane control room. When real-time data exceeds the normal threshold range, the audible and visual alarm is immediately activated, emitting an audible alarm signal and a flashing light signal to remind the staff in the control room to pay attention to the abnormal situation of the gantry crane. Second, a wireless communication module is integrated into the gantry crane data processing module. The wireless communication module sends abnormal operation status information, abnormal occurrence time data, gantry crane number information, and abnormal location data to the port management terminal and the staff's mobile terminal, so that management personnel can obtain abnormal information of the gantry crane in a timely manner no matter where they are, which facilitates the rapid arrangement of personnel for investigation and handling.

[0052] This embodiment effectively solves the problems of incomplete information acquisition, poor adaptability to complex environments, and lack of anomaly warning in traditional gantry crane operation status identification by fully executing four steps: multi-source sensor data acquisition, multi-source data fusion processing, environmental adaptive adjustment, and real-time monitoring and anomaly early warning. It can accurately identify the gantry crane operation status under different lighting and weather conditions, provide real-time feedback on the gantry crane's operating status, and issue timely warnings in case of anomalies. This significantly improves the safety and efficiency of port gantry crane operations, reduces operational delays caused by equipment malfunctions, and provides strong technical support for the efficient operation of ports.

[0053] Example 2

[0054] This embodiment applies to a scenario involving the operational status identification of gantry cranes at an inland river bulk cargo port. This port primarily handles the loading and unloading of bulk cargo such as coal and ore. The gantry cranes need to operate continuously in alternating low-light and high-light conditions, frequent rainy weather during the rainy season, and occasional foggy or snowy weather in winter. Furthermore, precise monitoring of the gantry crane's boom pitch and hook lifting status is required during operation to prevent safety accidents caused by bulk cargo overloading or equipment malfunction. (See [link to relevant documentation]). Figure 1 and Figure 2 Based on this scenario, the port gantry crane operation status recognition method of the present invention is adopted to realize status recognition and anomaly warning of the entire gantry crane operation process, ensuring the efficiency and safety of bulk cargo loading and unloading operations. The specific steps are as follows:

[0055] During the multi-source sensor data acquisition phase, sensor groups are first deployed at key locations based on the characteristics of gantry crane operations. Cameras are installed below the main beam and on the side of the column. The camera below the main beam focuses on the contact area between the hook and the bulk cargo, collecting image data of the cargo grabbing and lifting process; the camera on the side of the column covers the movement trajectory of the gantry crane boom, collecting image data of the boom's pitch and rotation processes. The PLC data acquisition module is connected to the gantry crane control system via wired communication to acquire real-time operational control data such as the gantry crane motor output power, brake opening and closing status, and winch speed, providing a basis for judging the gantry crane's load status.

[0056] LiDAR and millimeter-wave radar are installed at the bottom of the gantry crane trolley. The LiDAR accurately collects distance data between the hook and the cargo stack, and between the gantry crane and surrounding yard equipment. The millimeter-wave radar assists in collecting spatial position data in rainy, foggy, or snowy weather. The two work together to ensure the accuracy of position information in complex environments. Angle sensors are installed at the boom hinge point and the hook rotation axis. The sensor at the boom hinge point collects data on the boom pitch angle change, and the sensor on the hook rotation axis collects data on the hook's horizontal rotation angle, simultaneously acquiring multi-dimensional gantry crane operation status information.

[0057] To optimize the data processing order, a priority coefficient for sensor data acquisition is defined. Through formula Calculate the priority of each sensor's data. For the first The historical data recognition accuracy of sensors, such as PLC data which comes directly from the door machine control system, is relatively high. For the first The current signal stability of similar sensors, such as the better stability of millimeter-wave radar signals in rainy or snowy weather compared to lidar; For the first The criticality of sensor data for identifying operational status, such as angle sensor data being crucial for determining whether the boom is exceeding its limits; , , The weighting coefficients are determined iteratively using gradient descent to optimize data from the gantry crane's bulk cargo loading and unloading operations over the past 12 months. During actual data processing, sensor data with higher priority coefficients are prioritized to ensure critical status information is analyzed first, thus improving recognition efficiency.

[0058] The process then moves to the multi-source data fusion processing stage, employing a step-by-step data fusion algorithm. The first step is time synchronization. Using the sampling time of the PLC data acquisition module as the base time, each sensor records its data acquisition time via its built-in timestamp module. The difference between each sensor's data acquisition time and the base time is calculated, and a software algorithm corrects this difference. This ensures that data collected by cameras, LiDAR, angle sensors, etc., corresponds to the same operating time of the gantry crane, avoiding misjudgments of status due to data time differences.

[0059] The second step involves spatial coordinate transformation. A rectangular coordinate system is established with the intersection of the gantry crane's main track centerline and the column as the origin. The X-axis extends along the gantry crane's track, the Y-axis is perpendicular to the track direction, and the Z-axis is perpendicular to the ground and upwards. The installation coordinate parameters of the cameras, LiDAR, millimeter-wave radar, and angle sensors within this coordinate system are pre-obtained using specialized measuring equipment. The relative distance data collected by the LiDAR and millimeter-wave radar are converted into absolute coordinate data based on the installation coordinates. Through trigonometric function calculations, the boom pitch angle and hook rotation angle collected by the angle sensors are converted into absolute coordinate data of the boom endpoint and hook within the coordinate system, achieving uniformity of all sensor data in the same spatial dimension.

[0060] The third step is data fusion, using formulas. Calculate the fused data values, where For the first Raw data from sensors after time synchronization and spatial matching. For the first The fusion weights of sensor-like data. The importance judgment matrix of sensor data is first constructed by using the analytic hierarchy process (AHP) to compare the contribution of different sensor data to state recognition. Then, the rationality of the matrix is ​​verified by consistency check, unreasonable judgment items are eliminated, and finally the fusion weight of each sensor data is calculated to achieve complementary advantages and effective integration of multi-source data.

[0061] During the environmental adaptive adjustment phase, environmental parameters are collected in real time by a sensor array on top of the door operator. The light sensor collects the ambient light intensity value at fixed intervals, the temperature and humidity sensor collects temperature and humidity data simultaneously, and determines whether it is a foggy day based on the characteristic of continuously rising humidity and temperature within a specific range. The rain and snow sensor determines whether it is raining or snowing by detecting changes in the resistance of the contact surface. All environmental parameters are transmitted to the door operator's data processing module in real time.

[0062] The data processing module automatically adjusts image preprocessing parameters based on environmental parameters: when the light intensity is below the preset lower limit, such as during nighttime operations, it increases the image brightness gain parameter and extends the exposure time to ensure that the images of the hook and goods are clearly distinguishable; when the light intensity is above the preset upper limit, such as during midday operations, it decreases the image brightness gain parameter and shortens the exposure time, while increasing the image contrast parameter to avoid overexposure caused by strong light; when the rain and snow sensor detects rain or snow, it activates the image morphological filtering algorithm to remove rain and snow interference noise from the image; when the temperature and humidity sensor determines that it is in foggy weather, it uses the dark channel prior algorithm to defog the image and restore a clear image of the gantry crane's operating area.

[0063] Meanwhile, the data processing module adjusts the parameters of the gantry crane operation status recognition algorithm: when the environmental parameters are within the preset normal range, such as during a sunny day, conventional feature extraction weights are used to extract color, contour, and texture features in the image in a balanced manner; when the environmental parameters exceed the preset normal range, such as during rainy weather, the extraction weights of geometric features such as the gantry crane structure contour and hook shape are increased, as these features are less affected by the environment. At the same time, the extraction weights of color features are reduced to avoid color distortion caused by rain affecting the recognition results. The threshold parameters for feature matching are also adjusted simultaneously to improve the algorithm's adaptability to environmental changes.

[0064] To accurately calculate the adjusted feature extraction weights, the formula is used. Calculate the adjusted first Class feature extraction weights ,in For the first time under normal conditions Class feature extraction weights, For the first The anti-interference coefficient of class features in the current environment, such as the contour feature having a higher anti-interference coefficient than the color feature in a rainy environment; The weight adjustment coefficient is determined by repeatedly simulating gantry crane operations under different environments and back-calculating the target value of recognition accuracy to ensure that the adjusted weights can adapt to the current environment and improve recognition accuracy.

[0065] Finally, the real-time monitoring and anomaly warning stage begins. Based on the fused multi-source data and adjusted recognition algorithms, the gantry crane's operating status is analyzed and identified in real time. The recognition results are displayed on the screen in the gantry crane control room in the form of text and images, allowing operators to monitor the gantry crane's operating status in real time. Normal threshold ranges for boom pitch angle, hook lifting speed, and motor output power are pre-set according to the gantry crane's design parameters and operating specifications. The real-time identified gantry crane operating status data is then compared with these normal threshold ranges.

[0066] If real-time data exceeds the normal threshold range, an early warning mechanism is immediately triggered: On the one hand, the audible and visual alarm in the gantry crane control room is activated, emitting a continuous buzzing alarm and flashing red light signal to remind the operator to stop the machine for inspection in time; on the other hand, the wireless communication module in the gantry crane data processing module sends abnormal operation status information, such as boom angle exceeding the limit, motor power overload, time of abnormality, gantry crane number, and abnormal location data, such as the boom part exceeding the limit, to the port management terminal and the mobile terminal of the on-site maintenance personnel. The management terminal can display the location and status of the abnormal gantry crane in real time, and the maintenance personnel can quickly rush to the site to troubleshoot the fault after receiving the information, reducing the operation interruption time.

[0067] In summary, this embodiment effectively solves the problems of incomplete information acquisition, poor adaptability to complex environments, and untimely anomaly warnings in inland waterway bulk cargo port gantry crane operations by fully executing four steps: multi-source sensor data acquisition, multi-source data fusion processing, environmental adaptive adjustment, and real-time monitoring and anomaly early warning. It can accurately identify the gantry crane's operating status under different conditions such as day-night cycles, rain, fog, and snow, providing real-time feedback on operational information and rapid early warnings in case of anomalies. This significantly reduces the incidence of equipment failures and safety accidents, improves the efficiency of bulk cargo loading and unloading operations, and provides reliable technical support for the stable operation of inland waterway bulk cargo ports.

[0068] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for identifying the operating status of a port gantry crane, characterized in that, The method includes the following steps: S1. Multi-source sensor data acquisition: Deploy sensor groups including cameras, PLC data acquisition modules, lidar, millimeter-wave radar and angle sensors at key locations of the port gantry crane. The cameras acquire image data of the gantry crane operation, the PLC data acquisition module acquires the gantry crane operation control data, the lidar and millimeter-wave radar acquire spatial position and distance data of the gantry crane operation area, and the angle sensors acquire angle change data of the gantry crane components, thus simultaneously acquiring multi-dimensional gantry crane operation status information. S2. Multi-source data fusion processing: A data fusion algorithm is adopted. First, the collected multi-source sensor data is synchronized in time based on timestamp alignment technology. Then, spatial matching of multi-source sensor data is achieved through spatial coordinate transformation, and different types of sensor data are associated with the same spatial coordinate system. Finally, the multi-source data after time synchronization and spatial matching is fused and analyzed. S3. Environmental Adaptive Adjustment: Real-time collection of light intensity and weather conditions parameters of the port operation environment; based on the collected light intensity and weather conditions parameters, automatically adjust the image preprocessing parameters and the gantry crane operation status recognition algorithm parameters. The method for automatically adjusting image preprocessing parameters is as follows: When the light intensity value collected by the light sensor is lower than the preset lower limit, the brightness gain parameter and exposure time parameter of the image preprocessing are adjusted to increase the image brightness gain and extend the exposure time. When the light intensity value is higher than the preset upper limit, the image brightness gain is reduced and the exposure time is shortened, while the image contrast parameter is adjusted. When the rain and snow sensor detects rain or snow, it activates the image morphological filtering algorithm. When the temperature and humidity sensor determines that it is in foggy weather, a dark channel prior algorithm is used for image processing; The method for automatically adjusting the parameters of the state recognition algorithm is as follows: When the environmental parameters collected by the light sensor, temperature and humidity sensor, and rain and snow sensor are within the preset normal range, conventional feature extraction weights are used. When environmental parameters exceed the preset normal range, the feature extraction weights in the recognition algorithm need to be adjusted. Increase the extraction weights of the gantry crane structure outline and hook shape geometric features, decrease the extraction weights of color features, and adjust the threshold parameters of feature matching in the recognition algorithm. To accurately calculate the adjusted feature extraction weights, the formula is used. Calculate the adjusted first Class feature extraction weights ,in For the first time under normal conditions Class feature extraction weights, For the first The anti-interference coefficient of class features in the current environment. This is the weighting adjustment factor; S4. Real-time monitoring and anomaly warning: Based on the multi-source data after fusion processing and the adjusted recognition algorithm, the gantry crane operation status is identified and fed back in real time. The normal threshold range of the gantry crane operation status is preset. The real-time identified gantry crane operation status data is compared with the normal threshold range. If the real-time data exceeds the normal threshold range, the warning mechanism is triggered.

2. The port gantry crane operation status identification method according to claim 1, characterized in that, In step S1, the camera is installed below the main beam of the gantry crane and on the side of the gantry crane column. The PLC data acquisition module establishes a communication connection with the gantry crane control system. The lidar and millimeter-wave radar are installed at the bottom of the gantry crane trolley, and the angle sensor is installed at the hinge point of the gantry crane boom and the rotation axis of the hook. To clarify the priority order of different sensor data during processing, a sensor data acquisition priority coefficient is defined. Through formula Calculate the priority of data collected by each sensor, where For the first Historical data recognition accuracy of sensor-like devices For the first Current signal stability of the sensor For the first The criticality of sensor data for identifying the operational status determines the priority of sensor data during data processing, prioritizing the use of sensor data with higher priority coefficients.

3. The port gantry crane operation status identification method according to claim 1, characterized in that, In step S2, when the data fusion algorithm performs time synchronization, the sampling time of the PLC data acquisition module is used as the reference time. Each sensor records its data acquisition time through a timestamp module. The time difference between each sensor's data acquisition time and the reference time is calculated and corrected to ensure that the data acquired by all sensors correspond to the same operating time of the gantry crane. After completing time synchronization and spatial matching, the formula is used... The fused data values ​​are calculated to achieve effective integration of multi-source data. For the first Raw data from sensors after time synchronization and spatial matching. For the first The fusion weights of sensor-like data.

4. The port gantry crane operation status identification method according to claim 1, characterized in that, In step S2, during the spatial coordinate transformation, a spatial rectangular coordinate system is established with the intersection of the center line of the gantry crane track and the gantry crane column as the origin. The installation coordinate parameters of the camera, lidar, millimeter-wave radar, and angle sensor in this spatial rectangular coordinate system are measured in advance. The relative distance data collected by the lidar and millimeter-wave radar are converted into absolute coordinate data in the spatial rectangular coordinate system. The angle data collected by the angle sensor is converted into absolute coordinate data of the gantry crane boom endpoint and hook endpoint in the spatial rectangular coordinate system through trigonometric function calculation, so that all sensor data are unified into the same spatial rectangular coordinate system.

5. The port gantry crane operation status identification method according to claim 1, characterized in that, In step S3, environmental parameters are collected by installing a light sensor, a temperature and humidity sensor, and a rain and snow sensor on the top of the door operator. The light sensor collects the ambient light intensity value, the temperature and humidity sensor collects the ambient temperature and humidity data and judges the foggy weather by combining the humidity change trend, and the rain and snow sensor detects the rain and snow conditions. Each sensor transmits the collected environmental parameters to the door operator data processing module in real time.

6. The port gantry crane operation status identification method according to claim 1, characterized in that, The early warning mechanism in step S4 includes: An audible and visual alarm is installed in the gantry crane control room. When the real-time gantry crane operation status data exceeds the normal threshold range, the audible and visual alarm is activated and emits an audible alarm signal and a flashing light signal. A wireless communication module is integrated into the gantry crane data processing module. The wireless communication module sends abnormal operation status information, abnormal occurrence time data, gantry crane number information, and abnormal location data to the port management terminal and the staff's mobile terminal.

Images