Gantry safety operation early warning method, system, device and medium

Abstract

This invention provides an early warning method, system, device, and medium for quay crane safety operations, applicable to the field of safety monitoring technology. The invention acquires first multi-source data within the quay crane operation area, providing multi-source data support for subsequent risk assessment and early warning processes. Then, based on preprocessed second multi-source data, target object identification and target object status identification are performed to obtain a structured data stream. This structured data is then mapped onto a target coordinate system to form a time-series trajectory. Based on the time-series trajectory, risk assessment and risk prediction are performed on the quay crane operation area. Finally, the early warning status is controlled based on the risk assessment and risk prediction results. This allows for risk assessment and early warning control of the quay crane operation area using data under the same coordinate system, effectively improving the accuracy of risk assessment and safety early warning during quay crane operations.

Description

Technical Field

[0001] This invention relates to the field of safety monitoring technology, and in particular to an early warning method, system, equipment and medium for safe operation of quay cranes. Background Technology

[0002] Quay cranes are responsible for container transfer during ship loading and unloading operations, which involve high-speed operations, complex environments, and limited visibility. Currently, monitoring videos of quay crane operations are only used for post-operation evidence collection, making it impossible for relevant personnel to know in a timely manner the spatial relationships between the spreader, containers, vehicles, and personnel involved in the operation. Furthermore, the signals and visual data from different existing equipment are isolated from each other, resulting in low accuracy of risk assessment and safety warnings. Summary of the Invention

[0003] The main objective of this invention is to provide an early warning method, system, device, and medium for safe operation of quay cranes, which can effectively improve the accuracy of risk assessment and safety early warning during quay crane operations.

[0004] To achieve the above objectives, one embodiment of the present invention provides an early warning method for safe operation of quay cranes, the method comprising the following steps:

[0005] Acquire first multi-source data of the quay crane operation area. The first multi-source data includes status data detected by a preset controller, image data collected by multiple types of image acquisition devices at multiple angles, or distance data collected by multiple ranging devices at different locations.

[0006] The first multi-source data is preprocessed to obtain the second multi-source data;

[0007] The second multi-source data is subjected to target object identification and target object state identification to obtain a structured data stream, wherein each target object in the structured data stream is assigned a unique object identifier;

[0008] The structured data is mapped onto the target coordinate system to form a time-series trajectory. Each point in the time-series trajectory consists of the target object's object identifier, three-dimensional coordinates, velocity, and attitude.

[0009] Risk assessment and risk prediction are performed on the quay crane operation area based on the time-series trajectory.

[0010] Control the early warning status based on the results of risk assessment and risk prediction.

[0011] In some embodiments, the preprocessing of the first multi-source data to obtain the second multi-source data includes:

[0012] The first multi-source data is time-synchronized to obtain the third multi-source data.

[0013] The third multi-source data is subjected to noise processing to obtain the fourth multi-source data. The noise processing includes distortion correction, high dynamic range enhancement, or rain and fog removal.

[0014] The region of interest is cropped from the fourth multi-source data to obtain the second multi-source data.

[0015] In some embodiments, the step of performing target object identification and target object state identification on the second multi-source data to obtain a structured data stream includes:

[0016] The second multi-source data is subjected to target object identification to obtain target objects, which include static objects and dynamic objects;

[0017] Identify the state features of the target object in the second multi-source data to obtain state feature information;

[0018] Set the object identifier and attribute labels for the target object;

[0019] A structured data stream is constructed based on the target object, the state feature information, the object identifier, and the attribute label.

[0020] In some embodiments, mapping the structured data to a target coordinate system to form a time-series trajectory includes:

[0021] Based on the target coordinate system, the structured data is unified and calibrated to obtain the data to be located;

[0022] The target object is located and fused in the target coordinate system based on the data to be located, and a fusion result is obtained;

[0023] Based on the fusion result, the motion state of each target object is tracked to obtain the temporal trajectory.

[0024] In some embodiments, the step of performing risk assessment and risk prediction on the quay crane operation area based on the time-series trajectory includes:

[0025] Based on the time-series trajectory, predict the real-time risk score for each frame at future moments;

[0026] If the real-time risk score meets a preset trend for a consecutive preset number of frames, a risk prediction result is generated.

[0027] In some embodiments, the step of predicting the real-time risk score for each frame at a future time based on the time-series trajectory includes:

[0028] Predict the safety gaps, relative speeds, spreader swing angles, and danger zone markers between target objects based on the time-series trajectory;

[0029] Obtain the alignment error of the quay crane operation;

[0030] The real-time risk score for each frame in the future is calculated based on the safety gap, the relative speed, the spreader swing angle, the danger zone marker, and the quay crane operation alignment error.

[0031] In some embodiments, the method further includes the following steps:

[0032] Risk events are generated based on the results of risk assessment and risk prediction.

[0033] The data corresponding to the risk event is saved to the database. The data corresponding to the risk event includes the participants in the risk event, the real-time risk score of the risk event, the image data of the risk event, the video data of the risk event, and the review result of the risk event.

[0034] Another embodiment of the present invention provides an early warning system for safe operation of quay cranes, the system comprising:

[0035] The acquisition module is used to acquire the first multi-source data of the quay crane operation area. The first multi-source data includes status data detected by a preset controller, image data acquired by multiple types of image acquisition devices at multiple angles, or distance data acquired by multiple ranging devices at different locations.

[0036] The preprocessing module is used to preprocess the first multi-source data to obtain the second multi-source data;

[0037] The identification module is used to identify target objects and the state of target objects in the second multi-source data to obtain a structured data stream, wherein each target object in the structured data stream is assigned a unique object identifier.

[0038] The mapping module is used to map the structured data to the target coordinate system to form a time-series trajectory. Each point in the time-series trajectory consists of the object identifier, three-dimensional coordinates, velocity, and attitude of the target object.

[0039] The assessment and prediction module is used to perform risk assessment and risk prediction on the quay crane operation area based on the time-series trajectory.

[0040] The early warning module is used to control the early warning status based on the risk assessment results and risk prediction results.

[0041] Another aspect of the present invention provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the above-described method.

[0042] Another aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0043] The present invention provides the following beneficial effects: This embodiment acquires first multi-source data within the quay crane operation area, including status data detected by a preset controller, image data acquired by multiple image acquisition devices at multiple angles, or distance data acquired by multiple ranging devices at different locations. This provides data support from multiple sources for subsequent risk assessment and early warning processes. Then, after preprocessing the first multi-source data to obtain second multi-source data, target object identification and target object status identification are performed on the second multi-source data to obtain a structured data stream. The structured data is then mapped onto the target coordinate system to form a time-series trajectory. Based on the time-series trajectory, risk assessment and risk prediction are performed on the quay crane operation area. Furthermore, the early warning status is controlled based on the risk assessment and risk prediction results. Thus, risk assessment and early warning control of the quay crane operation area can be performed using data under the same coordinates, effectively improving the accuracy of risk assessment and safety early warning during quay crane operations. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart of an early warning method for safe operation of a quay crane provided in this application;

[0046] Figure 2 This is a schematic diagram of a module of an early warning system for safe operation of a quay crane provided in this application. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0048] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Unless otherwise expressly specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection; a mechanical connection or an electrical connection; a direct connection or an indirect connection through an intermediate medium; or a connection within two elements. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0049] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0050] In related technologies, quay cranes undertake container transfer tasks during ship loading and unloading operations. Their operations are characterized by high operating rates, complex environments, and limited visibility. Currently, monitoring videos during quay crane operations are only used for post-event evidence collection. This prevents relevant personnel from timely understanding the spatial relationships between the spreader, containers, vehicles, and personnel involved in the operation. Furthermore, the signals and visual data from different existing equipment are isolated from each other, resulting in low accuracy in risk assessment and safety warnings.

[0051] In view of this, embodiments of this application provide an early warning method, system, device, and medium for safe operation of quay cranes, which can effectively improve the accuracy of risk assessment and safety early warning during quay crane operations.

[0052] The embodiments of this application will be described in detail below with reference to the accompanying drawings:

[0053] Reference Figure 1 This application provides a schematic diagram of an early warning method for safe operation of a quay crane. Figure 1 The method shown includes, but is not limited to, steps S110 to S160:

[0054] Step S110: Obtain the first multi-source data of the quay crane operation area, wherein the first multi-source data includes status data detected by a preset controller, image data collected by multiple types of image acquisition devices at multiple angles, or distance data collected by multiple ranging devices at different locations.

[0055] Step S120: Preprocess the first multi-source data to obtain the second multi-source data;

[0056] Step S130: Perform target object identification and target object state identification on the second multi-source data to obtain a structured data stream, wherein each target object in the structured data stream is assigned a unique object identifier;

[0057] Step S140: Map the structured data to the target coordinate system to form a time-series trajectory, wherein each point in the time-series trajectory consists of the object identifier, three-dimensional coordinates, velocity and attitude of the target object;

[0058] Step S150: Conduct risk assessment and risk prediction for the quay crane operation area based on the time-series trajectory;

[0059] Step S160: Control the early warning status based on the risk assessment results and risk prediction results.

[0060] It is understood that the preset controller can be a programmable logic controller (PLC), which is used to detect the status change data of various operating equipment within the quay crane operating area, such as the lifting data, locking and unlocking data of the spreader, and the displacement data of the trolley. When the controller detects these status change data, this embodiment synchronously controls multiple image acquisition devices and multiple ranging devices to enter the working state. The image acquisition devices include, but are not limited to, various PTZ cameras and wide-angle cameras, which are used to acquire image data of various sub-areas within the quay crane operating area. The ranging devices can be laser rangefinders, which are used to obtain real-time distance data between various target objects within the quay crane operating area. Specifically, when acquiring status data, image data, and distance data, this embodiment also synchronously acquires the record timestamp, quay crane number, and container berth number corresponding to each data point for subsequent multi-source data alignment operations. In this embodiment, the container berth number is a 7-digit code used to identify the three-dimensional loading position of the container on the ship, consisting of the first 3 digits of the container number + the middle 2 digits of the column number + the last 2 digits of the layer number. This encoding follows a unified container location coordinate system and, in conjunction with ship stowage diagrams and electronic messages, enables unique and precise container location and information exchange during ship-shore collaborative operations. In this embodiment, the first multi-source data may further include real-time wind speed data collected by wind speed sensors within the quay crane operation area and lane data collected by lane sensors within the quay crane operation area. The lane data includes, but is not limited to, the total number of lanes, the remaining total number of lanes, and vehicle data within the lanes.

[0061] It is understood that in this embodiment, after obtaining the first multi-source data, the first multi-source data is preprocessed to obtain the second multi-source data. Specifically, this embodiment can obtain the third multi-source data by performing time synchronization processing on the first multi-source data, obtain the fourth multi-source data by performing noise processing on the third multi-source data, and obtain the second multi-source data by cropping the region of interest (ROI) on the fourth multi-source data. The time synchronization processing in this embodiment can be time synchronization processing on image data and distance data. Noise processing includes, but is not limited to, distortion correction, high dynamic range (HDR) enhancement, or rain / fog removal. Region of interest (ROI) cropping can be done by cropping data such as lifting equipment, lanes, and hatches from the fourth multi-source data. After completing the above processing, this embodiment outputs all the processed data in a unified format, thereby obtaining a "clear frame" sequence as the second multi-source data.

[0062] It is understood that, after obtaining the second multi-source data, this embodiment performs target object identification and target object state identification on the second multi-source data to obtain a structured data stream. Specifically, this embodiment can obtain target objects, including static and dynamic objects, by performing target object identification on the second multi-source data. Then, it can identify the state characteristics of the target objects in the second multi-source data to obtain state characteristic information. At the same time, it sets the object identifier (ID) and attribute tags of the target objects, and constructs a structured data stream based on the target objects, state characteristic information, object identifiers, and attribute tags. Static objects in this embodiment may include, but are not limited to, containers and dangerous goods, while dynamic objects may include, but are not limited to, spreaders, vehicles, and personnel. The static object characteristic information in the state characteristic information includes, but is not limited to, information such as the five-view diagram of the container, container number, container door orientation, seal status, dangerous goods identification, and vehicle roof number. Among them, a seal is a locking device used to close cargo containers, achieving sealing by applying a lock. The function of the seal is to prevent unauthorized opening or tampering, ensuring the integrity and safety of goods during transportation. The dynamic object feature information in the status feature information includes, but is not limited to, information such as the spreader attitude (pitch angle, swing angle), vehicle position, and personnel actions. Attribute tags can be used to characterize the object type, the confidence level of the data recognition result, and the timestamp corresponding to the data. In this embodiment, after completing the above data recognition and setting, the recognized data and the set related data are combined into a structured data stream.

[0063] It is understood that, after obtaining the structured data stream, this embodiment maps all data in the structured data stream to the target coordinate system. For example, data from different cameras, laser rangefinders, or PLCs in the structured data stream can be mapped to the same world coordinate system to achieve real-space positioning of the target object. Specifically, this embodiment can obtain the data to be located by unifying and calibrating the structured data based on the target coordinate system. Then, the target object is located and fused in the target coordinate system based on the data to be located to obtain the fusion result. Finally, the motion state of each target object is tracked according to the fusion result to obtain the temporal trajectory. In this embodiment, for data collected by cameras, this embodiment can obtain the extrinsic parameter matrix φ by offline calibration of each camera. Then, based on the extrinsic parameter matrix φ, the collected image data is mapped to the world coordinate system to obtain image coordinate point data. For distance data obtained by laser rangefinders, this embodiment uses the same target as the camera to map the distance data to the same world coordinate system to obtain the corresponding point cloud data. For the status data acquired by the PLC (such as lifting height and trolley travel), the Z-axis or X-axis of the world coordinate system can be used as a reference and uniformly mapped to the world coordinate system.

[0064] After completing the mapping operations for data from various sources and of various types, this example can locate and merge the target object based on the method shown in Table 1:

[0065] Table 1

[0066] target object Location source Fusion Method lifting gear Laser height + PLC displacement Laser range correction + displacement mapping container Dual-camera parallax + hoist association Triangulation + Constraint Fusion vehicle Side camera + ground laser Plane projection positioning personnel Single camera + projection transformation Keypoint detection + Plane mapping

[0067] Specifically, in this embodiment, when performing weighted fusion of the positioning results for each frame, the positioning results can be weighted and fused based on the confidence level and the geometric constraints within the quay crane operating area. The fusion process is as follows:

[0068] ;

[0069] In the formula, The world coordinate point, representing the three-dimensional coordinates of the estimated target point in the world coordinate system, can be expressed in the following form: ; This represents the weight coefficient corresponding to the i-th camera or the i-th viewpoint, that is, the confidence weight of the i-th camera or the i-th viewpoint to the final result, satisfying the condition... and The confidence weights are derived from image sharpness, viewing angle, depth estimation confidence, and whether the target is viewed directly (rather than obliquely). Let be the extrinsic parameter matrix for the i-th camera or the i-th viewpoint, representing the transformation of the target point from the camera coordinate system to the world coordinate system. The structure of the extrinsic parameter matrix is ​​as follows: , Represents the rotation matrix. Represents the translation vector. Represents a ray / point in the world coordinate system; Let be the intrinsic parameter matrix for the i-th camera or the i-th viewpoint, representing the transformation of pixel coordinates into normalized imaging plane coordinates in the camera coordinate system, where the original intrinsic parameter matrix is... , This represents the direction vector in the camera coordinate system; This represents the pixel coordinates of the target point in the i-th camera or the i-th viewpoint, and its form is homogeneous coordinates. .

[0070] Specifically, the above formula means that the pixel observations of the same target point in multiple cameras are weighted and fused according to their respective credibility after intrinsic reflection and extrinsic coordinate transformation, so as to obtain the final three-dimensional position of the target point in the world coordinate system.

[0071] It is understood that this embodiment can use Kalman filtering and IoU matching to track the motion state of each target object. IoU (Intersection over Union) is a core metric in object detection used to measure the degree of matching between the predicted bounding box and the ground truth bounding box. IoU evaluates the accuracy of object detection by calculating the ratio of the overlap area to the union area of ​​the predicted and ground truth regions. After completing the tracking of the target object's motion state, this embodiment can generate a temporal trajectory based on the tracking process. The output format of the temporal trajectory is shown below:

[0072] ;

[0073] In the formula, ID represents the object identifier of the target object; X(t), Y(t), and Z(t) represent the three-dimensional coordinates of the target object in frame t; V(t) represents the movement speed of the target object in frame t. This represents the pose sequence of the target object in frame t.

[0074] After obtaining the time-series trajectory, this embodiment can perform risk assessment and risk prediction for the quay crane operation area based on the time-series trajectory. Specifically, this embodiment can predict the real-time risk assessment for each frame at future moments based on the time-series trajectory, and generate a risk prediction result when the real-time risk scores for a consecutive preset number of frames meet a preset trend. The risk assessment and risk trend prediction in this embodiment involve real-time analysis of the spatial relationships between all dynamic objects. The risk calculation process involves analyzing distance, speed, and trend to obtain the risk level.

[0075] Understandably, this implementation's real-time risk scoring can predict the safety gap, relative speed, spreader swing angle, and danger zone markers between target objects based on the time-series trajectory. Simultaneously, after acquiring the quay crane operation alignment error, it calculates the real-time risk score R for each frame at future time points based on the safety gap, relative speed, spreader swing angle, danger zone markers, and quay crane operation alignment error. The calculation formula is as follows:

[0076] ;

[0077] In the formula, Indicates safety clearances (e.g., the distance between the spreader and the container, the distance between the spreader and the vehicle, the distance between the spreader and personnel, etc.). Represents relative velocity; Indicates the swing angle of the lifting device; Signs indicating dangerous areas (e.g., signs indicating that people or vehicles are entering dangerous areas); This indicates the positioning error of the quay crane operation; w1, w2, w3, w4 and w5 all represent weighting coefficients, which can be adjusted according to the actual situation of the equipment.

[0078] The numerical values ​​corresponding to the hazard zone signs can be derived from the values ​​corresponding to pre-defined zones. For example, the hazard zone sign for personnel entering zone 1 might be 60 points, for zone 2 it might be 70 points, and for zone 3 it might be 80 points. Quay crane alignment error refers to the permissible deviation between the actual parking or positioning position of the target object (such as a container truck, AGV, or ship hold) and the "target parking / alignment point" required by the quay crane control system during loading and unloading operations by a quay crane. This permissible deviation is typically measured separately in the **longitudinal direction (along the lane / ship bow / stern direction) and the **lateral direction (perpendicular to the lane / ship width direction)**, used to determine whether the safety and accuracy conditions for automatic / semi-automatic operations are met, thereby triggering subsequent actions such as container placement, lifting, or locking.

[0079] It is understood that in this embodiment, after obtaining the real-time risk score for each frame, if the real-time risk scores for a consecutive preset number of frames meet a preset trend, a risk prediction result is generated. For example, if the real-time risk scores for three consecutive frames show an upward trend, a "trend rising" flag is triggered, and the system enters the next risk level warning in advance, thereby preventing delayed warnings for sudden risks. Specifically, the process of classifying risk levels and controlling system actions based on real-time risk scores in this embodiment is shown in Table 2:

[0080] Table 2

[0081] grade scope System Actions Info (Information Prompt) R < 30 Only in VMT prompts Warning 30 ≤ R < 60 Yellow border flashing, sound alert Danger R ≥ 60 Red highlight + voice alarm + operation confirmation

[0082] It is understood that this embodiment controls the warning status after obtaining the risk assessment and risk prediction results. For example, this embodiment can display the warning visually on the VMT (Vehicle Management Terminal), while the central monitoring terminal simultaneously outputs the warning. During the warning process, information is provided via a window (Info), warnings are issued via audio prompts and yellow flashing (Warn), and danger is indicated via a full-screen red alarm, audio, and manual confirmation (Danger). Specifically, this embodiment does not directly control the hardware; it only intervenes in risks through software prompts, thereby ensuring the effective operation of the quay crane.

[0083] It is understood that this embodiment generates risk events after obtaining the risk assessment and risk prediction results, and saves the corresponding data of the risk events to the database. The data corresponding to the risk events includes the participants in the risk event (scrapers, people, vehicle lights), the real-time risk score corresponding to the risk event, the image data corresponding to the risk event, the video data corresponding to the risk event, and the review result corresponding to the risk event (notified, confirmed, etc.). Specifically, in this embodiment, the data in the database is only stored for a preset duration to improve the effective utilization rate of the database.

[0084] It is understood that the early warning review process in this embodiment can involve manually reviewing case events based on their priority, receiving the review results after manual review, and saving the review results to the corresponding location in the database. Specifically, the model corresponding to the early warning method in this embodiment can perform incremental learning based on the review data to update the model parameters, thereby improving the accuracy of the model in subsequent applications.

[0085] For example, when the method of this embodiment is applied to loading operations, vehicles and personnel near the hatch are detected as the spreader descends and approaches. If personnel intrusion or vehicles crossing the boundary are detected, R ≥ 60, triggering a red warning. If the pre-assigned container number on the ship's plan does not match the identification result, the system prompts "pre-assignment abnormality". When the method of this embodiment is applied to unloading operations, dangerous goods markings, container door orientation, and seal status are identified during the spreader's descent from the air. If the spreader lane identification result is inconsistent with the laser positioning, R ≈ 45, triggering a yellow warning. Simultaneously, the driver verifies and confirms before resuming normal operations.

[0086] As can be seen from the above, the method of this application embodiment establishes a unified spatiotemporal coordinate system through multi-camera visual recognition, laser ranging, and PLC signal fusion. It then identifies, locates, tracks, and performs risk analysis on dynamic objects such as spreaders, containers, vehicles, and personnel, thereby achieving graded safety early warning and traceable recording of the operation process and effectively improving the accuracy of risk assessment and safety early warning during quay crane operations.

[0087] Reference Figure 2 This application provides an early warning system for safe operation of quay cranes, the system comprising:

[0088] The acquisition module is used to acquire the first multi-source data of the quay crane operation area. The first multi-source data includes status data detected by a preset controller, image data acquired by multiple types of image acquisition devices at multiple angles, or distance data acquired by multiple ranging devices at different locations.

[0089] The preprocessing module is used to preprocess the first multi-source data to obtain the second multi-source data;

[0090] The identification module is used to identify target objects and their states from the second multi-source data to obtain a structured data stream, wherein each target object in the structured data stream is assigned a unique object identifier.

[0091] The mapping module is used to map structured data to the target coordinate system to form a time-series trajectory, where each point in the time-series trajectory consists of the target object's object identifier, three-dimensional coordinates, velocity, and attitude.

[0092] The assessment and prediction module is used to perform risk assessment and risk prediction on the quay crane operation area based on the time-series trajectory.

[0093] The early warning module is used to control the early warning status based on the risk assessment results and risk prediction results.

[0094] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0095] This application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement... Figure 1 The method shown.

[0096] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0097] This application provides a computer-readable storage medium storing a computer program, which is implemented when executed by a processor. Figure 1 The method shown.

[0098] It is understood that the content of the above method embodiments is applicable to this medium embodiment. The specific functions implemented in this medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical coding feature maps; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for early warning of safe operation of quay cranes, characterized in that, The method includes the following steps: Acquire first multi-source data of the quay crane operation area. The first multi-source data includes status data detected by a preset controller, image data collected by multiple types of image acquisition devices at multiple angles, or distance data collected by multiple ranging devices at different locations. The first multi-source data is preprocessed to obtain the second multi-source data; The second multi-source data is subjected to target object identification and target object state identification to obtain a structured data stream, wherein each target object in the structured data stream is assigned a unique object identifier; The structured data is mapped onto the target coordinate system to form a time-series trajectory. Each point in the time-series trajectory consists of the target object's object identifier, three-dimensional coordinates, velocity, and attitude. Risk assessment and risk prediction are performed on the quay crane operation area based on the time-series trajectory. Control the early warning status based on the results of risk assessment and risk prediction.

2. The early warning method according to claim 1, characterized in that, The preprocessing of the first multi-source data to obtain the second multi-source data includes: The first multi-source data is time-synchronized to obtain the third multi-source data. The third multi-source data is subjected to noise processing to obtain the fourth multi-source data. The noise processing includes distortion correction, high dynamic range enhancement, or rain and fog removal. The region of interest is cropped from the fourth multi-source data to obtain the second multi-source data.

3. The early warning method according to claim 1, characterized in that, The step of performing target object identification and target object state identification on the second multi-source data to obtain a structured data stream includes: The second multi-source data is subjected to target object identification to obtain target objects, which include static objects and dynamic objects; Identify the state features of the target object in the second multi-source data to obtain state feature information; Set the object identifier and attribute labels for the target object; A structured data stream is constructed based on the target object, the state feature information, the object identifier, and the attribute label.

4. The early warning method according to claim 1, characterized in that, The step of mapping the structured data to the target coordinate system to form a time-series trajectory includes: Based on the target coordinate system, the structured data is unified and calibrated to obtain the data to be located; The target object is located and fused in the target coordinate system based on the data to be located, and a fusion result is obtained; Based on the fusion result, the motion state of each target object is tracked to obtain the temporal trajectory.

5. The early warning method according to claim 1, characterized in that, The step of conducting risk assessment and risk prediction for the quay crane operation area based on the time-series trajectory includes: Based on the time-series trajectory, predict the real-time risk score for each frame at future moments; If the real-time risk score meets a preset trend for a consecutive preset number of frames, a risk prediction result is generated.

6. The early warning method according to claim 5, characterized in that, The real-time risk score prediction for each frame at future moments based on the time-series trajectory includes: Predict the safety gaps, relative speeds, spreader swing angles, and danger zone markers between target objects based on the time-series trajectory; Obtain the alignment error of the quay crane operation; The real-time risk score for each frame in the future is calculated based on the safety gap, the relative speed, the spreader swing angle, the danger zone marker, and the quay crane operation alignment error.

7. The early warning method according to claim 1, characterized in that, The method further includes the following steps: Risk events are generated based on the results of risk assessment and risk prediction. The data corresponding to the risk event is saved to the database. The data corresponding to the risk event includes the participants in the risk event, the real-time risk score of the risk event, the image data of the risk event, the video data of the risk event, and the review result of the risk event.

8. An early warning system for safe operation of quay cranes, characterized in that, The system includes: The acquisition module is used to acquire the first multi-source data of the quay crane operation area. The first multi-source data includes status data detected by a preset controller, image data acquired by multiple types of image acquisition devices at multiple angles, or distance data acquired by multiple ranging devices at different locations. The preprocessing module is used to preprocess the first multi-source data to obtain the second multi-source data; The identification module is used to identify target objects and the state of target objects in the second multi-source data to obtain a structured data stream, wherein each target object in the structured data stream is assigned a unique object identifier. The mapping module is used to map the structured data to the target coordinate system to form a time-series trajectory. Each point in the time-series trajectory consists of the object identifier, three-dimensional coordinates, velocity, and attitude of the target object. The assessment and prediction module is used to perform risk assessment and risk prediction on the quay crane operation area based on the time-series trajectory. The early warning module is used to control the early warning status based on the risk assessment results and risk prediction results.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images