Harbor loading and unloading production dynamic closed flow operation system based on deep learning and image recognition algorithm
By applying deep learning and image recognition algorithms to a dynamic closed-process operation system on port machinery, the problems of blind spots and complex equipment scheduling in port machinery have been solved, enabling real-time monitoring, automatic alarms, and intelligent scheduling, thereby improving the safety and efficiency of port operations.
Patent Information
- Application Number
- CN202610128778.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-30
- Publication Date
- 2026-05-19
AI Technical Summary
Port machinery and equipment have large blind spots and require frequent human-machine collaboration during operation, leading to frequent safety accidents. In addition, traditional fencing occupies a large area, has high maintenance costs, and has limited protective effect. Equipment scheduling is also complicated and management efficiency is low.
The port loading and unloading production dynamic closed-loop operation system, based on deep learning and image recognition algorithms, includes a port machinery surrounding environment monitoring and safety control system, a fatigue driving prevention system, and an intelligent scheduling system. It utilizes components such as electronic fences, high-definition night vision cameras, edge computing terminals, status monitoring cameras, behavior recognition modules, and intelligent scheduling systems to achieve real-time monitoring, automatic alarms, and intelligent scheduling, avoid human-machine interaction, prevent fatigue driving, and optimize equipment scheduling.
It significantly improves the safety and management efficiency of port machinery operations at night, reduces the accident rate, increases equipment utilization and operational efficiency, enhances safety protection capabilities and equipment utilization, and reduces the failure rate.
Smart Images
Figure CN122067183A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of port safety management technology, specifically to a dynamic closed-loop operation system for port loading and unloading production based on deep learning and image recognition algorithms. Background Technology
[0002] Port machinery is mainly divided into two categories: quayside machinery (such as gantry cranes, quay cranes, and ship unloaders) and mobile machinery (such as forklifts, loaders, and tractors). Due to the large size of the equipment, wide blind spots, and frequent human-machine collaboration, safety accidents are easily caused. To address this, Weifang Port Bulk Cargo Terminal Co., Ltd. has innovatively applied deep learning and image processing algorithms in the field of port safety management, combining current mainstream technologies. Summary of the Invention
[0003] The purpose of this invention is to provide a dynamic closed-loop operation system for port loading and unloading production based on deep learning and image recognition algorithms, so as to overcome the problems existing in the prior art.
[0004] To achieve the above objectives, the technical solution adopted by this invention is as follows: a dynamic closed-loop operation system for port loading and unloading production based on deep learning and image recognition algorithms, comprising a port machinery surrounding environment monitoring and safety control system, a port mobile machinery anti-fatigue driving system, and an intelligent dispatching system. The port machinery surrounding environment monitoring and safety control system includes an electronic fence, an environmental information acquisition camera, an edge computing terminal, a display device, and an electronic alarm. The edge computing terminal is electrically connected to the environmental information acquisition camera, the display device, and the electronic alarm. The electronic fence is formed in the dynamic closed-loop operation area of the port machinery, and the environmental information acquisition camera is set around the port machinery. The anti-fatigue driving system is installed on the port mobile machinery and includes a status monitoring camera, a behavior recognition module, and an alert module. The behavior recognition module is electrically connected to the status monitoring camera and the alert module. The status monitoring camera is installed in the cab of the port mobile machinery. The intelligent dispatching system is set in the dynamic closed-loop operation area of the port machinery and includes a cargo recognition camera, an edge computing chip, a cargo recognition sensor, a PLC controller, transportation equipment, and an industrial robot. The edge computing chip is electrically connected to the cargo recognition camera, the cargo recognition sensor, and the PLC controller. The PLC controller is electrically connected to the transportation equipment and the industrial robot.
[0005] Based on the above technical solution, the present invention can be further improved as follows: As a further improvement to the above technical solution, the specific implementation of setting up an electronic fence in the dynamic closed operation area of port machinery is to install spotlights at appropriate locations in the port machinery operation area.
[0006] As a further improvement to the above technical solution, the port machinery dynamic closed operation area is specifically divided into the shore machinery dynamic closed operation area and the mobile machinery dynamic closed operation area.
[0007] As a further improvement to the above technical solution, the edge computing device adopts an NVIDIA Xavier NX device, and the environmental information acquisition camera adopts a high-definition night vision camera.
[0008] As a further improvement to the above technical solution, the status monitoring camera is installed in the cab of the port's mobile machinery.
[0009] As a further improvement to the above technical solution, the port machinery surrounding environment monitoring and safety control system also includes high-brightness reflective stickers and / or reflective light strips installed on the port machinery.
[0010] As a further improvement to the above technical solution, the port machinery surrounding environment monitoring and safety control system also includes a lidar installed in the dynamic closed operation area of the port machinery. The lidar is electrically connected to the edge computing terminal, and collects obstacle information through the lidar and sends the sensed obstacle information to the edge computing terminal.
[0011] As a further improvement to the above technical solution, the edge computing chip is set at the edge node of the handling equipment, the transportation equipment is a forklift, and the cargo identification sensor includes a high-resolution vision sensor, LiDAR, ultrasonic sensor and radio frequency identification (RFID) tag.
[0012] The beneficial effects of this invention are: 1. Significantly improved safety and management efficiency of port machinery operations at night: The port machinery surrounding environment monitoring and safety control system achieves real-time monitoring and alert functions by setting up environmental information collection cameras and electronic alarms. The use of high-definition night vision cameras for environmental information collection further enhances the safety of port machinery operations at night. It is estimated that the above-mentioned port machinery surrounding environment monitoring and safety control system will reduce the accident rate of port machinery operations at night by more than 50%, significantly improving the safety and management efficiency of night operations. 2. Improved reliability and efficiency of intrusion prevention control: Traditional physical fences, due to their large footprint, high maintenance costs, and limited protective effect, are insufficient to meet the safety requirements of modern ports. The proposed port machinery surrounding environment monitoring and safety management system utilizes electronic fences, environmental information acquisition cameras, edge computing terminals, and electronic alarms. The edge computing terminal retrieves video data captured by the cameras and analyzes the content in real time. Deep learning object detection algorithms are used to detect the relative positions of key personnel, vehicles, and other obstacles within the footage, and the system performs intersection and union calculations with the dynamically enclosed operating area of the port machinery. When intrusion is detected, the system automatically issues an alarm and takes emergency measures, effectively preventing safety accidents and achieving full coverage monitoring and intelligent response of the operating area, significantly improving the safety protection level of the port operating area. This port machinery surrounding environment monitoring and safety management system is expected to increase the safety protection capability of the port operating area by more than 70%, reducing safety accidents caused by unauthorized personnel entry. 3. Significantly improved port equipment utilization and overall port operation efficiency: Traditional port operations are complex, requiring a high degree of coordination in equipment scheduling and management. Manual management is prone to problems of poor coordination and low efficiency. By setting up an intelligent scheduling system, which integrates cargo recognition cameras, edge computing chips, cargo recognition sensors, PLC controllers, transportation equipment, and industrial robots, the intelligent scheduling system uses big data and AI algorithms to optimize equipment scheduling and resource allocation, automatically completing cargo handling, equipment operation, and operation monitoring, reducing manual intervention. All aspects of port operations will achieve automated and intelligent management, significantly improving equipment utilization and overall port operation efficiency. It is estimated that this intelligent scheduling system will increase port operation efficiency by more than 40%, improve equipment utilization by 30%, and reduce the failure rate by 50%. Attached Figure Description
[0013] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0014] Figure 1 This is a system structure block diagram of a port loading and unloading dynamic closed-loop process operation system based on deep learning and image recognition algorithms provided in a preferred embodiment of the present invention. The diagram shows: 1. Port machinery surrounding environment monitoring and safety control system; 11. Electronic fence; 12. Environmental information acquisition camera; 13. Edge computing terminal; 14. Display device; 15. Electronic alarm; 16. Reflective sticker; 17. Reflective light strip; 18. LiDAR; 2. Anti-fatigue driving system; 21. Status monitoring camera; 22. Behavior recognition module; 23. Reminder module; 3. Intelligent dispatching system; 31. Cargo recognition camera; 32. Edge computing chip; 33. Cargo recognition sensor; 34. PLC controller; 35. Transportation equipment; 36. Industrial robot. Detailed Implementation
[0015] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. These drawings are simplified schematic diagrams, which are only used to illustrate the basic structure of the present invention and therefore only show the components relevant to the present invention.
[0016] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.
[0017] like Figure 1As shown, a preferred embodiment of the present invention provides a dynamic closed-loop operation system for port loading and unloading production based on deep learning and image recognition algorithms. The system includes a port machinery surrounding environment monitoring and safety control system 1, a port mobile machinery anti-fatigue driving system 2, and an intelligent dispatching system 3. The port machinery surrounding environment monitoring and safety control system 1 includes an electronic fence 11, an environmental information acquisition camera 12, an edge computing terminal 13, a display device 14, and an electronic alarm 15. The edge computing terminal 13 is electrically connected to the environmental information acquisition camera 12, the display device 14, and the electronic alarm 15. The electronic fence 11 is formed in the dynamic closed-loop operation area of the port machinery, and the environmental information acquisition camera 12 is installed around the port machinery. The anti-fatigue driving system 2... The driving system 2 is installed on the port mobile machinery and includes a status monitoring camera 21, a behavior recognition module 22, and an alert module 23. The behavior recognition module 22 is electrically connected to the status monitoring camera 21 and the alert module 23. The status monitoring camera 21 is installed in the driver's cab of the port mobile machinery. The intelligent dispatching system 3 is set in the dynamic closed operation area of the port machinery and includes a cargo recognition camera 31, an edge computing chip 32, a cargo recognition sensor 33, a PLC controller 34, a transportation equipment 35, and an industrial robot 36. The edge computing chip 32 is electrically connected to the cargo recognition camera 31, the cargo recognition sensor 33, and the PLC controller 34. The PLC controller 34 is electrically connected to the transportation equipment 35 and the industrial robot 36.
[0018] By setting up electronic fences 11 within the dynamic enclosed operation area of port machinery, the operating area is demarcated, reminding workers to stay outside the designated area to avoid accidentally entering dangerous zones and preventing accidents. Specifically, the electronic fences 11 are implemented by installing spotlights at appropriate locations within the port machinery operating area, replacing traditional physical fences. The electronic fences 11 can be adjusted in real-time according to the movement of the port machinery, reducing the personnel required for adjustments to the operating area. The dynamic enclosed operation area for port machinery is specifically divided into a shore-side machinery dynamic enclosed operation area and a mobile machinery dynamic enclosed operation area.
[0019] Specifically, devices such as NVIDIA Xavier NX can be used as edge computing devices, and the environmental information acquisition camera 12 can be a high-definition night vision camera. The video data captured by the environmental information acquisition camera 12 around the port machinery can be retrieved through the IP address and the content of the image can be analyzed in real time. The relative position of key personnel, vehicles and other obstacles in the image can be detected by using deep learning object detection algorithms, and the intersection and union of the two areas can be calculated with the dynamic closed operation area of the port machinery. When the two areas overlap to a set threshold, it is determined that there is an abnormal intrusion in the operation area, and the control signal is output to trigger the electronic alarm 15, reminding the intruder or machinery to stay away from the dynamic closed operation area of the port machinery. At the same time, the machinery operator is reminded to take measures to deal with the abnormal intrusion behavior, realizing a dual adjustment mode of active reminder and passive avoidance, eliminating human-machine cross-operation, and improving the level of on-site safety management.
[0020] Specifically, by installing a status monitoring camera 21 at a suitable location in the cab of the port mobile machinery, the behavior recognition module 22 collects and analyzes the driver's facial data from the status monitoring camera 21 in real time, and identifies the driver's video information through deep learning and image recognition algorithms; when the driver repeatedly closes his eyes and yawns within a certain period of time, it is determined that he is in a state of fatigue driving, triggering the reminder module 23 on the port mobile machinery to vibrate and broadcast a voice reminder to the driver to stop and rest, and automatically pushes the information to the site administrator's mobile phone; the site administrator ensures that the driver gets enough rest through personnel scheduling and rotation, while minimizing the impact on the on-site production operation process.
[0021] Preferably, the port machinery surrounding environment monitoring and safety control system 1 also includes high-brightness reflective stickers 16 and / or reflective light strips 17 installed on the port machinery. Traditional night operations have high safety hazards and management difficulties due to poor visibility and bad weather conditions. By installing high-brightness reflective stickers 16 and / or reflective light strips 17 on the port machinery, clear identification of night workers can be achieved, reducing the risk of misjudgment and collision.
[0022] Preferably, the port machinery surrounding environment monitoring and safety control system 1 further includes a lidar 18 installed in the dynamic closed operation area of the port machinery. The lidar 18 is electrically connected to the edge computing terminal 13. The lidar 18 collects obstacle information and sends the sensed obstacle information to the edge computing terminal 13. The edge computing terminal 13 determines that there is an abnormal intrusion in the operation area, outputs a control signal to trigger the electronic alarm 15, reminds the intruder or machinery to stay away from the dynamic closed operation area of the port machinery, and reminds the machinery operator to take measures to deal with the abnormal intrusion.
[0023] Specifically, the aforementioned intelligent scheduling system 3 is equipped with the cargo recognition camera 31, edge computing chip 32, cargo recognition sensor 33, PLC controller 34, transportation equipment 35, and industrial robot 36. Through the deep integration of deep learning and image recognition algorithms, it realizes closed-loop management from cargo recognition to full-process automation, automatically completing cargo handling, equipment operation, and operation monitoring, reducing manual intervention, and improving the reliability and efficiency of operations. Edge computing chip 32 is set at the edge node of the handling equipment. It processes data from cargo recognition camera 31 and cargo recognition sensor 33 in real time through deep learning and image recognition algorithms. The obstacle avoidance response time is less than 0.3 seconds, and it supports seamless switching between indoor and outdoor scenes. The transportation equipment 35 can be a forklift. The cargo recognition sensor 33 can include multiple types of sensors such as high-resolution vision sensors, LiDAR 18, ultrasonic sensors, and radio frequency identification (RFID) tags. The cargo recognition camera 31, combined with 3D laser SLAM technology, can build a three-dimensional point cloud map of the environment in real time with a positioning accuracy of ±10mm. It can dynamically identify pallet posture, cargo stacking status, and obstacles (such as 5cm low obstacles). The deep learning algorithm supports automatic adaptation of multiple types of pallets, and the fork entry error is controlled within ±2mm, achieving millimeter-level precise loading and unloading.
[0024] The specific implementation method of the aforementioned port loading and unloading dynamic closed-loop process operation system includes the following steps: Step 1: Set up an electronic fence 11 in the dynamic closed operation area of port machinery. By setting up the electronic fence 11, the operation area of port machinery is delineated, reminding workers to stay outside the operation area to avoid accidentally entering the dangerous area and prevent safety accidents. The specific implementation method is to install spotlights at appropriate locations in the operation area of port machinery. Step 2: Using NVIDIA Xavier NX as an edge computing device, video data captured by the port machinery's surrounding environment information acquisition camera 12 is retrieved via IP address and analyzed in real time. A deep learning object detection algorithm is used to detect the relative positions of key personnel, vehicles, and obstacles within the image, and the intersection and union of these positions with the port machinery's dynamic closed operating area is calculated. When the two areas overlap to a set threshold, it is determined that there is an abnormal intrusion into the operating area. A control signal is output to trigger the electronic alarm 15, reminding the intruder or port machinery to move away from the port machinery's dynamic closed operating area. At the same time, the machinery operator is reminded to take measures to deal with the abnormal intrusion. Step 3: Install the anti-fatigue driving system 2 on the port's mobile machinery. This is achieved by installing a status monitoring camera 21 at a suitable location in the cab. The behavior recognition module 22 collects and analyzes the driver's facial data from the status monitoring camera 21 in real time. Deep learning and image recognition algorithms are used to identify the driver's video information. When the driver repeatedly closes his eyes and yawns within a certain period, it is determined that he is in a state of fatigue driving. This triggers the reminder module 23 on the port's mobile machinery to vibrate and broadcast a voice reminder to the driver to stop and rest, and automatically pushes the information to the site administrator's mobile phone. The site administrator ensures that the driver gets sufficient rest through personnel scheduling and rotation measures, while minimizing disruption to the on-site production operation process. Step 4: An intelligent dispatch system 3 is set up in the dynamic closed operation area of port machinery. The intelligent dispatch system 3 is equipped with a cargo recognition camera 31, an edge computing chip 32, a cargo recognition sensor 33, a PLC controller 34, transportation equipment 35, and an industrial robot 36. Through the deep integration of deep learning and image recognition algorithms, a closed-loop management system from cargo recognition to full-process automation is realized, which automatically completes cargo handling, equipment operation, and operation monitoring, reduces manual intervention, and improves the reliability and efficiency of operations.
[0025] Weifang Port Bulk Cargo Terminal Co., Ltd., combining current mainstream deep learning and image processing technologies, has innovatively applied deep learning and image recognition algorithms in the field of port safety management. This invention, through a series of technological innovations and optimization measures, significantly improves key technical indicators and overall performance of port loading and unloading operations. These improvements are not only reflected in the technical performance of equipment and systems but will also bring significant economic and social benefits.
[0026] Weifang Port Bulk Cargo Terminal Co., Ltd. has achieved remarkable results in practical application of a series of innovative measures for port on-site production safety operation management. These measures have been actively promoted and participated in provincial and municipal employee innovation and efficiency improvement activities, earning numerous honors. The "Dynamic Closed-Loop Operation Method for Port Loading and Unloading" won the Silver Award in the 20th Shandong Provincial Youth Vocational Skills Competition's "Elite Talent Strengthens Shandong" Innovation and Efficiency Improvement Special Competition. The "Anti-Fatigue Driving System for Dump Trucks and Flatbed Trucks" received the title of Excellent Achievement in the Weifang City 2024 Fourth Quarter Safety Production "Golden Ideas" Rationalization Proposal Collection Activity. The main algorithm breakthroughs were published in the journal *IEEE ACCESS* under the title "SSCD-YOLO Semi-Supervised Cross-Domain YOLOv8 for Pedestrian Detection in Low-light Conditions," and two national invention patents, three invention patents, and one software copyright have been applied for.
[0027] Any descriptions not covered in the above specific embodiments of the present invention are known technologies in the field and can be implemented with reference to such known technologies.
[0028] Based on the preferred embodiments of the present invention described above, those skilled in the art can make various changes and modifications without departing from the inventive concept. The technical scope of this invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.
Claims
1. A dynamic closed-loop operation system for port loading and unloading production based on deep learning and image recognition algorithms, characterized in that: The system includes a port machinery surrounding environment monitoring and safety control system, a port mobile machinery anti-fatigue driving system, and an intelligent dispatching system. The port machinery surrounding environment monitoring and safety control system includes an electronic fence, environmental information acquisition cameras, an edge computing terminal, a display device, and an electronic alarm. The edge computing terminal is electrically connected to the environmental information acquisition cameras, display device, and electronic alarm. The electronic fence is formed within the dynamic closed operating area of the port machinery, and the environmental information acquisition cameras are positioned around the port machinery. The anti-fatigue driving system is installed on the port mobile machinery and includes a status monitoring camera, a behavior recognition module, and an alert module. The behavior recognition module is electrically connected to the status monitoring camera and alert module, and the status monitoring camera is installed in the driver's cab of the port mobile machinery. The intelligent dispatching system is located within the dynamic closed operating area of the port machinery and includes a cargo recognition camera, an edge computing chip, a cargo recognition sensor, a PLC controller, transportation equipment, and an industrial robot. The edge computing chip is electrically connected to the cargo recognition camera, cargo recognition sensor, and PLC controller, and the PLC controller is electrically connected to the transportation equipment and industrial robot.
2. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 1, characterized in that: The specific implementation method of setting up an electronic fence in the dynamic closed operation area of port machinery is to install spotlights at appropriate locations in the port machinery operation area.
3. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 1, characterized in that: The port machinery dynamic closed operation area is specifically divided into the shore machinery dynamic closed operation area and the mobile machinery dynamic closed operation area.
4. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 1, characterized in that: The edge computing device uses an NVIDIA Xavier NX device, and the environmental information acquisition camera uses a high-definition night vision camera.
5. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 1, characterized in that: The status monitoring camera is installed in the driver's cab of the port's mobile machinery.
6. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 1, characterized in that: The port machinery surrounding environment monitoring and safety control system also includes high-brightness reflective stickers and / or reflective light strips installed on the port machinery.
7. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 6, characterized in that: The port machinery surrounding environment monitoring and safety control system also includes a lidar installed in the dynamic closed operation area of the port machinery. The lidar is electrically connected to the edge computing terminal, and collects obstacle information through the lidar and sends the sensed obstacle information to the edge computing terminal.
8. The port loading and unloading dynamic closed-loop operation system based on deep learning and image recognition algorithms according to claim 1, characterized in that: The edge computing chip is set at the edge node of the handling equipment, which is a forklift. The cargo identification sensors include a high-resolution vision sensor, a lidar, an ultrasonic sensor, and a radio frequency identification (RFID) tag.