Port automatic driving control management digital twin system
Through the port autonomous driving control management digital twin system, the autonomous driving digital twin platform, intelligent scheduling platform and multi-intelligent unmanned driving technology are used to solve the problems of high manual dependence and low efficiency in port container transportation, and remote control and supervision are realized. Improve operational efficiency and safety, and reduce operating costs and accident rates.
Patent Information
- Application Number
- CN202411162096.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-19
AI Technical Summary
There is high manual dependence, low efficiency, lack of data management and automation control in port container transportation, and it is impossible to achieve seamless AI automation and hardware interconnection, resulting in low operating efficiency and safety risks.
Design a digital twin system for port autonomous driving control management, including a digital twin platform for autonomous driving, intelligent scheduling platform, multi-intelligent unmanned driving platform, IGV unmanned driving system and scene reproduction platform to realize remote control, supervision and scheduling, through manual Intelligent, V2X technology and multi-agent technology carry out vehicle collaborative control and data interaction, combining simulation systems and cloud computing services to realize environmental perception, path planning and abnormal event detection.
It reduces manual operation costs, improves work efficiency, realizes remote control, supervision and scheduling, ensures smooth and safe port operations, reduces cloud computing pressure, enhances system stability, and reduces operating costs and accident rates.
Smart Images

Figure CN118981903A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of autonomous driving technology, and in particular to a digital twin system for port autonomous driving control and management. Background Art
[0002] As a hot frontier in recent years, autonomous driving has attracted a series of giants to invest and develop. However, so far, due to issues such as reliability and technical adaptability to complex road conditions, there are few truly commercial autonomous driving applications. According to industry analysis, the first stop for the commercialization of unmanned driving must be: low-speed, closed scenes, small-scale rigid demand, suitable for the needs of ports, especially the closed scene of port container transportation; At present, the container transport in ports has the following disadvantages: 1. The daily throughput is huge, loading and unloading is non-stop for 24 hours, relying heavily on manual labor, and the operating efficiency is low; 2. Lack of data management and support, duplication of work in data entry at each stage, and inability to automatically update information flow; 3. There are third-party semi-automated transport and loading and unloading vehicles, but they cannot be seamlessly connected to the port information system and achieve AI automated control; 4. There are third-party autonomous driving platforms, but they cannot be interconnected and integrated with vehicles at the hardware level; Therefore, the present invention proposes a port autonomous driving control and management digital twin system to solve the problems existing in the prior art. Summary of the invention
[0003] In response to the above problems, the present invention proposes a digital twin system for port autonomous driving control and management, which can achieve the effects of remote control, remote supervision and remote scheduling. It not only reduces manual operation costs and improves work efficiency, but also provides the port with the latest policy and regulatory data applications, ensuring smooth and safe daily work.
[0004] To achieve the purpose of the present invention, the present invention is implemented through the following technical solutions: a port autonomous driving control and management digital twin system, including an autonomous driving digital twin platform, an intelligent scheduling platform, a multi-agent unmanned driving platform, an IGV unmanned driving system and a scene reproduction platform. The autonomous driving digital twin platform is used to monitor and control the unmanned vehicle team in the production environment in real time, and establish a digital twin environment of the yard terminal to simulate various scenarios and vehicle parameter changes; the intelligent scheduling platform is based on the TOS terminal business operating system, and uses artificial intelligence to analyze the terminal situation in real time, and schedules and coordinates the instructions of the TOS terminal business operating system, and coordinates the production operations of each vehicle in the scheduling area; the multi-agent unmanned driving platform is based on the support of the vehicle-road cloud platform, integrates all IGV vehicles, cooperates with each other and interacts with perception data, and simultaneously performs environmental perception positioning, decision planning and execution control on IGV vehicles, and relies on V2X technology to interact information between the vehicle and the outside world; The IGV unmanned driving system is built into the IGV vehicle, providing three operating modes: automatic driving, remote control, and manual control, and realizing positioning, automatic perception, and obstacle avoidance functions. It simultaneously combines the TOS terminal business operating system and ECS cloud computing services to perform path planning, as well as lane keeping, automatic following, lane borrowing judgment, and loading and unloading parking judgment functions; the scene reproduction platform reconstructs the three-dimensional road scene based on a fixed-pose monocular camera and a laser point cloud, estimates the comprehensive posture of vehicles and pedestrians, and tracks continuous trajectories. At the same time, it establishes time domain and spatial domain correlation models and abnormal event cost function models under complex road scenes to detect and warn abnormal events.
[0005] Further improvements are as follows: the autonomous driving digital twin platform is a combination of management software and simulation system. The management software is used to monitor and control the unmanned vehicle team in the production environment in real time. The simulation system is used to establish a digital twin environment of the yard terminal and simulate various realistic scenarios and vehicle parameter changes.
[0006] Further improvements are: the TOS terminal business operating system is used to guide the production plan of the terminal organization, record and store key data in the terminal operation process, allocate resources, and provide data statistics and information analysis for terminal production decisions. The intelligent scheduling platform is a supplement to the TOS terminal business operating system.
[0007] Further improvements are: the multi-agent unmanned driving platform adopts multi-agent technology, which is built into each IGV vehicle, driving each IGV vehicle to communicate with other IGV vehicles, and all IGV vehicles cooperate to complete complex tasks. At the same time, based on the support of the vehicle-road cloud platform, all IGV vehicles are integrated into one, and each IGV vehicle is perceived and linked with other IGV vehicles and roadside equipment to identify the entire scene.
[0008] Further improvements are as follows: the environmental perception and positioning is to obtain environmental indicators during vehicle driving through sensor technology, camera, and GPS technology, including lane line detection, traffic light recognition, traffic sign recognition, pedestrian detection, vehicle detection, obstacle recognition, and vehicle positioning, collect relevant data, unify them in the vehicle coordinate system, and establish metadata with time-stamped data association and fusion; the decision-making planning is to make judgments and guidance on the next behavior of the vehicle through the collected relevant data, including routing, traffic prediction, behavior decision, action planning, and feedback control signal output; the execution control is to use the wire-controlled actuator to complete the execution of the feedback control output instruction based on the vehicle dynamics system model, control the vehicle drive, throttle, brake, and steering, and make the vehicle change along the established driving trajectory; the V2X technology provides real-time communication services for information exchange between the vehicle and the outside world, and serves environmental perception and positioning and decision-making planning.
[0009] Further improvements are as follows: in the IGV unmanned driving system, when docking with the aerial rail, quay crane, rail crane, and yard crane, positioning introduces CPS information and differential GNSS+IMU+high-precision map positioning RSU; automatic perception is multi-sensor fusion perception, including lidar, ultrasonic radar, vehicle-mounted camera, RSU, with a single vehicle effective perception distance of not less than 100 meters of movement distance, realizing the perception of dynamic obstacles, static obstacles, lane lines, signs, and traffic signals; obstacle avoidance adopts automatic perception and segmentation technology based on artificial intelligence, adapted to a ground slope of 5%.
[0010] Further improvements are as follows: Path planning: tasks are jointly planned and issued by the cloud, and global path planning is performed, while local path planning is performed by the on-board equipment; Lane keeping function: the on-board camera, lidar and high-precision map are used to obtain the road conditions of the lane ahead, so that the IGV vehicle automatically maintains the current lane and achieves parameterized maximum speed; Automatic following function: the minimum following distance during driving is controlled to be no less than the safe distance; Borrowing lane judgment function: the road conditions of the borrowing lane are judged, the minimum overtaking distance is controlled to be no less than 5m, and the vehicle merges into the borrowing lane at a safe distance; Loading and unloading parking judgment function: supports single large box and single small box modes. When the CPS inputs information as positioning information, the parking position is determined according to the TOS input box area, bay position information and loading and unloading mode information. The optimal deceleration distance is calculated based on the parking position and vehicle weight, and the posture is fine-tuned at the end to achieve parking, loading and unloading.
[0011] Further improvements are: reconstruction of three-dimensional road scenes based on a fixed-pose monocular camera and laser point clouds: parsing the road scene relationship through the inverse projection matrix of the monocular camera, identifying the outline and three-dimensional size of the target object through the laser point cloud, obtaining a complete three-dimensional spatial dynamic structure diagram of the complex road scene, and transmitting it to the industrial computer processing unit in real time to achieve adaptive perception of intersection scenes.
[0012] Further improvements are: Comprehensive posture estimation and continuous trajectory tracking of vehicles and pedestrians: Under the constraints of high-dynamic scenes, we can design a deep learning model for real-time tracking to cope with various motion forms of vehicles and pedestrians, such as stationary, constant speed, variable speed, and turning. Based on multi-scene in-loop and actual road tests, we build a convolutional neural network architecture for feature extraction, combined with vehicle features and size prior knowledge, to obtain dynamic target tracking and positioning. In summary, a continuous trajectory tracking model for vehicles and pedestrians is established to perceive intersection scenes.
[0013] Further improvements are as follows: establishing time-domain and spatial-domain correlation models and abnormal event cost function models under complex road scenarios: collecting road time-domain and spatial-domain scene images, modeling based on the background extracted from the images, the three-dimensional position and size of dynamic objects, speed and direction, and multiple dimensions; establishing parameterizable abnormal event judgment indicators, setting the corresponding model relationship between the independent variable and the dependent variable of the indicator, constructing an abnormal event cost function, adaptively configuring the warning trigger threshold, and realizing timely detection and warning of abnormal events.
[0014] The beneficial effects of the present invention are: 1. The present invention is a full-function management platform, which replaces the original manual driving with platform management, and meets the four core technologies of autonomous driving: environmental perception, precise positioning, decision-making planning, and vehicle control, to achieve the effects of remote control, remote supervision, and remote dispatch. It not only reduces the cost of manual operation and improves work efficiency, but also provides the port with the latest policy and regulatory data applications, ensuring smooth and safe daily work.
[0015] 2. The present invention uses a multi-agent unmanned driving platform to integrate all IGV vehicles, so that they can cooperate with each other and interact with each other in terms of perception data, thereby decentralizing the computing and data transmission pressures of the cloud computing center within the framework of the multi-agent system. The damage and downtime of any single device will not cause the paralysis of the entire system, making it more stable. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 It is a composition diagram of the present invention; Figure 2 It is a schematic diagram of the environment perception transmission of the present invention; Figure 3 Schematic diagram of the V2X technology of the present invention. DETAILED DESCRIPTION
[0017] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention. Example
[0018] according to Figure 1 , 2 As shown in Figure 3, this embodiment proposes a port autonomous driving control and management digital twin system, including an autonomous driving digital twin platform, an intelligent scheduling platform, a multi-agent unmanned driving platform, an IGV unmanned driving system and a scene reproduction platform. The autonomous driving digital twin platform is used to monitor and control the unmanned driving vehicle team in the production environment in real time, and establish a digital twin environment of the yard terminal to simulate various scenarios and vehicle parameter changes; the intelligent scheduling platform is based on the TOS terminal business operating system, and uses artificial intelligence to analyze the terminal situation in real time, and schedules and coordinates the instructions of the TOS terminal business operating system, and coordinates the production operations of each vehicle in the scheduling area; the multi-agent unmanned driving platform is based on the support of the vehicle-road cloud platform, integrates all IGV vehicles, cooperates with each other and interacts with perception data, and simultaneously performs environmental perception positioning, decision-making planning and execution control on the IGV vehicle, and relies on V2X technology to interact information between the vehicle and the outside world; The IGV unmanned driving system is built into the IGV vehicle, providing three operating modes: automatic driving, remote control, and manual control, and realizing positioning, automatic perception, and obstacle avoidance functions. It simultaneously combines the TOS terminal business operating system and ECS cloud computing services to perform path planning, as well as lane keeping, automatic following, lane borrowing judgment, and loading and unloading parking judgment functions; the scene reproduction platform reconstructs the three-dimensional road scene based on a fixed-pose monocular camera and a laser point cloud, estimates the comprehensive posture of vehicles and pedestrians, and tracks continuous trajectories. At the same time, it establishes time domain and spatial domain correlation models and abnormal event cost function models under complex road scenes to detect and warn abnormal events. Using the most advanced cloud software technology, we have designed, developed and implemented a full-featured cloud management platform. In the form of application software services (SaaS), through standard public clouds or private clouds, we have replaced the original manual driving with platform management, meeting the four core technologies of autonomous driving (environmental perception, precise positioning, decision-making planning, and vehicle control), achieving the effects of remote control, remote supervision, and remote scheduling. This not only reduces manual operation costs and improves work efficiency, but also provides the port with the latest policy and regulatory data applications, ensuring smooth and safe daily work.
[0019] The autonomous driving digital twin platform is a combination of management software and simulation system. The management software is used to monitor and control the unmanned vehicle team in the production environment in real time. The simulation system is used to establish a digital twin environment of the yard terminal and simulate various realistic scenarios and vehicle parameter changes.
[0020] The TOS terminal business operating system is used to guide the production plan of the terminal organization, record and store key data in the terminal operation process, allocate resources, and provide data statistics and information analysis for terminal production decisions. The intelligent scheduling platform is a supplement to the TOS terminal business operating system.
[0021] The multi-agent unmanned driving platform adopts multi-agent technology and is built into each IGV vehicle, driving each IGV vehicle to communicate with other IGV vehicles, and all IGV vehicles cooperate to complete complex tasks. At the same time, based on the support of the vehicle-road cloud platform, all IGV vehicles are integrated into one, and each IGV vehicle perceives and interacts with other IGV vehicles and roadside equipment to identify the entire scene. The environmental perception and positioning is to obtain environmental indicators during vehicle driving through sensor technology, camera and GPS technology, including lane line detection, traffic light recognition, traffic sign recognition, pedestrian detection, vehicle detection, obstacle recognition and vehicle positioning, collect relevant data, unify them in the vehicle coordinate system, and establish metadata with time-stamped data association and fusion; the decision-making planning is to make judgments and guidance on the next behavior of the vehicle through the collected relevant data, including routing, traffic prediction, behavior decision, action planning and feedback control signal output; the execution control is to use the wire control actuator to complete the execution of feedback control output instructions based on the vehicle dynamics system model, control the vehicle drive, throttle, brake and steering, so that the vehicle changes according to the established driving trajectory; the V2X technology provides real-time communication services for information exchange between the vehicle and the outside world, and serves environmental perception and positioning and decision-making planning.
[0022] Multi-agent technology relies on the multi-agent system (MAS), which mainly refers to a system composed of multiple agents. It mainly decomposes large and complex systems into small and easy-to-manage systems that communicate and coordinate with each other (similar to the node in ROS (Robot Operating System)). Multi-agent systems have been rapidly applied in various fields such as robots, traffic control, autonomous driving, and games. At present, it has become a tool for analyzing and modeling complex systems. Multi-agent systems evolved from distributed artificial intelligence, and their research purpose is to solve large-scale complex real-world problems. In real-world problems, the decision-making ability of a single agent is far from enough. When using a centralized agent to solve problems, it will encounter various resource and condition limitations, resulting in a single agent being unable to cope with the complex real-world environment. Using multiple agents to collaborate with each other can solve many problems. It is currently mainly divided into several types, such as full cooperation, full competition, and coexistence of cooperation and competition. It has the following advantages: decentralization. In the framework of the multi-agent system, the use of the DDS (data distribution) concept similar to the ROS system (Robot Operating System) greatly reduces the computing pressure and data transmission pressure of the cloud computing center. System stability: A multi-agent system is composed of several agents. The damage or downtime of any single device will not cause the entire system to fail. The damaged equipment can be automatically or manually replaced or repaired to maintain the system operation. Compared with swarm intelligence systems or manual control systems, the downtime can even be infinitely close to 0, which is an obvious advantage.
[0023] In the IGV unmanned driving system, when docking with the aerial rail, quay crane, rail crane, and yard crane, positioning introduces CPS information and differential GNSS+IMU+high-precision map positioning RSU; automatic perception is multi-sensor fusion perception, including lidar, ultrasonic radar, vehicle-mounted camera, RSU, with a single vehicle effective perception distance of not less than 100 meters of movement distance, realizing the perception of dynamic obstacles, static obstacles, lane lines, signs, and traffic signals; obstacle avoidance adopts automatic perception and segmentation technology based on artificial intelligence, adapted to a ground slope of 5%. Path planning: The cloud jointly plans and issues tasks and performs global path planning, while the on-board equipment performs local path planning; Lane keeping function: Use on-board cameras, lidar and high-precision maps to obtain the road conditions of the lane ahead, so that the IGV vehicle automatically maintains the current lane and achieves parameterized maximum speed; Automatic following function: Control the minimum following distance during driving to be no less than the safe distance; Borrowing lane judgment function: Judge the road conditions of the borrowing lane, control the minimum overtaking distance to be no less than 5m, and merge into the borrowing lane at a safe distance; Loading and unloading parking judgment function: Supports single large box and single small box modes. When the CPS inputs information as positioning information, the parking position is determined according to the TOS input box area, bay position information and loading and unloading mode information. According to the parking position and vehicle weight, the optimal deceleration distance is calculated, and finally the posture is fine-tuned to achieve parking, loading and unloading.
[0024] Reconstruction of three-dimensional road scenes based on fixed-pose monocular cameras and laser point clouds: The road scene relationship is analyzed through the inverse projection matrix of the monocular camera, and the contour and three-dimensional size of the target object are identified through the laser point cloud to obtain a complete three-dimensional spatial dynamic structure diagram of the complex road scene, and transmit it to the industrial computer processing unit in real time to achieve adaptive perception of intersection scenes. Comprehensive posture estimation and continuous trajectory tracking of vehicles and pedestrians: Under the constraints of high-dynamic scenes, in response to various motion forms of vehicles and pedestrians such as static, constant speed, variable speed, and turning, a real-time tracking deep learning model is designed. Based on multi-scene in-loop and actual road testing, a convolutional neural network architecture for feature extraction is constructed, and dynamic target tracking and positioning are obtained by combining vehicle features and size prior knowledge. In summary, a continuous trajectory tracking model for vehicles and pedestrians is established to perceive intersection scenes. Establish the time domain and space domain correlation model and abnormal event cost function model in complex road scenes: collect road time domain and space domain scene images, and model them according to the background extracted from the images, the three-dimensional position and size of dynamic objects, speed and direction, etc., establish parameterizable abnormal event judgment indicators, set the corresponding model relationship between the independent variable and the dependent variable of the indicator, construct the abnormal event cost function, adaptively configure the warning trigger threshold, and realize timely detection and warning of abnormal events.
[0025] Specifically, it includes the following applications: Port AIV autonomous driving control and management digital twin platform: realizing digital twins of terminal environment, vehicles, equipment, etc., real-time monitoring and control of the entire unmanned vehicle system in the production environment, and forming a simulation environment to provide a platform for testing the port system.
[0026] Port AIV unmanned driving solution: realize fully automatic unmanned driving of port AIV transport vehicles in container terminal yards.
[0027] Port vehicle-road-cloud collaborative platform: combines the intelligent Internet of Things (AIOT), positioning system, port roadside sensing equipment (RSU) and the sensing equipment of the port unmanned loading vehicle to form an expansion of the unmanned driving system of the port transport vehicle itself, so that the unmanned vehicle can grasp the comprehensive information in the area without blind spots or dead ends.
[0028] Port autonomous driving data center: Using the latest data center technology, external data and internal data, real data and simulated data are stored and used in a unified manner, thereby improving efficiency.
[0029] Port multi-agent unmanned driving collaborative platform: Based on multi-agent technology (MARL), each unmanned transport vehicle and equipment in the port area is AI-controlled, and can coordinate and cooperate with each other to maximize the efficiency of yard traffic management. Example
[0030] according to Figure 1 , 2 As shown in , 3, this embodiment proposes a port automatic driving control management digital twin system, including the following: Autonomous driving digital twin platform In actual use, it can be used as a set of management software to monitor and control the entire unmanned vehicle team in the production environment in real time. It can also be used as a simulation system to establish a digital twin environment of a yard terminal. On the simulation platform, various realistic scenarios and vehicle parameter changes can be realized, and the system can be continuously optimized to achieve the use goal.
[0031] Intelligent dispatching system TOS terminal business operating system is an important "brain" of port operations, guiding the terminal to organize efficient and reasonable production plans, accurately recording and storing key data in the terminal operation process in real time, scientifically and effectively allocating resources, and providing complete data statistics and information analysis for terminal production decisions. To do a good job of port autonomous driving, it is necessary to deeply understand the port's operating process, and it is particularly necessary to achieve seamless connection with the TOS system.
[0032] The intelligent dispatching system is a supplement to the TOS system. Since it is difficult for the TOS system's instructions to take into account the real-time situation of the terminal, the intelligent dispatching system relies on artificial intelligence to analyze the terminal situation in real time, and to make further dispatch and coordination for the TOS system's instructions. It coordinates and dispatches every vehicle in the area, making the operation of the entire area more reasonable, and making the efficiency of the unmanned yard exceed that of the manned yard.
[0033] Multi-agent unmanned driving platform The emergence of multi-agent technology has subversively improved the overall performance of the multi-unmanned vehicle system, and in some aspects has completely broken through the long-term bottlenecks and shortcomings of the AGV model and single-vehicle intelligence. The intelligent scheduling / multi-agent unmanned driving platform of this embodiment adopts multi-agent technology.
[0034] Each IGV is truly intelligent. IGV is no longer a puppet that relies on cloud-based decision-making for all situations. Instead, it has the ability to communicate with other IGVs. All IGVs can actively and reasonably cooperate to complete complex tasks. Moreover, with the support of the vehicle-road cloud platform, all IGVs can be integrated. Each IGV can fully "use" the "glasses" and "ears" of other IGVs and roadside equipment to fully understand the entire scene.
[0035] Environmental perception and positioning: mainly through sensor technology, cameras, GPS and other technologies to obtain environmental indicators during the driving process of the car, including lane line detection, traffic light recognition, traffic sign recognition, pedestrian detection, vehicle detection, obstacle recognition and vehicle positioning, etc., to collect relevant data; Decision-making planning: divided into task planning, behavior planning and trajectory planning. The collected data is used to make judgments and guide the vehicle's next behavior. Execution control: Currently, most vehicles use wire control design. This project controls the vehicle's drive, braking, steering, etc. based on the vehicle dynamics system model, so that the vehicle changes along the set driving trajectory.
[0036] IGV driverless solution IGV driverless vehicles have three operating modes: automatic driving, remote control, and manual control. When using the positioning solution, CPS information and differential GNSS+IMU+high-precision map positioning RSU are introduced as auxiliary when accurately docking with the aerial rail, quay crane, rail crane, and yard crane. The perception solutions include: multi-sensor fusion perception, laser radar, ultrasonic radar, vehicle-mounted camera, and RSU. The effective perception distance of a single vehicle is not less than 100 meters, and there is no perception blind spot. It realizes the perception of dynamic obstacles, static obstacles, lane lines, markers, and traffic signals. It can maintain work under restricted conditions in abnormal environments such as rain, snow, fog, and night. For the technology of obstacle avoidance, not only automatic perception and segmentation technology based on artificial intelligence is adopted, but also other technical means are used as auxiliary to achieve the goal of complete safety. It can adapt to a ground slope of 5%. It supports special auxiliary perception means for special targets (such as pedestrians) to improve the safety factor of the system.
[0037] Path planning is combined with TOS and ECS. The cloud jointly plans and issues tasks and implements global path planning, while the on-board equipment implements local path planning.
[0038] Lane keeping uses on-board cameras, laser radar and high-precision maps to obtain the road conditions of the lane ahead, so that the IGV automatically keeps driving in the current lane with a driving error of no more than 50mm. It can adapt to cloudy and rainy weather, strong light, heavy fog, and lane line damage rate less than 50%. The speed limits of different sections of the terminal are different, such as main roads, terminal fronts, yard roads, etc. IGV can achieve parameterized maximum speed and meet different configuration requirements in different areas.
[0039] IGV has an automatic following function, and the minimum following distance during driving is not less than the safe distance.
[0040] When IGV borrows lane, it will judge the road condition of the borrow lane to ensure safe merging into the borrow lane. When there are no moving machinery and other obstacles in the borrow lane within 40m before and 20m after the driving section, and there is no operation in the empty track and RMG lane within 40m before and 20m after the driving section, it can borrow lane, and the minimum overtaking distance is not less than 5m. (The above parameters can be adjusted according to actual conditions) Supports single large box, single small box (front, middle, rear) and other modes. The CPS input information is used as positioning information, and the precise parking position is determined based on the box area, bay position information and loading and unloading mode information input by TOS. According to the parking position and vehicle weight, the optimal deceleration distance is calculated, and the posture is fine-tuned at the end to finally achieve precise parking, loading and unloading boxes at one time.
[0041] 1:1 scene reproduction technology Reconstruction of three-dimensional road scenes based on fixed-pose monocular cameras and laser point clouds. The perception system is a monocular long-focal-length camera and a multi-threaded laser radar. Since the test base station needs to perceive road scenes 150 meters away, and the camera projection matrix causes it to have a weak perception of distant effects, the distribution density of the laser radar point cloud data at near and far points is very different. Therefore, how to parse the road scene relationship through the monocular camera inverse projection matrix, identify the contour and three-dimensional size of the target object through the laser point cloud, obtain a clear and complete three-dimensional spatial dynamic structure diagram of the complex road scene, and transmit it to the industrial computer processing unit in real time is the basis for realizing adaptive perception of intersection scenes.
[0042] Vehicle-pedestrian comprehensive posture estimation and continuous trajectory tracking. The core issue of this project is to establish a continuous trajectory tracking model for vehicles and pedestrians. To this end, it is necessary to design a deep learning model with real-time tracking and high precision to cope with various motion forms of vehicles and pedestrians, such as static, constant speed, variable speed, and turning, under the constraints of high-dynamic scenes. Therefore, how to study the convolutional neural network architecture for fast feature extraction based on multi-scenario in-loop and actual road tests, combined with prior knowledge such as vehicle characteristics and size, to obtain accurate dynamic target tracking and positioning, is a key issue in intersection scene perception.
[0043] Establishment of time-domain-spatial domain correlation model and abnormal event cost function modeling in complex road scenes. Building a road time-domain-spatial domain scene model is the key point to perceive abnormal events, such as accidents, congestion and other emergencies that affect normal road traffic. To this end, it is necessary to study and establish parameterizable abnormal event judgment indicators based on the background extracted from the image, the three-dimensional position and size of dynamic objects, speed and direction, and other dimensions, establish the corresponding model relationship between the independent variable and the dependent variable of the indicator, and then construct the abnormal event cost function, adaptively configure the warning trigger threshold, and realize timely detection and warning of abnormal events.
[0044] V2X Technology An autonomous vehicle is a complex information-physical fusion system that integrates mechatronics, software and hardware, and ultimately replaces human operations. It is mainly composed of perception, decision-making and execution subsystems. Autonomous driving technology involves key technologies such as environmental perception, decision-making planning, control execution, and V2X communication.
[0045] Environmental perception technology uses on-board sensor equipment (such as GPS / INS system, millimeter wave radar / camera) and 5G network to obtain the traffic environment information and vehicle status information (position, posture) of the car, and unifies the output information of multiple sensors in the vehicle coordinate system to establish metadata with time-stamped data association and fusion to serve the decision-making and planning of autonomous driving. Decision-making and planning technology implements routing, traffic prediction, behavior decision-making, action planning and feedback control signal output functions based on the output information of the environmental perception subsystem. Control execution technology uses wire-controlled actuators to complete the execution of feedback control output instructions to achieve steering, throttle and braking control. V2X technology provides real-time and reliable communication services for information exchange between the vehicle and the outside world, and serves environmental perception and decision-making and planning.
[0046] In the above embodiments, TOS: terminal operation system; AGV: automatic guided vehicle; IGV: intelligent automatic guided vehicle; CPS: a system that integrates computing, networking, and physical processes to combine the physical world and the digital world; GNSS: global satellite navigation system; IMU: a sensor that measures the acceleration, angular velocity, and speed of an object; RSU: intelligent traffic management system; ECS: cloud computing service.
[0047] The digital twin system for the control and management of the port's autonomous driving is a full-featured management platform that replaces the original manual driving with platform management, meeting the four core technologies of autonomous driving: environmental perception, precise positioning, decision-making planning, and vehicle control, achieving the effects of remote control, remote supervision, and remote dispatch, which not only reduces the cost of manual operation and improves work efficiency, but also provides the port with the latest policy and regulatory data applications, ensuring smooth and safe daily work. In addition, the present invention uses a multi-agent unmanned driving platform to integrate all IGV vehicles, to cooperate with each other and interact with perceived data, thereby decentralizing the computing pressure and data transmission pressure of the cloud computing center within the framework of the multi-agent system. Any damage or downtime of a single device will not cause the paralysis of the overall system, making it more stable. It has been verified that after the port area was renovated, the operating cost was reduced by 20%, the throughput was increased by 30%, the accident rate was reduced by 100%, and the casualty rate was reduced by 100%, making it safer.
[0048] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. The port autonomous driving control and management digital twin system includes an autonomous driving digital twin platform, an intelligent dispatching platform, a multi-agent unmanned driving platform, an IGV unmanned driving system, and a scene reproduction platform, which is characterized by: The autonomous driving digital twin platform is used to monitor and control the unmanned vehicle team in the production environment in real time, and establish a digital twin environment of the yard terminal to simulate various scenarios and vehicle parameter changes; the intelligent scheduling platform is based on the TOS terminal business operating system, and uses artificial intelligence to analyze the terminal situation in real time, schedule and coordinate the instructions of the TOS terminal business operating system, and coordinate the production operations of each vehicle in the scheduling area; the multi-agent unmanned driving platform is based on the support of the vehicle-road cloud platform, integrates all IGV vehicles, cooperates with each other and interacts with perception data, and simultaneously performs environmental perception positioning, decision-making planning and execution control on IGV vehicles, and relies on V2X technology to interact information between the vehicle and the outside world; The IGV unmanned driving system is built into the IGV vehicle, providing three operating modes: automatic driving, remote control, and manual control, and realizing positioning, automatic perception, and obstacle avoidance functions. It simultaneously combines the TOS terminal business operating system and ECS cloud computing services to perform path planning, as well as lane keeping, automatic following, lane borrowing judgment, and loading and unloading parking judgment functions; the scene reproduction platform reconstructs the three-dimensional road scene based on a fixed-pose monocular camera and a laser point cloud, estimates the comprehensive posture of vehicles and pedestrians, and tracks continuous trajectories. At the same time, it establishes time domain and spatial domain correlation models and abnormal event cost function models under complex road scenes to detect and warn abnormal events.
2. The port automatic driving control and management digital twin system according to claim 1 is characterized by: The autonomous driving digital twin platform is a combination of management software and simulation system. The management software is used to monitor and control the unmanned vehicle team in the production environment in real time. The simulation system is used to establish a digital twin environment of the yard terminal and simulate various realistic scenarios and vehicle parameter changes.
3. The port automatic driving control and management digital twin system according to claim 1 is characterized by: The TOS terminal business operating system is used to guide the production plan of the terminal organization, record and store key data in the terminal operation process, allocate resources, and provide data statistics and information analysis for terminal production decisions. The intelligent scheduling platform is a supplement to the TOS terminal business operating system.
4. The port automatic driving control management digital twin system according to claim 1 is characterized by: The multi-agent unmanned driving platform adopts multi-agent technology and is built into each IGV vehicle, driving each IGV vehicle to communicate with other IGV vehicles, and all IGV vehicles cooperate to complete complex tasks. At the same time, based on the support of the vehicle-road cloud platform, all IGV vehicles are integrated into one, and each IGV vehicle perceives and interacts with other IGV vehicles and roadside equipment to identify the entire scene.
5. The port automatic driving control management digital twin system according to claim 4 is characterized by: The environmental perception and positioning is to obtain environmental indicators during vehicle driving through sensor technology, camera and GPS technology, including lane line detection, traffic light recognition, traffic sign recognition, pedestrian detection, vehicle detection, obstacle recognition and vehicle positioning, collect relevant data, unify them in the vehicle coordinate system, and establish metadata with time-stamped data association and fusion; the decision-making planning is to make judgments and guidance on the next behavior of the vehicle through the collected relevant data, including routing, traffic prediction, behavior decision, action planning and feedback control signal output; the execution control is to use the wire control actuator to complete the execution of feedback control output instructions based on the vehicle dynamics system model, control the vehicle drive, throttle, brake and steering, so that the vehicle changes according to the established driving trajectory; the V2X technology provides real-time communication services for information exchange between the vehicle and the outside world, and serves environmental perception and positioning and decision-making planning.
6. The port automatic driving control management digital twin system according to claim 1 is characterized by: In the IGV unmanned driving system, when docking with the aerial rail, quay crane, rail crane, and yard crane, positioning introduces CPS information and differential GNSS+IMU+high-precision map positioning RSU; automatic perception is multi-sensor fusion perception, including laser radar, ultrasonic radar, vehicle-mounted camera, RSU, with a single vehicle effective perception distance of not less than 100 meters, realizing the perception of dynamic obstacles, static obstacles, lane lines, signs, and traffic signals; Obstacle avoidance uses automatic perception and segmentation technology based on artificial intelligence, and is adapted to a ground slope of 5%.
7. The port automatic driving control management digital twin system according to claim 6 is characterized by: Path planning: The cloud jointly plans and issues tasks and performs global path planning, while the on-board equipment performs local path planning; Lane keeping function: Use on-board cameras, lidar and high-precision maps to obtain the road conditions of the lane ahead, so that the IGV vehicle automatically maintains the current lane and achieves parameterized maximum speed; Automatic following function: Control the minimum following distance during driving to be no less than the safe distance; Borrowing lane judgment function: Judge the road conditions of the borrowing lane, control the minimum overtaking distance to be no less than 5m, and merge into the borrowing lane at a safe distance; Loading and unloading parking judgment function: Supports single large box and single small box modes. When the CPS inputs information as positioning information, the parking position is determined according to the TOS input box area, bay position information and loading and unloading mode information. According to the parking position and vehicle weight, the optimal deceleration distance is calculated, and finally the posture is fine-tuned to achieve parking, loading and unloading.
8. The port automatic driving control and management digital twin system according to claim 1 is characterized by: Reconstruction of road 3D scenes based on fixed-pose monocular camera and laser point cloud: The road scene relationship is analyzed through the inverse projection matrix of the monocular camera, and the outline and 3D size of the target object are identified through the laser point cloud to obtain a complete 3D spatial dynamic structure diagram of the complex road scene, which is transmitted to the industrial computer processing unit in real time to achieve adaptive perception of intersection scenes.
9. The port automatic driving control management digital twin system according to claim 8 is characterized by: Comprehensive posture estimation and continuous trajectory tracking of vehicles and pedestrians: Under the constraints of high-dynamic scenes, we design a deep learning model for real-time tracking to cope with various motion forms of vehicles and pedestrians, such as stationary, constant speed, variable speed, and turning. Based on multi-scene in-loop and actual road tests, we build a convolutional neural network architecture for feature extraction, and combine vehicle features and size prior knowledge to obtain dynamic target tracking and positioning. In summary, we establish a continuous trajectory tracking model for vehicles and pedestrians to perceive intersection scenes.
10. The port automatic driving control management digital twin system according to claim 9 is characterized by: Establish the time domain and space domain correlation model and abnormal event cost function model in complex road scenes: collect road time domain and space domain scene images, and model them according to the background extracted from the images, the three-dimensional position and size of dynamic objects, speed and direction, etc., establish parameterizable abnormal event judgment indicators, set the corresponding model relationship between the independent variable and the dependent variable of the indicator, construct the abnormal event cost function, adaptively configure the warning trigger threshold, and realize timely detection and warning of abnormal events.
Citation Information
Cited By
Port automatic driving vehicle simulation test method
CN119882480A
Autonomous travel system based on artificial intelligence and automatic driving technology
CN120143675A
High-precision container truck vehicle positioning system and method in container area
CN120491120A
Perception and safety protection system and method for obstacles above unmanned vehicle at port
CN120840600A
A port unmanned vehicle overhead obstacle perception and safety protection system and method
CN120840600B
