An intelligent port dispatching and command system and method for handling container trucks

Abstract

The present invention relates to the field of integrated port information management and communication technology, and discloses an intelligent port dispatching and commanding system for container trucks, comprising a database server, a BDS receiver serving as a base station component, at least one BDS wireless terminal, a video monitoring subsystem, a GPU server equipped with a vehicle path planning subsystem, and a BDS server equipped with a distributed processing subsystem. The present invention further discloses an intelligent port dispatching and commanding method for container trucks, which, through the distributed processing subsystem, executes the following steps: S51, obtaining the current positioning data of the container truck; S52, obtaining a real-time image and pushing it to a genetic algorithm software module; S53, establishing a calculation model for the container truck's operating path; S54, performing image recognition based on a convolutional neural network; and S55, establishing an optimal planning model for the container truck's path and sending it to the vehicle path planning subsystem. The present invention achieves the goal of intelligent port dispatching and commanding, improving port operating efficiency and safety protection levels.

Description

Technical Field

[0001] The present invention relates to the field of integrated port information management and communication technology, and in particular to an intelligent port dispatching and commanding system and method for handling container trucks. Background Art

[0002] China's BeiDou Navigation Satellite System (BDS) is a global satellite navigation system independently developed by China. It is the third mature satellite navigation system after the US Global Positioning System (GPS) and Russia's GLONASS. A Geographic Information System (GIS) is a specific and crucial spatial information system. It is a technical system, supported by computer hardware and software, that collects, stores, manages, calculates, analyzes, displays, and describes geographic distribution data for the entire or partial Earth's surface (including the atmosphere). A BDS receiver device must consist of at least an antenna capable of receiving BDS signals, a BDS receiver, a processor, and a data output control interface. The processor is equipped with software tools for calculating and outputting latitude / longitude position data. BDS wireless terminals are specifically designed for BeiDou sub-meter positioning applications in various scenarios. They feature wireless network communication capabilities and a mainboard with a public network communication module, a built-in BeiDou positioning system module, integrated solar panels, and a rechargeable battery.

[0003] In the management of port equipment, transportation, and information technology, the mobile machinery involved in port operations is diverse and numerous, including loaders, excavators, tire cranes, forklifts, dump trucks, and more. Combined with port logistics vehicles and non-production vehicles, the large number of vehicles traversing the port area and surrounding cargo yards places tremendous pressure on port safety and production. Strengthening the monitoring of port production vehicles, standardizing vehicle operation procedures, improving terminal efficiency and cargo transportation safety, and further reducing production costs are urgent challenges facing major ports. Existing port traffic control systems mostly rely on pure video monitoring for target identification, tracking, and vehicle monitoring. Currently, Chinese port traffic control systems still rely on this existing approach. However, due to the unique nature of port operations, existing video monitoring systems are unable to meet the unified information management needs of large-scale ship loading and unloading machinery, truck transportation, and storage yards. Consequently, they fail to improve vehicle operational potential, reduce freight accidents, or reduce worker workload. Summary of the Invention

[0004] In view of this, it is necessary to propose an intelligent port dispatching and command system and method for container trucks to address the above-mentioned problems, so as to solve the shortcomings of the above-mentioned background technology, enable container trucks to meet the dispatching and command requirements at the lowest overall cost throughout the entire process of port operations, and achieve the management goals of container trucks in a high-efficiency, real-time, standardized, scientific and intelligent manner.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention proposes an intelligent port dispatching and command system for container trucks, comprising a database server, a BDS receiver as a base station component, at least one BDS wireless terminal, a video monitoring subsystem, a GPU server equipped with a vehicle path planning subsystem, and a BDS server equipped with a distributed processing subsystem.

[0007] The database server is used to store the received Beidou satellite positioning data, the calculation result data of the vehicle path planning subsystem and the business data of the distributed processing subsystem;

[0008] The BDS server is connected to the video surveillance subsystem, the database server, and the GPU server through an intranet, the BDS server is connected to the extranet, and the BDS server is connected to the BDS receiver through a line;

[0009] The video monitoring subsystem includes a plurality of cameras in communication connection, and the cameras are used to shoot and collect images of the container truck and the work area in real time;

[0010] The distributed processing subsystem includes a Beidou satellite positioning data receiving software module, a video stream access software module and a genetic algorithm software module; the Beidou satellite positioning data receiving software module is used to access Beidou satellite positioning data for BDS receivers and BDS wireless terminals; the video stream access software module is used to process video stream data captured by the video monitoring subsystem; the genetic algorithm software module is used to calculate the optimal path algorithm model based on Beidou satellite positioning data and port geographic information data;

[0011] The vehicle path planning subsystem plans and issues early warnings on the optimal path for container trucks through the operation path calculation model established by the distributed processing subsystem.

[0012] Furthermore, the vehicle path planning subsystem includes a map rendering software module, a three-dimensional visualization software module, a path planning software module, and an early warning software module;

[0013] The map rendering software module is responsible for displaying a panoramic map of the port and its surrounding environment;

[0014] The 3D visualization software module is responsible for displaying the running path and status of the container truck in a 3D visualization manner in combination with the GIS map, as well as the 3D presentation of lane lines, cameras, obstacles and other structures;

[0015] The path planning software module is responsible for integrating the container truck BDS positioning data, the relative position of the truck camera, the driver's status and obstacle information data, and then planning the optimal driving path for the container truck;

[0016] The early warning software module is responsible for issuing early warning notifications of abnormal operating conditions of container trucks, obstacle collision detection, and dangerous driver behaviors.

[0017] Furthermore, the genetic algorithm software module runs a plurality of dispatching and commanding programs based on genetic algorithms, and utilizes genetic algorithms to calculate the optimal path specification and obtain the optimal solution of the function.

[0018] Furthermore, the vehicle path planning subsystem performs a multi-source data comparison between port geographic information, location information of port static structures, and port dynamic information including container truck information and obstacle information; the vehicle path planning subsystem plans the optimal driving path for the container truck and notifies and warns of abnormal conditions during the driving of the container truck, including lane departure, collision warning, and dangerous driving behavior.

[0019] The present invention further proposes an intelligent port dispatching and commanding method for handling container trucks. The intelligent port dispatching and commanding method is applied to an intelligent port dispatching and commanding system for handling container trucks. The intelligent port dispatching and commanding system includes a database server, a BDS receiver as a base station, a BDS wireless terminal, a video monitoring subsystem, a GPU server equipped with a vehicle path planning subsystem, and a BDS server equipped with a distributed processing subsystem.

[0020] The database server is used to store the received Beidou satellite positioning data, the calculation result data of the vehicle path planning subsystem and the business data of the distributed processing subsystem;

[0021] The BDS server is connected to the database server and the GPU server through an intranet, the BDS server is connected to the extranet, and the BDS server is connected to the BDS receiver through a line;

[0022] The video monitoring subsystem includes a plurality of cameras in communication connection, and the cameras are used to shoot and collect images of the container truck and the work area in real time;

[0023] The vehicle path planning subsystem plans and issues early warnings for the optimal path of the container truck using the operation path calculation model established by the distributed processing subsystem;

[0024] The intelligent port dispatching and commanding method performs the following steps through the distributed processing subsystem:

[0025] S51, the BDS wireless terminal installed on the container truck forwards the relevant data to the distributed processing subsystem after calibration by the BDS receiver to obtain the current positioning data of the container truck;

[0026] S52, the video monitoring subsystem pushes the original code stream information of the camera image to the distributed processing subsystem, which decodes the original code stream information and obtains a real-time image of the container truck's current operation dynamics or working area, and pushes the real-time image to the genetic algorithm software module;

[0027] S53, based on the real-time image obtained in S52, a genetic algorithm is used to describe the optimal path planning problem using a directed graph, and an operation path calculation model for the container truck is established;

[0028] S54, performing image recognition based on a convolutional neural network, processing, calculating, and comparing the current real-time image through a corresponding detection network, obtaining an operating status detection result of the container truck in operation, and sending the operating status detection result to the vehicle path planning subsystem;

[0029] S55, establishing an optimal container truck path planning model by combining the operation path calculation model and the container truck operation status through a genetic algorithm, and sending the optimal container truck path planning model to the vehicle path planning subsystem;

[0030] Between S54 and S55, the intelligent port dispatching and command method further executes: S5455, analyzing the operation status detection result by the vehicle path planning subsystem, and sending an external warning if it is determined to be an unsafe state;

[0031] After S55 , the intelligent port dispatching and commanding method further executes: S56 , wherein the vehicle path planning subsystem provides optimal dispatching and commanding information through the received container truck path optimal planning model.

[0032] Furthermore, in S53 , the path planning is converted into a multi-objective optimization problem, and the objective function is established with the shortest path, the shortest operation time, and the lowest fuel consumption as the goals.

[0033] Furthermore, in S54, the detection and recognition objects of the detection network include: lane detection, obstacle recognition and driver behavior; S54 includes the following steps:

[0034] S541, establishing image samples of lane lines, obstacles, and driver behavior based on the existing historical data of the video monitoring subsystem and creating a data set;

[0035] S542: Label the sample images, labeling lane line image samples into lane line class and background class, labeling obstacle image samples into normal class and obstacle class, and labeling driver behavior image samples into normal behavior class and dangerous behavior class;

[0036] S543, using the marked categories as labels corresponding to the data set, and dividing the data set into a training set, a validation set, and a test set;

[0037] S544: Build a detection network model encoder-decoder based on a convolutional neural network. The network consists of a downsampling part and an upsampling part. The network input is the image to be detected, and the network output is the effect image of lane detection, obstacle monitoring, and driver dangerous behavior detection. The network model is trained using the training data set.

[0038] S545: Pushing the original code stream information of the camera image to the distributed processing subsystem. The distributed processing subsystem uses an audio / video digital conversion software tool to extract the image data of the current frame from the original code stream information and notifies the distributed processing subsystem to detect the image through an internal message queue.

[0039] S546, after receiving the notification of image detection, extract the image to be detected from the internal message queue, input the extracted image to be detected into the trained detection network, and then output the detection result as a detection image.

[0040] Furthermore, the intelligent port dispatching and commanding method further performs the following steps through the vehicle path planning subsystem:

[0041] S41, performing a multi-source data comparison between the port geographic information, the location information of the port static structure, and the port dynamic information, including the container truck information and the obstacle information;

[0042] S42, planning the optimal driving path of the container truck and providing notifications and warnings for abnormal conditions during the driving of the container truck, including lane departure, collision warning, and dangerous driving behavior.

[0043] Furthermore, S41 includes S411-S415 as follows:

[0044] S411: Build an offline map engine tool GeoServer and an offline map engine tool MapBox, import the port map data and publish the map tile layer service in the local area network;

[0045] S412: Use the offline map engine tool MapBox to mark the static structures in the port;

[0046] S413: Building a 3D visualization scene based on Unity3D, a map service software tool, and importing the port's geographic data as terrain and map data for the 3D scene using MapBox-Unity-SDK, a map software development kit for Unity3D, and also importing a container truck model created in advance using a digital content creation software tool;

[0047] S414, generating a low-poly 3D model corresponding to the static structure of the port by marking the location of the static structure in advance and drawing a planned driving path in the 3D visualization scene;

[0048] S415, driving the container truck to move in the three-dimensional visualization scene using the container truck location information published by the distributed processing subsystem;

[0049] S42 includes S421-S423, as follows:

[0050] S421, calculating the optimal path using the container truck planning path calculation model of the distributed processing subsystem, and drawing the planned driving path in the scene;

[0051] S422, generating a low-poly 3D model of the obstacle in the 3D visualization scene using the obstacle information including the distance relative to the camera and the distance to the lane line from the distributed processing subsystem;

[0052] S423, using the physics engine system of the map service software tool Unity3D to detect the specific state of the container truck, which is the congestion and collision that will occur in the current driving route and the planned driving route. Then, combined with the driver's dangerous behavior and lane departure information released by the distributed processing subsystem, an early warning is sent to the outside.

[0053] The present invention further provides a computer-readable storage medium storing a program for executing the steps of the above method.

[0054] The beneficial effects of the present invention are:

[0055] The present invention proposes an intelligent port dispatching and commanding system and method for container trucks. By installing a Beidou positioning terminal and utilizing an existing remote-controlled camera, the system efficiently integrates the geographic positioning information of the container truck and the image information of the operating status of the container truck, and plans the optimal route of the vehicle in advance, thereby realizing highly intelligent port dispatching and commanding management, and effectively improving the operating efficiency of the port. Moreover, the present invention also utilizes the efficient integration of high-precision Beidou satellite positioning data and real-time video data provided by the camera, adopts a multi-source data fusion processing strategy based on a neural network, an optimal path planning method based on a genetic algorithm, and integrates GIS technology and three-dimensional visualization technology to realize intelligent dispatching and commanding of container trucks, thereby meeting the requirements of port management services at the lowest comprehensive cost, thereby realizing the high-efficiency, real-time, standardized, scientific and intelligent management goals of port dispatching and commanding for container trucks, and effectively improving the port operating efficiency and safety protection level. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are included to provide a further understanding of the present invention and are incorporated into and constitute a part of this specification. The accompanying drawings illustrate exemplary embodiments of the present invention and, together with the description, serve to explain the principles of the present invention. These drawings are for illustration purposes only and are not intended to limit the present invention.

[0057] Figure 1 This is a schematic structural diagram of an intelligent port dispatching and commanding system for container trucks according to the present invention;

[0058] Figure 2 A flow chart of steps executed by a distributed processing subsystem involved in an intelligent port dispatching and commanding method for container trucks according to the present invention;

[0059] Figure 3 for Figure 2 A flow chart of the steps executed by S54 involved;

[0060] Figure 4 A flowchart of the steps executed by the vehicle path planning subsystem involved in the intelligent port dispatching and command method for container trucks of the present invention;

[0061] Figure 5 for Figure 4 A flow chart of the steps performed in S41;

[0062] Figure 6 for Figure 4 A flowchart of the steps performed by S42 involved;

[0063] Description of reference numerals:

[0064] BDS receiver 1; BDS wireless terminal 2; video monitoring subsystem 3; vehicle path planning subsystem 41; GPU server 4; distributed processing subsystem 51; BDS server 5; database server 6; Beidou satellite positioning data receiving software module 511; video stream access software module 512; genetic algorithm software module 513; map rendering software module 411; 3D visualization software module 412; path planning software module 413; early warning software module 414. DETAILED DESCRIPTION

[0065] To make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be further clearly and completely described below in conjunction with the embodiments of the present invention. It should be noted that the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0066] Terms such as "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Therefore, the definition of "first," "second," "third," and "fourth" may explicitly or implicitly include one or more of such features.

[0067] The following is a detailed description of an embodiment of the present invention as depicted in the accompanying drawings. The embodiments are detailed in order to clearly convey the present invention. However, the amount of detail provided is not intended to limit the intended variations of the embodiments; rather, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present invention as defined by the appended claims.

[0068] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent to one skilled in the art that the embodiments of the present invention may be practiced without some of these specific details.

[0069] Embodiments of the present invention include various steps, which are described below. These steps may be performed by hardware components or may be embodied in machine-executable instructions that can be used to program a general-purpose or special-purpose processor with the instructions to perform the steps. Alternatively, the steps may be performed by a combination of hardware, software, and firmware, and / or by a human operator.

[0070] The various methods described herein can be practiced by combining one or more machine-readable storage media containing code according to the present invention with appropriate standard computer hardware to execute the code contained therein. Apparatus for implementing various embodiments of the present invention may include one or more computers (or one or more processors within a single computer) and a storage system containing or having network access to a computer program encoded according to the various methods described herein, and the method steps of the present invention may be accomplished by modules, routines, subroutines, or sub-parts of a computer program product.

[0071] If the specification states that a component or feature "may," "could," "might," or "might" include or have a feature, that particular component or feature is not required to be included or have that feature.

[0072] As used in the specification herein and the claims that follow, the meanings of "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Further, as used in the description herein, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.

[0073] Exemplary embodiments will now be described more fully below with reference to the accompanying drawings, in which exemplary embodiments are shown. These exemplary embodiments are provided for illustrative purposes only, and to make the present invention thorough and complete, and to fully convey the scope of the present invention to those of ordinary skill in the art. However, the disclosed invention can be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein. Various modifications are apparent to those skilled in the art. Without departing from the spirit and scope of the present invention, the general principles defined herein can be applied to other embodiments and applications. In addition, all statements of the embodiments of the present invention and their specific examples described herein are intended to cover their structural and functional equivalents. In addition, these equivalents are intended to include currently known equivalents and equivalents developed in the future (i.e., any element developed that performs the same function, regardless of structure). Moreover, the terms and wording used are for the purpose of describing exemplary embodiments and should not be considered restrictive. Therefore, the present invention will be given the widest scope, including a variety of replacements, modifications, and equivalents consistent with the disclosed principles and features. For the sake of clarity, the details of technical materials known in the technical field related to the present invention are not described in detail to avoid unnecessarily obscuring the present invention.

[0074] Thus, for example, one of ordinary skill in the art will understand that schematic diagrams, schematic diagrams, diagrams, and the like represent conceptual views or processes embodying the systems and methods of the present invention. The functions of the various elements shown in the figures can be provided by using dedicated hardware and hardware capable of executing related software. Similarly, any switches shown in the figures are merely conceptual. Their functions can be performed by the operation of program logic, by dedicated logic, by the interaction of program control and dedicated logic, or even manually, with specific techniques being selected by the entity implementing the present invention. One of ordinary skill in the art will further understand that the exemplary hardware, software, processes, methods, and / or operating systems described herein are for illustrative purposes and are therefore not intended to be limited to any particular named elements.

[0075] Embodiments of the present invention may be provided as a computer program product that may include a machine-readable storage medium having instructions tangibly implemented thereon, which may be used to program a computer (or other electronic device) to perform a process. The term "machine-readable storage medium" or "computer-readable storage medium" includes, but is not limited to, fixed (hardware) drives, magnetic tapes, floppy disks, optical disks, compact disk read-only memories (CD-ROMs) and magneto-optical disks, semiconductor memories such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other types of media / machine-readable media suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware). Machine-readable media may include non-transitory media in which data may be stored and does not include carrier waves and / or transient electronic signals propagated over wireless or wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact disks (CDs) or digital versatile disks (DVDs), flash memory, memory, or memory devices. A computer program product may include code and / or machine-executable instructions, which may represent any combination of a procedure, function, subroutine, program, routine, subroutine, module, software package, class, or instruction, data structure, or program statement. A code segment may be coupled to another code segment or hardware circuit by passing and / or receiving information, data, variables, parameters, or memory contents. Information, variables, parameters, data, etc. may be passed, forwarded, or transmitted by any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0076] Furthermore, the embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments (e.g., a computer program product) that perform the necessary tasks may be stored in a machine-readable medium. A processor may perform the necessary tasks.

[0077] The systems depicted in some of the figures may be provided in various configurations. In some embodiments, the system may be configured as a distributed system, where one or more components of the system are distributed across one or more networks in a cloud computing system.

[0078] Each of the appended claims defines a separate invention, which, for infringement purposes, is recognized as including equivalents to the various elements or limitations specified in the claims. Depending on the context, all references below to the "invention" may, in some cases, refer only to certain specific embodiments. In other cases, it will be recognized that references to the "invention" will refer to the subject matter recited in one or more, but not necessarily all, of the claims.

[0079] Unless otherwise indicated herein or clearly contradicted by context, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., "such as") provided with respect to certain embodiments herein is intended only to better illustrate the present invention and is not intended to limit the scope of the claimed invention. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the present invention.

[0080] Various terms used herein are as follows. To the extent a term used in a claim is not defined below, it should be given the broadest definition persons in the relevant art have given that term as reflected in printed publications and issued patents at the time of filing this application.

[0081] Example 1

[0082] like Figure 1 As shown:

[0083] This embodiment proposes an intelligent port dispatching and command system for container trucks, including a database server 6, a BDS receiver 1 as a base station component, at least one BDS wireless terminal 2, a video monitoring subsystem 3, a GPU server 4 equipped with a vehicle path planning subsystem 41, and a BDS server 5 equipped with a distributed processing subsystem 51.

[0084] The database server 6 is used to store the received Beidou satellite positioning data, the calculation result data of the vehicle path planning subsystem 41 and the business data of the distributed processing subsystem 51;

[0085] The BDS server 5 is connected to the video surveillance subsystem 3, the database server 6, and the GPU server 4 through an intranet (specifically a local area network), the BDS server 5 is connected to the extranet (specifically a network that can access the BDS data network), and the BDS server 5 is connected to the BDS receiver 1 through a line;

[0086] The video monitoring subsystem 3 includes a number of communication-connected cameras, which are used to shoot and collect images of the container truck and the work area in real time;

[0087] The distributed processing subsystem 51 includes a Beidou satellite positioning data receiving software module 511, a video stream access software module 512 and a genetic algorithm software module 513; the Beidou satellite positioning data receiving software module 511 is used to access Beidou satellite positioning data for the BDS receiver 1 and the BDS wireless terminal 2; the video stream access software module 512 is used to process the video stream data captured by the video monitoring subsystem 3; the genetic algorithm software module 513 is used to calculate the optimal path algorithm model based on the Beidou satellite positioning data and the port geographic information data;

[0088] The vehicle path planning subsystem 41 plans and issues early warnings on the optimal path for the container truck through the operation path calculation model established by the distributed processing subsystem 51 .

[0089] In this embodiment, the vehicle path planning subsystem 41 includes a map rendering software module 411 , a three-dimensional visualization software module 412 , a path planning software module 413 , and an early warning software module 414 ;

[0090] The map rendering software module 411 is responsible for displaying a panoramic map of the port and its surrounding environment;

[0091] The 3D visualization software module 412 is responsible for displaying the running path and status of the container truck in a 3D visualization manner in combination with the GIS map, as well as the 3D presentation of lane lines, cameras, obstacles and other structures;

[0092] The path planning software module 413 is responsible for integrating the container truck BDS positioning data, the relative position of the truck camera, the driver's status and obstacle information data, and then planning the optimal driving path of the container truck;

[0093] The early warning software module 414 is responsible for issuing early warning notifications of abnormal operating status of the container truck, obstacle collision detection, and dangerous driver behavior.

[0094] Specifically, the BDS wireless terminal 2 adopts the JT / T808-2011 standard communication protocol; the BDS server 5 is a dual-machine hot standby server cluster; and the database server 6 is a three-machine hot standby server cluster.

[0095] Specifically, the video surveillance subsystem 3 also includes a field monitoring subsystem, which includes a streaming media server (using 128 channels) and a high-speed SSD hard disk burner (whose storage capacity is greater than 500TB); the field monitoring subsystem pushes the original code stream information of the camera image to the distributed processing subsystem 51 through the streaming media server, and the distributed processing subsystem 51 uses FFmpeg to extract the image data of the current frame from the original code stream every ten frames, and notifies the genetic algorithm software module 513 through the internal message queue to detect the image.

[0096] Furthermore, in this embodiment, the genetic algorithm software module 513 runs a plurality of scheduling and command programs based on genetic algorithms, and utilizes genetic algorithms to calculate the optimal path specification and obtain the optimal solution of the function.

[0097] Furthermore, in this embodiment, the vehicle routing subsystem 41 performs a multi-source data comparison between port geographic information, including the location information of static port structures, and dynamic port information, including container truck information and obstacle information. The vehicle routing subsystem 41 plans the optimal driving path for the container truck and provides notifications and warnings for abnormal driving conditions, including lane departures, collision warnings, and dangerous driving behaviors. Specifically, the static port structures include the location information of buildings, cameras, and lane markings.

[0098] Furthermore, in this embodiment, the genetic algorithm software module 513 converts path planning into a multi-objective optimization problem, and establishes an objective function with the shortest path, shortest operation time, and lowest fuel consumption as the goals; the detection and recognition objects of the detection network of the genetic algorithm software module 513 include: lane detection, obstacle recognition, and driver behavior.

[0099] Furthermore, in this embodiment, the path planning program of the genetic algorithm software module 513 uses a directed graph based on the genetic algorithm to describe the optimal path planning problem and establishes an operating path calculation model for the container truck; the genetic algorithm software module 513 performs image recognition based on a convolutional neural network, and the corresponding detection network processes, calculates, and compares the current real-time image to obtain the operating status detection result of the container truck under the operating state, and sends the operating status detection result to the vehicle path planning subsystem 41; the vehicle path planning subsystem 41 analyzes the operating status detection result, and if it is determined to be an unsafe state, it sends an early warning to the outside; the genetic algorithm of the genetic algorithm software module 513 combines the operating path calculation model and the operating state of the container truck to establish an optimal planning model for the container truck path and sends the optimal planning model for the container truck path to the vehicle path planning subsystem, and the vehicle path planning subsystem 41 provides optimal scheduling command information through the optimal planning model for the container truck path.

[0100] Example 2

[0101] like Figure 1 、 Figure 2 As shown:

[0102] This embodiment provides an intelligent port dispatching and commanding method for handling container trucks. The intelligent port dispatching and commanding method is applied to an intelligent port dispatching and commanding system for handling container trucks. The intelligent port dispatching and commanding system includes a database server 6, a BDS receiver 1 serving as a base station, a BDS wireless terminal 2, a video monitoring subsystem 3, a GPU server 4 equipped with a vehicle path planning subsystem 41, and a BDS server 5 equipped with a distributed processing subsystem 51.

[0103] The database server 6 is used to store the received Beidou satellite positioning data, the calculation result data of the vehicle path planning subsystem 41 and the business data of the distributed processing subsystem 51;

[0104] The BDS server 5 is connected to the database server 6 and the GPU server 4 through an intranet, the BDS server 5 is connected to the external network (specifically, a network that can access the BDS data network), and the BDS server 5 is connected to the BDS receiver 1 through a line;

[0105] The video monitoring subsystem 3 includes a number of communication-connected cameras, which are used to shoot and collect images of the container truck and the work area in real time;

[0106] The vehicle path planning subsystem 41 plans and issues early warnings for the optimal path of the container truck using the operation path calculation model established by the distributed processing subsystem 51;

[0107] The intelligent port dispatching and commanding method performs the following steps through the distributed processing subsystem 51:

[0108] S51, the BDS wireless terminal 2 installed on the container truck forwards the relevant data to the distributed processing subsystem 51 after calibration by the BDS receiver 1 to obtain the current positioning data of the container truck;

[0109] S52: The video monitoring subsystem 3 pushes the original bitstream information of the camera image to the distributed processing subsystem 51. The distributed processing subsystem 51 decodes the original bitstream information and obtains a real-time image of the container truck's current operation or working area. The real-time image is then pushed to the genetic algorithm software module 513. Specifically, a hardware decoding module of the distributed processing subsystem 51 decodes the original bitstream information.

[0110] S53, based on the real-time image obtained in S52, a genetic algorithm is used to describe the optimal path planning problem using a directed graph, and an operation path calculation model for the container truck is established;

[0111] S54, performing image recognition based on a convolutional neural network, processing, calculating, and comparing the current real-time image through a corresponding detection network, obtaining an operating status detection result of the container truck in operation, and sending the operating status detection result to the vehicle routing subsystem 41;

[0112] S55, using a genetic algorithm to combine the operation path calculation model and the container truck operation status to establish an optimal container truck path planning model, and sending the optimal container truck path planning model to the vehicle path planning subsystem 41;

[0113] Between S54 and S55, the intelligent port dispatching and commanding method further executes: S5455, the vehicle path planning subsystem 41 analyzes the running state detection result, and sends an external warning if it is determined to be an unsafe state;

[0114] After S55 , the intelligent port dispatching and commanding method further executes: S56 , wherein the vehicle path planning subsystem 41 provides optimal dispatching and commanding information through the received container truck path optimal planning model.

[0115] In this embodiment, in S53 , the path planning is converted into a multi-objective optimization problem, and the objective function is established with the shortest path, the shortest operation time, and the lowest fuel consumption as the goals.

[0116] In this embodiment, further, in S54 , the detection and recognition objects of the detection network include: lane detection, obstacle recognition, and driver behavior.

[0117] In this embodiment, further, Figure 3 As shown, S54 includes the following steps:

[0118] S541: Create image samples of lane lines, obstacles, and driver behavior based on the existing historical data of the video monitoring subsystem 3 and create a data set;

[0119] S542: Label the sample images, labeling lane line image samples into lane line class and background class, labeling obstacle image samples into normal class and obstacle class, and labeling driver behavior image samples into normal behavior class and dangerous behavior class;

[0120] S543, using the marked categories as labels corresponding to the data set, and dividing the data set into a training set, a validation set, and a test set;

[0121] S544: Build a detection network model encoder-decoder based on a convolutional neural network. The network consists of a downsampling part and an upsampling part. The network input is the image to be detected, and the network output is the effect image of lane detection, obstacle monitoring, and driver dangerous behavior detection. The network model is trained using the training data set.

[0122] S545: Push the original code stream information of the camera image to the distributed processing subsystem 51. The distributed processing subsystem 51 uses FFmpeg, an audio / video digital conversion software tool, to extract the image data of the current frame from the original code stream information and notifies the distributed processing subsystem 51 through its internal message queue to perform image detection.

[0123] S546, after receiving the notification of image detection, extract the image to be detected from the internal message queue, input the extracted image to be detected into the trained detection network, and then output the detection result as a detection image.

[0124] In this embodiment, further, Figure 4 As shown, the intelligent port dispatching and commanding method further performs the following steps through the vehicle path planning subsystem 41:

[0125] S41, performing a multi-source data comparison between the port geographic information, the location information of the port static structure, and the port dynamic information, including the container truck information and the obstacle information;

[0126] S42, planning the optimal driving path of the container truck and providing notifications and warnings for abnormal conditions during the driving of the container truck, including lane departure, collision warning, and dangerous driving behavior.

[0127] In this embodiment, further, Figure 5 、 Figure 6 As shown, S41 includes S411-S415, as follows:

[0128] S411: Build an offline map engine tool GeoServer and an offline map engine tool MapBox, import the port map data and publish the map tile layer service in the local area network;

[0129] S412: Use the offline map engine tool MapBox to mark the static structures in the port;

[0130] S413: Building a 3D visualization scene based on Unity3D, a map service software tool, and importing the port's geographic data as terrain and map data for the 3D scene through MapBox-Unity-SDK, a map software development kit for Unity3D, and also importing a container truck model created in advance using DCC, a digital content creation software tool;

[0131] S414, generating a low-poly 3D model corresponding to the static structure of the port by marking the location of the static structure in advance and drawing a planned driving path in the 3D visualization scene;

[0132] S415 , driving the container truck to move in the three-dimensional visualization scene using the container truck position information published by the distributed processing subsystem 51 ;

[0133] S42 includes S421-S423, as follows:

[0134] S421, calculating the optimal path using the container truck planning path calculation model of the distributed processing subsystem 51, and drawing the planned driving path in the scene;

[0135] S422, generating a low-poly 3D model of the obstacle in the 3D visualization scene using the obstacle information including the distance relative to the camera and the distance to the lane line from the distributed processing subsystem 51;

[0136] S423, using the physics engine system of the map service software tool Unity3D to detect the specific state of the container truck, which is the congestion and collision that will occur in the current driving route and the planned driving route, and then sending an external warning in combination with the driver's dangerous behavior and lane departure information released by the distributed processing subsystem 51.

[0137] This embodiment further provides a computer-readable storage medium storing a program that performs the steps of any of the methods described above. The method can be described in the general context of computer-executable instructions. Generally, computer-executable instructions may include routines, programs, objects, components, data structures, procedures, modules, functions, and the like that perform specific functions or implement specific abstract data types. The method can also be implemented in a distributed computing environment, where functions are performed by remote processing devices linked by a communications network. In a distributed computing environment, computer-executable instructions can be located in local and remote computer storage media, including memory storage devices. The order of the described method is not intended to be construed as limiting, and any number of the described method blocks may be combined to implement the method or alternative methods. Furthermore, individual blocks may be deleted from the method without departing from the spirit and scope of the subject matter described herein. Furthermore, the method may be implemented using any suitable hardware, software, firmware, or a combination thereof. However, for ease of explanation, in the embodiments described above, the method may be considered to be implemented in the aforementioned system.

[0138] The intelligent port dispatching and commanding system for container trucks according to embodiment 1 and the intelligent port dispatching and commanding method for container trucks according to embodiment 2 of the present invention can at least achieve the following functions:

[0139] (1) Status monitoring function. The BDS wireless terminal 2 is installed on the container truck and transmits the collected data to the distributed processing subsystem 51 in real time through the local area network. The transmitted data includes positioning, running distance, running time, startup status, etc. The present invention combines the running distance, running time and startup status with the video stream data transmitted by the video monitoring subsystem 3 for fusion analysis, and performs three-dimensional visualization through the distributed processing subsystem 51, so as to accurately monitor the status of the container truck;

[0140] (2) Truck path planning function. The present invention uses an optimal path planning method based on a genetic algorithm to analyze Beidou satellite positioning data, establishes an objective function and a calculation model with the shortest path, shortest operation time, and lowest fuel consumption as the goals; the vehicle path planning subsystem 41 performs visual optimal path planning and early warning monitoring for trucks based on the calculation model.

[0141] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An intelligent port dispatching and command system for container trucks, characterized by: The system comprises a database server (6), a BDS receiver (1) as a base station component, at least one BDS wireless terminal (2), a video monitoring subsystem (3), a GPU server (4) equipped with a vehicle path planning subsystem (41), and a BDS server (5) equipped with a distributed processing subsystem (51); The database server (6) is used to store received Beidou satellite positioning data, calculation result data of the vehicle path planning subsystem (41) and business data of the distributed processing subsystem (51); The BDS server (5) is connected to the video monitoring subsystem (3), the database server (6), and the GPU server (4) via an intranet, the BDS server (5) is connected to the external network, and the BDS server (5) is connected to the BDS receiver (1) via a line; The video monitoring subsystem (3) includes a plurality of cameras connected in communication, and the cameras are used to shoot and collect images of the container truck and the work area in real time; The distributed processing subsystem (51) includes a Beidou satellite positioning data receiving software module (511), a video stream access software module (512) and a genetic algorithm software module (513); the Beidou satellite positioning data receiving software module (511) is used to access Beidou satellite positioning data to the BDS receiver (1) and the BDS wireless terminal (2); the video stream access software module (512) is used to process the video stream data captured by the video monitoring subsystem (3); and the genetic algorithm software module (513) is used to calculate an optimal path algorithm model based on the Beidou satellite positioning data and the port geographic information data. The vehicle path planning subsystem (41) plans and issues early warnings for the optimal path of the container truck using the operation path calculation model established by the distributed processing subsystem (51); The vehicle path planning subsystem (41) includes a map rendering software module (411), a three-dimensional visualization software module (412), a path planning software module (413), and an early warning software module (414); The map rendering software module (411) is responsible for displaying a panoramic map of the port and its surrounding environment; The three-dimensional visualization software module (412) is responsible for displaying the running path and status of the container truck in a three-dimensional visualization manner in combination with the GIS map, as well as the three-dimensional presentation of lane lines, cameras, obstacles and other structures; The path planning software module (413) is responsible for integrating the container truck BDS positioning data, the relative position of the container truck camera, the driver's status and the obstacle information data, and then planning the optimal driving path of the container truck; The early warning software module (414) is responsible for issuing early warning notifications of abnormal operating conditions of container trucks, obstacle collision detection, and dangerous driver behaviors; The genetic algorithm software module (513) runs a plurality of dispatching and commanding programs based on the genetic algorithm, and uses the genetic algorithm to calculate the optimal path specification and obtain the optimal solution of the function; The vehicle path planning subsystem (41) performs a multi-source data comparison between port geographic information, location information of port static structures, and port dynamic information, including container truck information and obstacle information. The vehicle path planning subsystem (41) plans the optimal driving path for the container truck and issues notifications and warnings for abnormal states of the container truck during driving, including lane departure, collision warning, and dangerous driving behavior.

2. An intelligent port dispatching and command method for container trucks, characterized in that: The intelligent port dispatching and commanding method is applied to an intelligent port dispatching and commanding system for handling container trucks. The intelligent port dispatching and commanding system comprises a database server (6), a BDS receiver (1) serving as a base station, a BDS wireless terminal (2), a video monitoring subsystem (3), a GPU server (4) equipped with a vehicle path planning subsystem (41), and a BDS server (5) equipped with a distributed processing subsystem (51); The database server (6) is used to store received Beidou satellite positioning data, calculation result data of the vehicle path planning subsystem (41) and business data of the distributed processing subsystem (51); The BDS server (5) is connected to the database server (6) and the GPU server (4) via an intranet, the BDS server (5) is connected to the external network, and the BDS server (5) is connected to the BDS receiver (1) via a line; The video monitoring subsystem (3) includes a plurality of cameras connected in communication, and the cameras are used to shoot and collect images of the container truck and the work area in real time; The vehicle path planning subsystem (41) plans and issues early warnings for the optimal path of the container truck using the operation path calculation model established by the distributed processing subsystem (51); The intelligent port dispatching and commanding method further performs the following steps through the vehicle path planning subsystem (41): S41, performing a multi-source data comparison between the port geographic information, the location information of the port static structure, and the port dynamic information, including the container truck information and the obstacle information; S42, planning the optimal driving path of the container truck and providing notifications and warnings for abnormal conditions during the driving of the container truck, including lane departure, collision warning, and dangerous driving behavior; S41 includes S411-S415, as follows: S411: Build an offline map engine tool GeoServer and an offline map engine tool MapBox, import the port map data and publish the map tile layer service in the local area network; S412: Use the offline map engine tool MapBox to mark the static structures in the port; S413: Building a 3D visualization scene based on Unity3D, a map service software tool, and importing the port's geographic data as terrain and map data for the 3D scene using MapBox-Unity-SDK, a map software development kit for Unity3D, and also importing a container truck model created in advance using a digital content creation software tool; S414, generating a low-poly 3D model corresponding to the static structure of the port by marking the location of the static structure in advance and drawing a planned driving path in the 3D visualization scene; S415, driving the container truck to move in the three-dimensional visualization scene by using the container truck position information published by the distributed processing subsystem (51); The intelligent port dispatching and commanding method performs the following steps through the distributed processing subsystem (51): S51, the BDS wireless terminal (2) installed on the container truck forwards the relevant data to the distributed processing subsystem (51) after calibration by the BDS receiver (1) to obtain the current positioning data of the container truck; S52, the video monitoring subsystem (3) pushes the original code stream information of the camera image to the distributed processing subsystem (51), and the distributed processing subsystem (51) decodes the original code stream information and then obtains the real-time image of the current operation dynamics or working area of ​​the container truck, and pushes the real-time image to the genetic algorithm software module (513); S53, based on the real-time image obtained in S52, a genetic algorithm is used to describe the optimal path planning problem using a directed graph, and an operation path calculation model for the container truck is established; S54, performing image recognition based on a convolutional neural network, processing, calculating, and comparing the current real-time image through a corresponding detection network, obtaining an operating status detection result of the container truck under the operating state, and sending the operating status detection result to the vehicle path planning subsystem (41); S55, establishing an optimal container truck path planning model by combining the operation path calculation model and the container truck operation status through a genetic algorithm, and sending the optimal container truck path planning model to the vehicle path planning subsystem (41); Between S54 and S55, the intelligent port dispatching and commanding method further executes: S5455, analyzing the running state detection result by the vehicle path planning subsystem (41), and sending an early warning if it is determined to be an unsafe state; After S55, the intelligent port dispatching and commanding method further executes: S56, providing optimal dispatching and commanding information by the vehicle path planning subsystem (41) through the received container truck path optimal planning model; In S53 , the path planning is converted into a multi-objective optimization problem, and the objective function is established with the shortest path, the shortest operation time, and the lowest fuel consumption as the objectives; In S54 , the detection and recognition objects of the detection network include lane detection, obstacle recognition, and driver behavior.

3. The intelligent port dispatching and commanding method for container trucks according to claim 2 is characterized in that: S54 includes the following steps: S541, based on the existing historical data of the video monitoring subsystem (3), image samples of lane lines, obstacles and driver behavior are established and a data set is created; S542: Label the sample images, labeling lane line image samples into lane line class and background class, labeling obstacle image samples into normal class and obstacle class, and labeling driver behavior image samples into normal behavior class and dangerous behavior class; S543, using the marked categories as labels corresponding to the data set, and dividing the data set into a training set, a validation set, and a test set; S544: Build a detection network model encoder-decoder based on a convolutional neural network. The network consists of a downsampling part and an upsampling part. The network input is the image to be detected, and the network output is the effect image of lane detection, obstacle monitoring, and driver dangerous behavior detection. The network model is trained using the training data set. S545, pushing the original code stream information of the camera image to the distributed processing subsystem (51), the distributed processing subsystem (51) uses an audio / video digital conversion software tool to extract the image data of the current frame from the original code stream information, and notifies the internal message queue of the distributed processing subsystem (51) to detect the image; S546, after receiving the notification of image detection, extract the image to be detected from the internal message queue, input the extracted image to be detected into the trained detection network, and then output the detection result as a detection image.

4. The intelligent port dispatching and commanding method for container trucks according to claim 3 is characterized in that: S42 includes S421-S423, as follows: S421, calculating the optimal path by the container truck planning path calculation model of the distributed processing subsystem (51), and drawing the planned driving path in the scene; S422, generating a low-poly three-dimensional model of the obstacle in the three-dimensional visualization scene using the obstacle information including the distance relative to the camera and the distance to the lane line from the distributed processing subsystem (51); S423, using the physics engine system of the map service software tool Unity3D to detect the specific state of the container truck, which is the congestion and collision that will occur in the current driving path and the planned driving path, and then sending an external warning in combination with the driver's dangerous behavior and lane departure information released by the distributed processing subsystem (51).

5. A computer-readable storage medium storing a program for executing the steps of the method according to any one of claims 2 to 4.

Images