Port Material Movement Scheduling Method Based on Multimodal Pathfinding Algorithm

Abstract

The present invention discloses a port material movement scheduling method based on a multimodal pathfinding algorithm, which includes obtaining satellite remote sensing images and device sensor data of the port area, performing adaptive encoding to obtain a multimodal factor library, constructing a multimodal pathfinding model based on the multimodal factor library, decomposing the scheduling operation into multiple sub-threads, using the multimodal pathfinding model to pre-search the sub-threads to obtain an initial set of sub-thread paths, performing global optimization on all the initial sets of sub-thread paths, taking the obtained global optimal solution as the multi-thread path, simulating the multi-thread path, and performing path correction based on the path crossing and obstacle collision situations in the simulation results, and taking the corrected path as the mobile scheduling path. This method effectively grasps the comprehensiveness of multimodal data, and at the same time, by dividing the scheduling task into sub-threads, it reduces the computational complexity of the scheduling task and has good interpretability.

Description

Technical Field

[0001] The present invention relates to the field of path planning and scheduling, and particularly to a port material movement scheduling method based on a multimodal pathfinding algorithm. Background Art

[0002] In modern port operations, the rapid and accurate movement of materials is crucial for maintaining the efficient operation of the port. With the rapid development of intelligent technologies, the automation and intelligence levels of port operations have been continuously improved. Material movement scheduling has become a key factor in port operation efficiency. The application of multimodal pathfinding algorithms in port material movement scheduling can ensure the high efficiency and safety of equipment operation, and avoid path conflicts and resource waste.

[0003] Traditional port material movement scheduling methods mainly rely on physical experiments, but such methods have problems such as high cost, long cycle, and poor repeatability. In recent years, with the rapid development of technologies, more and more fields have begun to use algorithm models for path planning to reduce costs and improve efficiency.

[0004] Currently, although there are some port material movement scheduling methods, these methods often only focus on optimizing a single path, lack the analysis of the global path in a complex port environment, and only focus on a single data source, ignoring the impact of multi-source data on path optimization and lacking the consideration of multimodal data. Summary of the Invention

[0005] The object of the present invention is to provide a port material movement scheduling method based on a multimodal pathfinding algorithm.

[0006] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0007] The first aspect of the present invention provides a port material movement scheduling method based on a multimodal pathfinding algorithm, including the following steps:

[0008] S1 Obtain satellite remote sensing images and device sensor data of the port area, perform adaptive encoding on the satellite remote sensing data and device sensor data of the port area to obtain a multimodal factor library;

[0009] S2 Construct a multimodal pathfinding model based on the multimodal factor library;

[0010] S3 Decompose the scheduling operation into multiple sub-threads, use the multimodal pathfinding model to perform pre-search on the sub-threads to obtain an initial set of sub-thread paths, perform global optimization on all initial sets of sub-thread paths, and use the obtained global optimal solution as the multi-thread path;

[0011] S4 simulates the multi-threaded path, corrects the path based on the path crossing and obstacle collision situations in the simulation results, and uses the corrected path as the mobile scheduling path.

[0012] Further, the method for adaptively encoding the satellite remote sensing data and device sensor data of the port area includes:

[0013] S11 extracts the spatial data modality from the satellite remote sensing image, including: spatial distribution, equipment, and material distribution, and extracts the structured data modality from the device sensor data, including: device status report, scheduling task, and cargo parameter information;

[0014] S12 designs a recurrent neural network adaptive encoder for the spatial data modality and a fully connected layer neural network adaptive encoder for the structured data modality;

[0015] S13 extracts the encoded feature representations through the corresponding auto-encoding models for the spatial data modality and the structured data modality, and performs normalized scale matching processing on the extracted encoded feature representations to obtain a dimensionless multi-modal factor library.

[0016] Further, the method for constructing a multi-modal path finding model based on the multi-modal factor library includes:

[0017] S21 designs a path finding method, rasterizes the port area, assumes that there is an attraction force on the material in the target area during material movement and a repulsive force on the material in the obstacle area. The calculation formula for the attraction force is:

[0018] F at =k a d(x - x goat )

[0019] where k a is the proportionality coefficient of the gravitational potential field, d(x - x goat ) is the distance from the current area point of the material to the target area. Calculate the repulsive force of the obstacle area on the material. The calculation formula is:

[0020]

[0021] where is the repulsive range of the obstacle, d min is the shortest distance of the material from the obstacle, K r is the proportionality coefficient of the repulsive potential field, is the density change sensitivity, is the density change of the material to the shortest obstacle, is the density change of the material from the current area to the target area. By calculating the resultant force at the position point of the material, the path finding from the starting area to the target area is completed;

[0022] S22 constructs a multi-modal pathfinding model based on a neural network. The construction process is as follows:

[0023] S221 constructs a multi-layer feedforward neural network structure composed of an input layer, a hidden layer, and an output layer. Among them, the input layer obtains a multi-modal factor library, the hidden layer obtains the parameters required in the pathfinding algorithm based on the multi-modal factor library, and the output layer combines the pathfinding algorithm to output the path. The calculation formula for the weight increment between the output layer and the hidden layer is:

[0024] Δw jk =(d0(n)-y0(n))δ j ηy j f(net k )

[0025] Among them, d0(n) and y0(n) respectively represent the expected output value and the actual output value of the neural network, δ j is the error gradient of the j-th neuron in the output layer, η is the learning rate, y j is the output of the j-th neuron in the output layer, f(net k ) is the derivative of the transfer function of the output layer. Calculate the weight increment between the input layer and the hidden layer:

[0026]

[0027] Among them, L is the number of neurons in the output layer, δ k is the error gradient of the k-th neuron in the input layer, η is the learning rate, w jk is the weight increment between the output layer and the hidden layer, x i is the input of the i-th neuron in the input layer, f(net j ) is the derivative of the transfer function of the input layer;

[0028] S222 updates the weights of the neural network through the gradient descent method and optimizes the neural network through the root mean square backpropagation algorithm.

[0029] Furthermore, the method for pre-searching the multiple sub-threads through the multi-modal pathfinding model includes:

[0030] S31 designs a distance heuristic function through the Manhattan distance function to obtain the estimated cost of the material from the current node to the target node;

[0031] S32 sorts the nodes according to the estimated cost values, constructs a priority queue, selects the node with the smallest estimated cost value from the priority queue for expansion, explores the selected node to generate its successor nodes, and applies a diversification strategy to the successor nodes of the selected node; randomly selects some successor nodes to add to the open list instead of only adding the optimal successor nodes; adjusts the heuristic function and increases randomness to explore different search paths;

[0032] S33 adds the newly generated nodes to the open list and updates their estimated cost values, and collects all the explored paths or path candidates from the open list to form an initial path set.

[0033] Further, the method for globally optimizing the initial path set of all the sub-threads includes:

[0034] S41 calculates the utility rate of each path in the sub-thread path set, and the calculation formula is:

[0035]

[0036] where, T j is the scheduling utility rate of the j-th path on the sub-thread, B U is the maximum amount of materials that the port can handle in 1h, p i is the urgency coefficient of the i-th material, h i is the processing efficiency of the i-th material, i.e., the scheduling completion time, U n is the path scheduling material set, f i is the scheduling device, Π l(i) f i is the set of devices related to the scheduling path l(i), u(f i , j) is the performance index of the device f i on the scheduling path j, U(f i , S) is the average performance index of the device f i on all the path sets S of the sub-thread, and N0 is the average time for the materials on this sub-thread to complete one scheduling;

[0037] Selects one path from each sub-thread path set to obtain multiple path combinations, and calculates the global optimal utility rate of the path combinations. The calculation formula is:

[0038]

[0039] where, T all is the global utility rate of the path combination, m is the number of sub-threads, n is the number of paths in the i-th sub-thread, is the utility rate of the j-th path in the i-th sub-thread, B is the total weight of the materials globally scheduled within 1 hour, v is the total area of the materials globally scheduled within 1 hour, |P| is the length of the j-th path in the i-th sub-thread, and r i is the number of device resources consumed by the j-th path in the i-th sub-thread;

[0040] S43 uses the path combination with the maximum global optimal utility rate as the global optimal path.

[0041] Furthermore, the method for path correction based on the path crossing and obstacle collision situations in the simulation results includes: for the situation where two sub-thread schedules appear on the crossing path, obtain the priorities of the two sub-threads. The materials in the low-priority sub-thread regard the materials in the high-priority sub-thread as obstacles and perform local path optimization; for the situation where the materials collide with the obstacle area during the scheduling process, calculate a new learning rate, adjust the model, and complete the optimization of the path.

[0042] Furthermore, the method for performing local path optimization includes:

[0043] S51 obtains the pheromone and heuristic information on the low-priority sub-thread through the ant colony algorithm, and uses the adaptive pseudo-random proportional selection rule to determine the local path optimization direction;

[0044] S52 takes the obstacle as the repulsive force source, introduces the distance factor to obtain the improved repulsive force potential field function. The expression of the repulsive force potential field function is:

[0045]

[0046] where K rep is the proportional coefficient of the repulsive force potential field, ρ(p, p obs ) is the distance between the two materials, ρ0 is the influence range of the repulsive force source, and ρ m (p, p g ) is the m-th power of the density function from the position p to the attraction center p g . Obtain the repulsive force in the local path optimization direction according to the repulsive force potential field function, combine it with the gravitational force calculated before for the path, and calculate the resultant force to complete the local path optimization.

[0047] Furthermore, the method for calculating the new learning rate includes:

[0048] S61 calculates the collision risk:

[0049]

[0050] where d min is the minimum distance between the obstacle and the material, is the projection of the material velocity in the d min direction, k t is the adjustment parameter, T is the scheduling time of the sub-thread, and d s is the safety distance of the material;

[0051] S62 obtains a new learning rate based on the collision risk, and the calculation formula is:

[0052]

[0053] where α RN is the penalty coefficient, α VTA is the sensitivity adaptively adjusted with the collision risk, k1 and k2 are the scaling factors of α RN and α VTA respectively, R is the collision risk, μ(ξ j ) is the forgetting factor of the neuron firing, and ξ j is the number of neuron firings.

[0054] In a second aspect, an embodiment of the present application further provides an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, and the executable instructions, when executed, cause the processor to execute the method steps described in the first aspect.

[0055] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, and the computer-readable storage medium stores one or more programs, and when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute the method steps described in the first aspect.

[0056] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0057] (1) The present invention provides a port material movement scheduling method based on a multimodal pathfinding algorithm. By integrating data from different sources, adaptively encoding the characteristics of data from different sources, obtaining a multimodal factor library, constructing a multimodal pathfinding algorithm model based on the multimodal factor library, integrating multi-source data, and performing encoding fusion, the influence of multi-source data on path optimization is considered, and the comprehensiveness of port material movement scheduling is improved.

[0058] (2) By decomposing the scheduling tasks into multiple sub-thread tasks, the present invention uses a multi-modal pathfinding model to pre-search the sub-threads, obtains an initial set of sub-thread paths, performs global optimization on all the initial sets of sub-thread paths, and uses the obtained global optimal solution as the multi-thread path. Compared with most methods that only optimize a single path, it is more practical. By decomposing into sub-threads, the computational complexity of a single problem can be reduced, and it can adapt to different transportation modes and material characteristics, improving the efficiency and practicality of the port material movement scheduling method. Brief Description of the Drawings

[0059] Figure 1 It is a flowchart of the steps of the port material movement scheduling method based on the multi-modal pathfinding algorithm of the present invention.

[0060] Figure 2 It is a schematic structural diagram of an electronic device in an embodiment of this specification. Detailed Embodiment

[0061] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0062] Referring to Figure 1 As shown, the present invention provides a port material movement scheduling method based on a multi-modal pathfinding algorithm, including:

[0063] S1 Obtain the satellite remote sensing image and device sensor data of the port area, perform adaptive coding on the satellite remote sensing data and device sensor data of the port area to obtain a multi-modal factor library;

[0064] In actual evaluation, obtain the satellite remote sensing image and device sensor data of the port area, extract the image data from the satellite remote sensing data as spatial modal data, extract the device, material parameters, and scheduling tasks from the device sensor data as semantic modal data, perform feature extraction on the three modal data, and obtain relevant features, including:

[0065] Spatial data: Based on the remote sensing image, complete the identification of the material area, device area, obstacle area, and berth area, including: material areas 1, 2, 3, 4, device areas 1, 2, 3, obstacle area, berth areas 1, 2, 3;

[0066] Device parameter data: Quayside crane 1, start state, quayside crane 2, start state; conveyor belt 1, in area 1, conveyor belt 2 in area 1; forklift 1, forklift 2, forklift 3;

[0067] Material parameter data: dry cargo containers, 4 boxes, with sizes of 30 feet, 25 feet, 20 feet, and 40 feet respectively, and weights of 20 tons, 17 tons, 17 tons, and 40 tons respectively; refrigerated cargo containers, 2 boxes, with sizes of 20 feet, 40 feet, 15 feet, and 30 feet respectively, and weights of 20 tons, 45 tons, 17 tons, and 22 tons respectively; the urgency of each material;

[0068] Scheduling tasks: Transport dry cargo containers to berth area 1, and transport refrigerated cargo containers to berth areas 2 and 3.

[0069] S2 Construct a multimodal pathfinding model based on the multimodal factor library;

[0070] In the actual evaluation, by setting the pathfinding method, designing the neural network structure, and constructing a multimodal pathfinding model based on the spatial data, equipment parameter data, and material parameter data in the multimodal factor library.

[0071] S3 Decompose the scheduling operation into multiple sub-threads, use the multimodal pathfinding model to pre-search the sub-threads to obtain the initial set of sub-thread paths, perform global optimization on all the initial sets of sub-thread paths, and use the obtained global optimal solution as the multi-thread path;

[0072] In the actual evaluation, divide the scheduling tasks. Based on the different destinations, divide the scheduling tasks into three types of sub-threads. The scheduling task for transporting to berth area 1 is used as sub-thread 1, the scheduling task for transporting to berth area 2 is used as sub-thread 2, and the scheduling task for transporting to berth area 3 is used as sub-thread 3;

[0073] In the actual evaluation, use the multimodal pathfinding model to pre-search sub-thread 1, sub-thread 2, and sub-thread 3 respectively. A total of 6 paths are obtained for the initial set of sub-thread 1, 12 paths for the initial set of sub-thread 2, and 7 paths for the initial set of sub-thread 3. Perform global optimization on all the initial sets of sub-thread paths based on the utility rate of global scheduling, and use the obtained global optimal solution as the multi-thread path. The multi-thread paths obtained are: Path 1 in sub-thread 1, Path 2 in sub-thread 2, and Path 3 in sub-thread 3.

[0074] S4 Simulate the multi-thread path, perform path correction based on the path crossing and obstacle collision situations in the simulation results, and use the corrected path as the mobile scheduling path;

[0075] In the actual evaluation, when modeling the obtained multi-threaded paths, it is found that path crossing occurs between path one and path two, and path three collides with the obstacle area. Based on the path crossing situation between path one and path two, it is obtained that the material urgency on path one is greater than that on path two. The material on path one is used as an obstacle, and local path re-optimization is performed on path two. Based on the situation that path three collides with the obstacle area, a new learning rate is obtained by calculating the collision risk degree, and the multi-modal multi-threaded path finding algorithm model is adjusted based on the new learning rate to obtain a re-planning of path three in thread 3. The corrected path two and path three are combined with path one as the material movement scheduling path.

[0076] In this embodiment, the method for adaptively encoding the satellite remote sensing data and equipment sensor data of the port area includes:

[0077] S11 Extract the spatial data modality from the satellite remote sensing image, including: spatial distribution, equipment, and material distribution. Extract the structured data modality from the equipment sensor data, including: equipment status report, scheduling task, and cargo parameter information;

[0078] S12 Design a recurrent neural network adaptive encoder for the spatial data modality and a fully connected layer neural network adaptive encoder for the structured data modality;

[0079] S13 Extract the encoded feature representations of the spatial data modality and the structured data modality through the corresponding auto-encoding models, and perform normalized scale matching processing on the extracted encoded feature representations to obtain a dimensionless multi-modal factor library.

[0080] In this embodiment, the method for constructing a multi-modal path finding model based on the multi-modal factor library includes:

[0081] S21 Design a path finding method, rasterize the port area, assume that the target area has an attraction to the material during material movement, and the obstacle area has a repulsion to the material. The calculation formula for the attraction is:

[0082] F at = k a d(x - x goat )

[0083] where k a is the proportionality coefficient of the gravitational potential field, and d(x - x goat ) is the distance from the current area point of the material to the target area. Calculate the repulsion of the obstacle area to the material, and the calculation formula is:

[0084]

[0085] where is the repulsive range of the obstacle, d min is the shortest distance from the material to the obstacle, K r is the proportionality coefficient of the repulsive potential field is the sensitivity of density change is the density change of the material to the shortest obstacle is the density change of the material from the current area to the target area. By calculating the resultant force at the position point where the material is located, the path finding from the starting area to the target area is completed;

[0086] S22 constructs a multi-modal path finding model based on a neural network. The construction process is as follows:

[0087] S221 constructs a multi-layer feedforward neural network structure composed of an input layer, a hidden layer, and an output layer. Among them, the input layer obtains the multi-modal factor library, the hidden layer obtains the parameters required in the path finding algorithm based on the multi-modal factor library, and the output layer outputs the path in combination with the path finding algorithm. The calculation formula for the weight increment between the output layer and the hidden layer is:

[0088] Δw jk =(d0(n)-y0(n))δ j ηy j f(net k )

[0089] where d0(n) and y0(n) respectively represent the expected output value and the actual output value of the neural network, δ j is the error gradient of the j-th neuron in the output layer, η is the learning rate, y j is the output of the j-th neuron in the output layer, f(net k ) is the derivative of the transfer function of the output layer. Calculate the weight increment between the input layer and the hidden layer:

[0090]

[0091] where L is the number of neurons in the output layer, δ k is the error gradient of the k-th neuron in the input layer, η is the learning rate, w jk is the weight increment between the output layer and the hidden layer, x i is the input of the i-th neuron in the input layer, f(net j ) is the derivative of the transfer function of the input layer;

[0092] S222 updates the neural network weights through the gradient descent method and optimizes the neural network through the root mean square backpropagation algorithm.

[0093] In this embodiment, the method for pre-searching the multiple sub-threads through the multi-modal path finding model includes:

[0094] S31 designs a distance heuristic function through the Manhattan distance function to obtain the estimated cost of the material from the current node to the target node;

[0095] S32 sorts the nodes according to the estimated cost values, constructs a priority queue, selects the node with the smallest estimated cost value from the priority queue for expansion, explores the selected node to generate its successor nodes, and applies a diversification strategy to the successor nodes of the selected node; randomly selects some successor nodes to add to the open list instead of only adding the optimal successor nodes; adjusts the heuristic function and increases randomness to explore different search paths;

[0096] S33 adds the newly generated nodes to the open list and updates their estimated cost values, and collects all the explored paths or path candidates from the open list to form an initial path set.

[0097] In this embodiment, the method for globally optimizing the initial path set of all the sub-threads includes:

[0098] S41 calculates the utility rate of each path in the sub-thread path set, and the calculation formula is:

[0099]

[0100] where, T j is the scheduling utility rate of the j-th path on the sub-thread, B U is the maximum amount of material that the port can handle in 1 hour, p i is the emergency coefficient of the i-th material, h i is the processing efficiency of the i-th material, i.e., the scheduling completion time, U n is the path scheduling material set, f i is the scheduling device, Π l(i) f i is the set of devices related to the scheduling path l(i), u(f i , j) is the performance index of the device f i on the scheduling path j, U(f i , S) is the average performance index of the device f i on all the path sets S of the sub-thread, and N0 is the average time for the materials on this sub-thread to complete one scheduling;

[0101] Selects one path from each sub-thread path set to obtain multiple path combinations, and calculates the global optimal utility rate of the path combinations. The calculation formula is:

[0102]

[0103] where, T allis the global utility rate of the path combination, m is the number of sub-threads, n is the number of paths in the i-th sub-thread, is the utility rate of the j-th path in the i-th sub-thread, B is the total weight of the materials globally scheduled within 1 hour, v is the total area of the materials globally scheduled within 1 hour, |P| is the length of the j-th path in the i-th sub-thread, r i is the number of device resources consumed by the j-th path in the i-th sub-thread;

[0104] S43 takes the path combination with the maximum global optimal utility rate as the global optimal path.

[0105] In this embodiment, the method for path correction based on the path crossing and obstacle collision situations in the simulation results includes: for the situation where two sub-threads' scheduling appears on the path in the crossing path, obtain the priorities of the two sub-threads. The materials in the sub-thread with the lower priority regard the materials in the sub-thread with the higher priority as obstacles and perform local path optimization; for the situation where the materials collide with the obstacle area during the scheduling process, calculate a new learning rate, adjust the model, and complete the optimization of the path.

[0106] In this embodiment, the method for performing local path optimization includes:

[0107] S51 obtains the pheromone and heuristic information on the sub-thread with the lower priority through the ant colony algorithm, and uses the adaptive pseudo-random proportional selection rule to determine the direction of its local path optimization;

[0108] S52 takes the obstacle as the repulsive force source, introduces the distance factor to obtain the improved repulsive force potential field function. The expression of the repulsive force potential field function is:

[0109]

[0110] where K rep is the proportional coefficient of the repulsive force potential field, ρ(p, p obs ) is the distance between the two materials, ρ0 is the influence range of the repulsive force source, ρ m (p, p g ) is the m-th power of the density function from the position p to the attraction center p g . Obtain the repulsive force in the direction of local path optimization according to the repulsive force potential field function, combine it with the gravitational force calculated before for the path, and calculate the resultant force to complete the local path optimization.

[0111] In this embodiment, the method for calculating the new learning rate includes:

[0112] S61 calculates the collision risk:

[0113]

[0114] Among them, d min is the minimum distance between the obstacle and the material, is the projection of the material velocity in the d min direction, k t is an adjustment parameter, T is the scheduling time of the sub-thread, d s is the safety distance of the material;

[0115] S62 obtains a new learning rate based on the collision risk degree, and the calculation formula is:

[0116]

[0117] Among them, α RN is the penalty coefficient, α VTA is the sensitivity adaptively adjusted with the collision risk degree, k1 and k2 are the scaling factors of α RN and α VTA respectively, R is the collision risk degree, μ(ξ j ) is the forgetting factor of neuron firing, ξ j is the number of neuron firings.

[0118] Figure 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 2 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0119] The processor, network interface, and memory can be interconnected through an internal bus, and the internal bus can be an ISA (Industry Standard Architecture, industrial standard architecture) bus, a PCI (Peripheral Component Interconnect, peripheral component interconnect standard) bus, or an EISA (Extended Industry Standard Architecture, extended industrial standard structure) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 2 only a bidirectional arrow is used in

[0120] A memory for storing programs. Specifically, the program may include program codes, and the program codes include computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0121] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an information push system for intelligent manufacturing services at the logical level. The processor executes the program stored in the memory and is specifically used to execute any one of the foregoing information push methods for intelligent manufacturing services.

[0122] The above as in this application Figure 1 The port material movement scheduling method based on the multimodal pathfinding algorithm disclosed in the embodiments shown in this application can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or the instructions in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute each method, step, and logic block diagram disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of this application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0123] The electronic device can also execute Figure 1 the port material movement scheduling method based on the multimodal pathfinding algorithm in Figure 1 and implement the functions of the embodiments shown in this application. The embodiments of this application will not be elaborated here.

[0124] An embodiment of the present application also provides a computer-readable storage medium storing one or more programs, where the one or more programs include instructions that, when executed by an electronic device including multiple application programs, perform any of the foregoing port material movement scheduling methods based on a multimodal pathfinding algorithm.

[0125] Those skilled in the art should understand that the embodiments of the present application may be provided as a method, a system, or a computer program product. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a system for implementing the specified functions in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0127] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction system that implements the specified functions in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in Figure 1 one or more of the flows Figure 1 or multiple flows and / or blocks

[0129] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0130] The memory may include non-permanent memory in the form of computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of computer-readable media.

[0131] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0132] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0134] The above content is only an example and explanation of the structure of the present invention. Those skilled in the art of this technology can make various modifications or supplements to the described specific embodiments or use similar methods for substitution. As long as they do not deviate from the structure of the invention or exceed the scope defined by this claim book, they shall fall within the protection scope of the present invention.

Claims

1. A port material movement scheduling method based on a multimodal pathfinding algorithm is characterized in that: The following steps are involved: S1 obtains a satellite remote sensing image and equipment sensor data of a port area, and adaptively encodes the satellite remote sensing data and equipment sensor data of the port area to obtain a multimodal factor library; S2 constructs a multimodal pathfinding model based on the multimodal factor library; S3 divides the scheduling job into multiple sub-threads, performs a pre-search on the multiple sub-threads through the multi-modal path-finding model, obtains an initial set of paths of all sub-threads, performs a global optimization on the initial set of paths of all sub-threads, and uses the obtained global optimal solution as the multi-thread path; The method for globally optimizing the initial set of paths of all sub-threads comprises: S41 calculates the utility rate of each path in the sub-thread path set, and the calculation formula is: in, is the scheduling utility rate of the jth path on the child thread, is the maximum amount of material that the port can handle within 1 hour, is the urgency coefficient of the i-th material, is the processing efficiency of the i-th material, i.e., the scheduling completion time, Schedule a collection of materials for a route, For dispatching equipment, For the scheduling path Related equipment collection, For scheduling path On the device performance indicators, For equipment In the child thread all paths are set The average performance index on The average time it takes to complete a scheduling of the material on this sub-thread; S42 selects a path from each sub-thread path set to obtain multiple path combinations, and calculates the global optimal utility rate of the path combination. The calculation formula is: in, is the global utility rate of the path combination, is the number of child threads, For the The number of paths in the subthreads, For the The first of the subthreads The utility rate of the path, is the total weight of materials dispatched globally within 1 hour, is the total area of ​​materials scheduled globally within 1 hour, For the The first of the subthreads The length of the path, For the In the subthread The number of device resources consumed by each path; S43 takes the path combination with the maximum value of the global optimal utility rate as the global optimal path; S4 simulates the multi-thread path, corrects the path based on the path intersection and collision obstacle conditions in the simulation results, and uses the corrected path as the mobile scheduling path.

2. The port material movement scheduling method based on multimodal pathfinding algorithm according to claim 1 is characterized in that: The method for adaptively encoding the port area satellite remote sensing data and equipment sensor data comprises: S11 extracts spatial data modalities from satellite remote sensing images, including spatial distribution, equipment, and material distribution, and extracts structured data modalities from equipment sensor data, including equipment status reports, scheduling tasks, and cargo parameter information; S12 designs a recurrent neural network adaptive encoder for spatial data modality and a fully connected layer neural network adaptive encoder for structured data modality; S13 extracts encoded feature representations of the spatial data modality and the structured data modality through the corresponding autoencoder model, performs normalized scale matching processing on the extracted encoded feature representations, and obtains a dimensionless multimodal factor library.

3. The port material movement scheduling method based on multimodal pathfinding algorithm according to claim 1 is characterized in that: The method for constructing a multimodal pathfinding model based on the multimodal factor library comprises: S21 designed a pathfinding method and gridded the port area. It was assumed that the target area had an attractive force on the material during material movement, and the obstacle area had a repulsive force on the material. The formula for calculating the attractive force was: in, is the proportional coefficient of the gravitational potential field, is the distance from the current area point to the target area of ​​the material, and the repulsion of the obstacle area on the material is calculated. The calculation formula is: in, is the repulsive force range of the obstacle, is the minimum distance between the obstacle and the material, is the positive proportionality coefficient of the repulsive potential field, is the sensitivity to intensive changes, For the dense change of materials to the shortest obstacle, For the dense change of materials from the current area to the target area, the path search from the starting area to the target area is completed by calculating the resultant force of the material location points; S22 builds a multimodal pathfinding model based on a neural network. The construction process is as follows: S221 constructs a multi-layer feedforward neural network structure consisting of an input layer, a hidden layer, and an output layer, wherein the input layer obtains a multimodal factor library, the hidden layer obtains the parameters required in the pathfinding algorithm based on the multimodal factor library, and the output layer outputs the path in combination with the pathfinding algorithm. The calculation formula for the weight increment of the output layer and the hidden layer is: in, and They represent the expected output value and the actual output value of the neural network respectively. is the error gradient of the jth neuron in the output layer, is the learning rate, is the output of the jth neuron in the output layer, The derivative of the output layer transfer function is used to calculate the weight increments of the input layer and the hidden layer: in, is the number of neurons in the output layer, The input layer The error gradient of each neuron is is the learning rate, is the weight increment of the output layer and the hidden layer, is the input of the ith neuron in the input layer, The derivative of the transfer function of the input layer; S222 updates the neural network weights through the gradient descent method and optimizes the neural network through the root mean square back propagation algorithm.

4. The port material movement scheduling method based on multimodal pathfinding algorithm according to claim 1 is characterized in that: The method of pre-searching the multiple sub-threads through the multimodal pathfinding model includes: S31 designs a distance heuristic function through the Harmanton distance function to obtain the estimated cost of the material from the current node to the target node; S32 sorts the nodes according to the estimated cost values, builds a priority queue, selects the node with the smallest estimated cost value from the priority queue for expansion, explores the selected node, generates its successor node, and applies a diversification strategy to the successor nodes of the selected node; randomly selects some successor nodes to add to the open list instead of adding only the best successor node; adjusts the heuristic function and increases randomness to explore different search paths; S33 adds the newly generated nodes to the open list and updates their estimated cost values, and collects all explored paths or path candidates from the open list to form an initial path set.

5. The port material movement scheduling method based on multimodal pathfinding algorithm according to claim 1 is characterized in that: The method for correcting a path based on path intersections and collisions with obstacles in simulation results includes: for a situation where two sub-threads are scheduled on an intersection path, the priorities of the two sub-threads are obtained, and the materials in the low-priority sub-thread regard the materials in the high-priority sub-thread as obstacles to perform local path optimization; for a situation where materials collide with an obstacle area during scheduling, a new learning rate is calculated, the model is adjusted, and the path optimization is completed.

6. The port material movement scheduling method based on multi-modal path finding algorithm according to claim 5 is characterized in that: The method for performing local path optimization comprises: S51 obtains pheromone and heuristic information on low-priority subthreads through ant colony algorithm, and uses adaptive pseudo-random proportional selection rule to determine its local path optimization direction; S52 takes the obstacle as the repulsive force source and introduces the distance factor to obtain an improved repulsive force potential field function. The expression of the repulsive force potential field function is: in, is the positive proportionality coefficient of the repulsive potential field, is the distance between the two materials, is the influence range of the repulsive force source, For location To the center of attraction The density function of The repulsive force in the direction of local path optimization is obtained according to the repulsive potential field function, and the resultant force is calculated by combining it with the gravitational force calculated before the path to complete the local path optimization.

7. The port material movement scheduling method based on multi-modal path finding algorithm according to claim 5 is characterized in that: The method for calculating a new learning rate comprises: S61 calculates the collision risk: in, is the minimum distance between the obstacle and the material, The material speed is The projection in the direction, To adjust the parameters, T is the scheduling time of the child thread, The safe distance for materials; S62 obtains a new learning rate based on the collision risk, and the calculation formula is: in, is the penalty coefficient, is the sensitivity that is adaptively adjusted according to the collision risk. and They are and The scaling factor is R, which is the collision risk. The forgetting factor released by neurons, is the number of neuron firings.

8. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the method described in any one of claims 1 to 7.

9. A computer-readable storage medium storing one or more programs, which, when executed by an electronic device including a plurality of application programs, enables the electronic device to execute the method according to any one of claims 1 to 7.

Images