Port berth allocation method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of shipping ports, and discloses a port berth allocation method and device, computer equipment and a storage medium, and the method comprises the steps: obtaining a berth of a target port and first multi-modal data of a ship; processing the first multi-modal data through a preset time sequence model to obtain predicted arrival time of the ship; berthing priority data and departure time data of the ship are acquired, and a priority coefficient corresponding to the ship is calculated by using the predicted arrival time, the berthing priority data and the departure time data; acquiring real-time second multi-modal data of the target port, and processing the first multi-modal data, the priority coefficient and the second multi-modal data through a preset multi-modal large model to obtain a real-time berth allocation result corresponding to the target port; through the mode, the waiting time of ship service is shortened, the production operation efficiency of a port is improved, the resource waste of manpower and equipment of the port is reduced, and the operation efficiency of the port is improved.

Description

Technical Field

[0001] The present invention relates to the field of shipping technology, and in particular to a port berth allocation method, device, computer equipment and storage medium. Background Art

[0002] In the current shipping port management, the typical planning method is often used for logistics organization, that is, all operation plans are based on accurate prediction of future traffic flow, so that the operation strategy can be formulated in a targeted manner, so that the allocation of port loading and unloading capacity and expected demand can be matched as best as possible. However, traditional port berth management often generates corresponding port ship scheduling results based on empirical rules and static plans, which makes it difficult to quickly adjust the scheduling plan according to the changes in real-time demand, and thus cannot cope with the ever-changing shipping environment and increasingly complex data situations, which ultimately leads to a reduction in the overall utilization efficiency of port berths and a reduction in port throughput capacity, which in turn leads to a reduction in the overall operation efficiency of the port. Summary of the invention

[0003] The embodiments of the present invention provide a port berth allocation method, device, computer equipment and storage medium method and device, which can improve the port berth utilization rate and port operation efficiency.

[0004] In a first aspect, an embodiment of the present application provides a port berth allocation method, comprising: acquiring first multimodal data of berths and ships at a target port; processing the first multimodal data through a preset time series model to obtain a predicted arrival time of the ship; acquiring berthing priority data and departure time data of the ship, and using the predicted arrival time, the berthing priority data and the departure time data to calculate a priority coefficient corresponding to the ship; acquiring real-time second multimodal data of the target port, processing the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain a real-time berth allocation result corresponding to the target port.

[0005] Optionally, the step of processing the first multimodal data through a preset time series model to obtain the predicted arrival time of the ship includes: preprocessing the first multimodal data and converting the first multimodal data into corresponding multivariate time series data; embedding the multivariate time series data into the historical observation value sequence of the ship's arrival time through an embedding layer in the time series model to obtain an embedded feature time series; processing the embedded feature time series using a preset first number of converter modules in the time series model to obtain a predicted arrival time feature vector; processing the predicted arrival time feature vector through a projection layer in the time series model to obtain the predicted arrival time of the ship.

[0006] Optionally, the step of using a preset first number of converter modules in the timing model to process the embedded feature time series to obtain a predicted arrival time feature vector includes: inputting the embedded feature time series to the adaptive multi-scale dual attention layer of the converter module for processing to obtain a first predicted time feature sequence containing an attention distribution; using the instance normalization submodule of the converter module to normalize the first predicted time feature sequence to obtain a first normalized feature sequence; inputting the first normalized feature sequence to a preset feedforward neural network of the converter module, and receiving a second predicted time feature sequence output by the feedforward neural network; using the layer normalization submodule of the converter module to normalize the second predicted time feature sequence to obtain a second normalized feature sequence; and looping the steps of inputting the embedded feature time series to the adaptive multi-scale dual attention submodule of the converter module for processing to obtaining the second normalized feature sequence for a preset number of times to obtain the predicted arrival time feature vector, wherein the preset number of times is equal to the preset first number.

[0007] Optionally, the step of inputting the embedded feature time series to the adaptive multi-scale dual attention layer of the converter module for processing to obtain a first predicted time feature sequence containing attention distribution includes: routing the embedded feature time series to time series segmentation submodules of different scales in the multi-scale segmentation module of the adaptive multi-scale dual attention layer through multi-scale routing of the adaptive multi-scale dual attention layer, and generating a sequence weight value of the embedded feature time series; using the time series segmentation submodule to perform block segmentation processing on the embedded feature time series to obtain a time series segmentation block set of time series blocks of different sizes; sending the time series segmentation set to the multi-scale converter submodule of the adaptive multi-scale dual attention layer, performing dual attention processing on the time series segmentation set through the multi-scale converter submodel, and extracting a multi-scale feature time series; performing adaptive feature weighted aggregation processing on the sequence weight value and the multi-scale feature time series through the multi-scale aggregation submodule of the adaptive multi-scale dual attention layer to obtain the first predicted time feature sequence containing multi-scale features.

[0008] Optionally, the step of processing the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain a real-time berth allocation result corresponding to the target port includes: processing the first multimodal data and the second multimodal data through a multimodal encoder of the multimodal large model to obtain a first multimodal feature representation vector; inputting the first multimodal feature representation vector after spatial alignment processing into the main network of the multimodal large model, and performing semantic understanding, semantic reasoning and semantic decision processing in sequence through the main network of the multimodal large model to obtain a second multimodal feature representation vector; and generating the real-time berth allocation results corresponding to different modalities according to the priority coefficient and the second multimodal feature representation vector through the multimodal generator of the multimodal large model.

[0009] Optionally, after the step of processing the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain the real-time berth allocation result corresponding to the target port, it also includes: sending the real-time berth allocation result to the control center of the target port; monitoring the berth allocation execution result of the real-time berth allocation result at the target port through a preset monitoring method.

[0010] Optionally, the step of monitoring the execution result of the real-time berth allocation result at the target port through a preset monitoring method includes: obtaining multimodal monitoring data of the target port through the preset monitoring method, converting the multimodal monitoring data into a multimodal monitoring feature vector, wherein the multimodal monitoring feature vector includes a first monitoring image feature vector and a first monitoring text feature vector; processing the first monitoring image feature vector using a preset image encoder, representing the berth and the ship using a bounding box and a cropped image, and obtaining a second monitoring image feature vector; processing the second monitoring image feature vector and the first monitoring text feature vector in turn through at least two alternating cross-attention layers and self-attention layers in a preset guided decoder to obtain an association relationship feature vector between the berth and the ship; and converting the association relationship feature vector into the berth allocation execution result.

[0011] In the second aspect, an embodiment of the present application provides a port berth allocation device, comprising: a multimodal data acquisition module, used to acquire first multimodal data of berths and ships at a target port; a predicted arrival time module, used to process the first multimodal data through a preset time series model to obtain a predicted arrival time of the ship; a priority coefficient module, used to acquire berthing priority data and departure time data of the ship, and use the predicted arrival time, the berthing priority data and the departure time data to calculate the priority coefficient corresponding to the ship; a berth allocation result module, used to acquire real-time second multimodal data of the target port, and process the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain a real-time berth allocation result corresponding to the target port.

[0012] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store computer programs; and the processor is used to implement the steps of the port berth allocation method as described above when executing the program stored in the memory.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the port berth allocation method as described above are implemented.

[0014] The embodiments of the present application provide a port berth allocation method, apparatus, computer equipment and storage medium, which obtain first multimodal data of berths and ships at a target port; process the first multimodal data through a preset time series model to obtain a predicted arrival time of the ship; obtain berthing priority data and departure time data of the ship, and use the predicted arrival time, berthing priority data and departure time data to calculate the priority coefficient corresponding to the ship; obtain real-time second multimodal data of the target port, process the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain a real-time berth allocation result corresponding to the target port; through the above-mentioned method, the waiting time for ship service is shortened, the efficiency of port production operations is improved, the waste of manpower and equipment resources of the port is reduced, and the operation efficiency of the port is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0016] Figure 1 It is a schematic diagram of the application environment of a port berth allocation method provided by the present application.

[0017] Figure 2 It is a schematic diagram of the execution flow of the port berth allocation method provided in an embodiment of the present application.

[0018] Figure 3 It is a schematic diagram of the execution flow of obtaining the predicted arrival time in the port berth allocation method provided in the embodiment of the present application.

[0019] Figure 4 It is a schematic diagram of the execution flow of obtaining an embedded feature time series in the port berth allocation method provided in an embodiment of the present application.

[0020] Figure 5 It is a schematic diagram of the execution flow of the adaptive multi-scale dual attention layer in the port berth allocation method provided in an embodiment of the present application.

[0021] Figure 6 This is an example diagram of the network structure of the adaptive multi-scale dual attention layer in the port berth allocation method provided in an embodiment of the present application.

[0022] Figure 7 It is a schematic diagram of the execution flow of generating real-time berth allocation results of different modes in the port berth allocation method provided in an embodiment of the present application.

[0023] Figure 8 It is an example diagram of real-time berth allocation results of different modes generated in the port berth allocation method provided in an embodiment of the present application.

[0024] Fig. 9 It is a schematic diagram of the execution flow of obtaining the berth allocation execution result in the port berth allocation method provided in the embodiment of the present application.

[0025] Fig.10 This is an example diagram of real-time analysis of berth allocation execution results in the port berth allocation method provided in an embodiment of the present application.

[0026] Fig.11 It is a schematic diagram of the system structure of the port berth allocation device provided in an embodiment of the present application.

[0027] Fig.12 A schematic diagram of the structure of a computer device according to an embodiment of the present application.

[0028] Fig.13 A schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0030] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0031] In the embodiments of the present application, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0032] Moreover, the terms "comprises," "comprising," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0033] In the description of the embodiments of the present application, words such as "example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "example" or "for example" in the embodiments of the present application is not to be interpreted as being more preferred or having more advantages than another embodiment or design. The use of words such as "example" or "for example" is intended to present relative concepts in a clear manner.

[0034] In addition, the "plurality" in the embodiments of the present application refers to two or more than two. In view of this, in the embodiments of the present application, "plurality" can also be understood as "at least two". "At least one" can be understood as one or more, for example, one, two or more. For example, including at least one means including one, two or more, and there is no limit on which ones are included. For example, including at least one of A, B and C, then A, B, C, A and B, A and C, B and C, or A, B and C can be included.

[0035] See also Figure 1 , Figure 1 This is a schematic diagram of the application environment of a port berth allocation method provided by the present application. Figure 1 As shown, the executor of the port berth allocation method provided in the embodiment of the present application is generally an electronic device with certain computing capabilities, such as a computer device. In some possible implementations, the port berth allocation method can be implemented by a processor calling a computer-readable instruction stored in a memory. Figure 1 The computer device in the term can be a server. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. It can be understood that Figure 1 The number of computer devices in the figure is only for reference and can be expanded to any number according to actual needs.

[0036] An embodiment of the present application provides a port berth allocation method, Figure 2 It is a schematic diagram of the execution flow of the port berth allocation method provided in the embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in the order shown in the embodiment or the figure or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 2 As shown, the port berth allocation method may include the following steps S1 to S4.

[0037] S1. Acquire first multimodal data of berths and ships at a target port.

[0038] Among them, multimodal data refers to a data set containing multiple data types or sources, and these data types can include various forms such as text, images, videos, and audio. In multimodal data, different types of data can be interrelated or complementary to each other, thereby providing more comprehensive and rich information. The multimodal data of the berths and ships of the target port obtained in the embodiment of the present application can be weather data, hydrological data, etc. of the geographical location of the port, and can also be the location distribution data of the berth in the port, the monitoring video data of the berth, the real-time loading and unloading monitoring data of the berth, etc., and can also be the real-time navigation data of the ship, the hydrological data of the area where the ship is located, the weather data, etc., the aforementioned multimodal data are associated with the berths and ships of the port, which will affect the utilization efficiency of the berths and the port operation efficiency after the port berth allocation, and the berth allocation result that improves the utilization efficiency of the port berths and the port operation efficiency can be obtained by analyzing and processing the aforementioned multimodal data.

[0039] S2. Process the first multimodal data using a preset time series model to obtain a predicted arrival time of the ship.

[0040] See also Figure 3 , Figure 3 : is a schematic diagram of the execution flow of obtaining the predicted arrival time in the port berth allocation method provided in the embodiment of the present application, such as Figure 3 As shown, the following steps S21 to S24 may be included.

[0041] S21. Preprocess the first multimodal data and convert the first multimodal data into corresponding multivariate time series data.

[0042] S22. The multivariate time series data is embedded into the historical observation value sequence of the ship's arrival time through the embedding layer in the time series model to obtain an embedded feature time series.

[0043] S23: Use a preset first number of converter modules in the time series model to process the embedded feature time series to obtain a predicted arrival time feature vector.

[0044] As an implementation mode, the converter module in the above step S23 is constructed based on the Transformer neural network. The multi-head attention mechanism in the Transformer helps the model to focus on different parts of the input sequence at the same time, so as to better capture the relationship between different features. For the prediction of ship arrival time, the multi-head attention mechanism helps to discover key time dependencies and complex associations between features.

[0045] Specifically, Figure 4 is a schematic diagram of the execution flow of obtaining an embedded feature time series in the port berth allocation method provided in the embodiment of the present application, such as Figure 4As shown, the above step S23 may specifically include the following steps S231 to S235.

[0046] S231, input the embedded feature time series to the adaptive multi-scale dual attention layer of the converter module for processing to obtain a first predicted time feature sequence containing attention distribution.

[0047] As an implementation, see Figure 5 , Figure 5 is a schematic diagram of the execution flow of the adaptive multi-scale dual attention layer in the port berth allocation method provided in the embodiment of the present application, such as Figure 5 As shown, it can specifically include the following steps S2311 to S2314.

[0048] S2311. Through the multi-scale routing of the adaptive multi-scale dual attention layer, the embedded feature time series is routed to the time series segmentation sub-modules of different scales in the multi-scale segmentation module of the adaptive multi-scale dual attention layer, and the sequence weight value of the embedded feature time series is generated.

[0049] Among them, multiscale routing refers to the process of using multiple scales or different levels of features in a neural network to transmit information and make decisions. It is usually used to process input data with multiple hierarchies or multiple resolutions to better capture different information and features in the data. Through multiscale routing, the neural network model can focus on fine-grained local features and global features at the same time, thereby improving the model's understanding and representation of input data, and helping to enhance the model's ability to model complex data.

[0050] As an implementation method, seasonal decomposition can also be added to multi-scale routing to enhance the multi-scale routing's ability to handle time dynamics. Seasonal decomposition is a commonly used technique in time series analysis, which is used to decompose time series data into three parts: trend, seasonality, and residual, so as to better understand and analyze different components in the data. Trend is the long-term changing part of time series data, reflecting the overall trend of data changes on a longer time scale. Trend usually describes the overall growth or decline of data, which can help predict the future trend of data. Seasonality is a periodic repetitive pattern in time series data, usually appearing on a shorter time scale. The seasonal component shows the repetitive changes in data within a specific time period, such as weekly, monthly or annual seasonal changes. The residual part is the part of the data that cannot be explained by trend and seasonality except for trend and seasonality. The residual contains random fluctuations and noise in the data, which can help detect whether the model can fully capture the characteristics of the data.

[0051] S2312. Use the time series segmentation submodule to perform block segmentation processing on the embedded feature time series to obtain a time series segmentation block set of time series blocks of different sizes.

[0052] Specifically, the time series segmentation submodule contains M segmentation blocks, each segmentation block size S corresponds to a block segmentation operation, and the set of M blocks S = {S1, S2, ..., S M}, for the input time series X = {x1, x2, ..., x T}∈R T×N Each block segmentation operation with a block size of S divides X into P blocks, namely {X1,X2,...,X p}, where the calculation formula for the number of blocks P is:

[0053] S2313. Send the time series segmentation set to the multi-scale converter sub-module of the adaptive multi-scale dual attention layer, perform dual attention processing on the time series segmentation set through the multi-scale converter sub-model, and extract the multi-scale feature time series.

[0054] S2314. Perform adaptive feature weighted aggregation processing on the sequence weight value and the multi-scale feature time series through the multi-scale aggregation sub-module of the adaptive multi-scale dual attention layer to obtain a first predicted time feature sequence containing multi-scale features.

[0055] As an example, see Figure 6 , Figure 6 is an example diagram of the network structure of the adaptive multi-scale dual attention layer in the port berth allocation method provided in the embodiment of the present application, such as Figure 6 As shown, the embedded feature time series is sent by multi-scale routing to the time series segmentation submodules of different scales in the multi-scale segmentation module of the adaptive multi-scale dual attention layer, and the sequence weight value of the embedded feature time series is generated. M segmentation results can be obtained through the multi-scale segmentation module, and then the M segmentation results are subjected to feature extraction through the dual attention submodule in the multi-scale Transformer submodule. Finally, the extracted features and weights are aggregated through the multi-scale aggregation submodule to obtain the corresponding predicted time feature sequence.

[0056] S232: Use the instance normalization submodule of the converter module to normalize the first prediction time feature sequence to obtain a first normalized feature sequence.

[0057] S233, inputting the first normalized feature sequence into a preset feedforward neural network of the converter module, and receiving a second predicted time feature sequence output by the feedforward neural network.

[0058] S234. Use the layer normalization submodule of the converter module to normalize the second prediction time feature sequence to obtain a second normalized feature sequence.

[0059] S235, loop input the embedded feature time series to the adaptive multi-scale dual attention sub-module of the converter module for processing until a preset number of steps between the second normalized feature sequence is obtained, and a predicted arrival time feature vector is obtained, wherein the preset number of times is equal to the preset first number.

[0060] S24. Process the predicted arrival time feature vector through the projection layer in the time series model to obtain the predicted arrival time of the ship.

[0061] As an implementation method, the projection layer in the above step S24 can be a fully connected neural network. In the fully connected neural network, the connection between each pair of adjacent layers of neurons is fully connected, that is, each neuron is connected to all neurons in the previous layer. The fully connected neural network can transmit information at each layer of the network, which helps to integrate the global information in the sequence and helps the network to better understand the structure and pattern of the entire sequence, thereby improving the accuracy of the prediction. The fully connected neural network can learn feature representations at different levels, which helps to extract diverse features from long sequence data. This diverse feature learning helps the model better understand the complex patterns and regularities in the sequence.

[0062] S3. Obtain the berthing priority data and the departure time data of the ship, and calculate the priority coefficient corresponding to the ship using the predicted arrival time, the berthing priority data and the departure time data.

[0063] Priority coefficients refer to coefficients used to quantify the importance of different factors or objects in decision-making or optimization problems. They are used to guide models or algorithms on how to allocate weights or priorities when dealing with multiple factors, so as to make decisions or allocate resources more effectively. Priority can be reasonably allocated to the obtained multimodal data. Setting different priority coefficients can make the obtained ship arrival time more accurate.

[0064] S4. Acquire the real-time second multimodal data of the target port, process the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model, and obtain a real-time berth allocation result corresponding to the target port.

[0065] As an implementation, see Figure 7 , Figure 7 is a schematic diagram of the execution flow of generating real-time berth allocation results of different modes in the port berth allocation method provided in the embodiment of the present application, such as Figure 7 As shown, it may specifically include the following steps S41 to S43.

[0066] S41. Process the first multimodal data and the second multimodal data through a multimodal encoder of a multimodal large model to obtain a first multimodal feature representation vector.

[0067] S42, the first multimodal feature representation vector after input space alignment processing is input to the main network of the multimodal large model, and semantic understanding, semantic reasoning and semantic decision processing are performed in sequence through the main network of the multimodal large model to obtain a second multimodal feature representation vector.

[0068] S43. Generate real-time berth allocation results corresponding to different modes according to the priority coefficient and the second multimodal feature representation vector through the multimodal generator of the multimodal large model.

[0069] As an example, see Figure 8 , Figure 8 It is an example diagram of generating real-time berth allocation results of different modalities in the port berth allocation method provided in an embodiment of the present application. Multimodal video data, image data, and audio data are input into the multimodal encoding process, and then the corresponding processing results are projected to the main network of the large language model through the input projector for processing, and semantic understanding, semantic reasoning, and decision calculation are performed, etc. Then, the processing results output by the main network of the large language model are projected to the multimodal generator through the output projector, and the corresponding real-time berth allocation results of different modalities are output through the multimodal generator, for example, it can be a berth allocation picture description, it can also be a berth allocation text description, and it can also be a berth allocation audio description.

[0070] As an implementation mode, after the above step S4 is executed, the execution result of the generated berth allocation result can also be monitored accordingly. First, the real-time berth allocation result is sent to the control center of the target port. Then, the execution result of the berth allocation of the real-time berth allocation result at the target port is monitored by a preset monitoring method.

[0071] As an implementation, please continue to see Fig. 9 , Fig. 9 It is a schematic diagram of the execution flow of obtaining the berth allocation execution result in the port berth allocation method provided in the embodiment of the present application, that is, the process of monitoring the real-time berth allocation result at the target port through the preset monitoring method, which can specifically include the following steps S51 to S55.

[0072] S51. Acquire multimodal monitoring data of a target port through a preset monitoring method, and convert the multimodal monitoring data into a multimodal monitoring feature vector, wherein the multimodal monitoring feature vector includes a first monitoring image feature vector and a first monitoring text feature vector.

[0073] S52: Use a preset image encoder to process the monitoring image feature vector, use a bounding box and the cropped image to represent the berth and the ship, and obtain a second monitoring image feature vector.

[0074] S53, sequentially processing the second monitoring image feature vector and the first monitoring text feature vector through at least two alternating cross-attention layers and self-attention layers in a preset guided decoder to obtain a correlation relationship feature vector between the berth and the ship.

[0075] S54: Convert the association relationship feature vector into a berth allocation execution result.

[0076] As an example, see Fig.10 , Fig.10 is an example diagram of real-time analysis of the berth allocation execution results in the port berth allocation method provided in the embodiment of the present application, such as Fig.10 As shown, the image and trajectory feature vectors are obtained from the real-time port berth and ship monitoring images through a preset image encoder, and a multimodal prompt is obtained according to the berth allocation result, and the multimodal prompt is converted into a corresponding multimodal prompt feature vector using a multimodal encoder, and a corresponding guide feature vector is generated. Then, the corresponding text feature vector and image feature vector are processed in turn through a preset number of cross-attention modules and sub-attention modules to obtain the final berth allocation execution result.

[0077] The port berth allocation method provided in the embodiment of the present application utilizes a multimodal large language model to process multimodal data associated with port berths and ships, thereby strengthening the synergy between berth allocation and ship scheduling, thereby achieving the purpose of improving the quality of transportation services, shortening the waiting time for ship services, and improving the efficiency of port production operations, and then formulating a port ship scheduling production operation plan with high real-time performance and excellent efficiency.

[0078] Please continue reading Fig.11 , an embodiment of the present application provides a port berth allocation device, Fig.11 : is a schematic diagram of the system structure of the port berth allocation device provided in the embodiment of the present application, such as Fig.11As shown, the port berth allocation device 80 includes: a multimodal data acquisition module 81, a predicted arrival time module 82, a priority coefficient module 83 and a berth allocation result module 84. Among them, the multimodal data acquisition module 81 is used to obtain the first multimodal data of the berth and the ship of the target port. The predicted arrival time module 82 is used to process the first multimodal data through a preset time series model to obtain the predicted arrival time of the ship. The priority coefficient module 83 is used to obtain the berthing priority data and departure time data of the ship, and use the predicted arrival time, berthing priority data and departure time data to calculate the priority coefficient corresponding to the ship. The berth allocation result module 84 is used to obtain the real-time second multimodal data of the target port, and process the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain the real-time berth allocation result corresponding to the target port.

[0079] As an implementation mode, the predicted arrival time module 82 is specifically used to preprocess the first multimodal data and convert the first multimodal data into corresponding multivariate time series data; embed the multivariate time series data into the historical observation value sequence of the ship's arrival time through the embedding layer in the time series model to obtain an embedded feature time series; use a preset first number of converter modules in the time series model to process the embedded feature time series to obtain a predicted arrival time feature vector; process the predicted arrival time feature vector through the projection layer in the time series model to obtain the predicted arrival time of the ship.

[0080] As an implementation mode, the predicted arrival time module 82 is also specifically used to input the embedded feature time series to the adaptive multi-scale dual attention layer of the converter module for processing to obtain a first predicted time feature sequence containing attention distribution; use the instance normalization submodule of the converter module to normalize the first predicted time feature sequence to obtain a first normalized feature sequence; input the first normalized feature sequence to the preset feedforward neural network of the converter module, and receive the second predicted time feature sequence output by the feedforward neural network; use the layer normalization submodule of the converter module to normalize the second predicted time feature sequence to obtain a second normalized feature sequence; cyclically input the embedded feature time series to the adaptive multi-scale dual attention submodule of the converter module for processing until the second normalized feature sequence is obtained for a preset number of steps to obtain a predicted arrival time feature vector, wherein the preset number of times is equal to the preset first number.

[0081] As an implementation mode, the predicted arrival time module 82 is also specifically used to route the embedded feature time series to the time series segmentation submodules of different scales in the multiscale segmentation module of the adaptive multiscale dual attention layer through the multiscale routing of the adaptive multiscale dual attention layer, and generate a sequence weight value of the embedded feature time series; use the time series segmentation submodule to perform block segmentation processing on the embedded feature time series to obtain a time series segmentation block set of time series blocks of different sizes; send the time series segmentation set to the multiscale converter submodule of the adaptive multiscale dual attention layer, perform dual attention processing on the time series segmentation set through the multiscale converter submodel, and extract the multiscale feature time series; perform adaptive feature weighted aggregation processing on the sequence weight value and the multiscale feature time series through the multiscale aggregation submodule of the adaptive multiscale dual attention layer to obtain a first predicted time feature sequence containing multiscale features.

[0082] As an implementation mode, the berth allocation result module 84 is also specifically used to process the first multimodal data and the second multimodal data through the multimodal encoder of the multimodal large model to obtain a first multimodal feature representation vector; input the first multimodal feature representation vector after spatial alignment processing to the main network of the multimodal large model, and perform semantic understanding, semantic reasoning and semantic decision processing in sequence through the main network of the multimodal large model to obtain a second multimodal feature representation vector; through the multimodal generator of the multimodal large model, real-time berth allocation results corresponding to different modalities are generated according to the priority coefficient and the second multimodal feature representation vector.

[0083] As an embodiment, the port berth allocation device 80 also includes a berth allocation result monitoring module 85, which is specifically used to send real-time berth allocation results to the control center of the target port; and monitor the berth allocation execution results of the real-time berth allocation results at the target port through a preset monitoring method.

[0084] As an implementation mode, the berth allocation result monitoring module 85 is also specifically used to obtain multimodal monitoring data of the target port through a preset monitoring method, convert the multimodal monitoring data into a multimodal monitoring feature vector, the multimodal monitoring feature vector includes a first monitoring image feature vector and a first monitoring text feature vector; use a preset image encoder to process the monitoring image feature vector, use a bounding box and a cropped image to represent the berth and the ship, and obtain a second monitoring image feature vector; process the second monitoring image feature vector and the first monitoring text feature vector in turn through at least two alternating cross-attention layers and self-attention layers in a preset guided decoder to obtain an association relationship feature vector between the berth and the ship; convert the association relationship feature vector into a berth allocation execution result.

[0085] An embodiment of the present application provides a computer device, see Fig.12 As shown, the computer device 90 includes a processor 91, a communication interface 92, a memory 93 and a communication bus 94, wherein the processor 91, the communication interface 92 and the memory 93 communicate with each other via the communication bus 94; the memory 93 is used to store computer programs; the processor 91 is used to implement the steps of the above-mentioned port berth allocation method when executing the program stored in the memory 93.

[0086] The processor 91 may also be a CPU (Central Processing Unit). The processor 91 may also be an integrated circuit chip having signal processing capabilities. The processor 91 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0087] The memory 93 may be, for example, a read-only memory (ROM), and the processor 91 and the memory 93 are connected to each other via a communication bus 94. The communication interface 92 is also connected to the communication bus 94.

[0088] See also Fig.13 , Fig.13 It is a structural diagram of a computer-readable storage medium of an embodiment of the present application. The computer-readable storage medium 100 of the embodiment of the present application stores a computer program 101 that can implement all the above methods. The computer program 101 is a computer executable instruction or a computer program, wherein the computer program 101 can be stored in the above computer-readable storage medium in the form of a software product, and can include several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The computer-readable storage medium 100 can be a non-volatile storage medium or a volatile storage medium. The aforementioned computer-readable storage medium 100 can include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0090] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

[0091] The above is only an implementation method of the present application. It should be pointed out that a person skilled in the art can make improvements without departing from the inventive concept of the present application, but these improvements are within the scope of protection of the present application.

Claims

1. A port berth allocation method, characterized in that: The method comprises: Acquire the first multimodal data of berths and ships at the target port; Processing the first multimodal data by using a preset time series model to obtain a predicted arrival time of the ship; Acquire the berthing priority data and the port departure time data of the ship, and calculate the priority coefficient corresponding to the ship using the predicted port arrival time, the berthing priority data and the port departure time data; The real-time second multimodal data of the target port is acquired, and the first multimodal data, the priority coefficient and the second multimodal data are processed by a preset multimodal macro model to obtain a real-time berth allocation result corresponding to the target port.

2. The port berth allocation method according to claim 1, characterized in that: The step of processing the first multimodal data by using a preset time series model to obtain the predicted arrival time of the ship includes: Preprocessing the first multimodal data to convert the first multimodal data into corresponding multivariate time series data; The multivariate time series data is embedded into the historical observation value sequence of the ship's arrival time through the embedding layer in the time series model to obtain an embedded feature time series; Process the embedded feature time series using a preset first number of converter modules in the time series model to obtain a predicted arrival time feature vector; The predicted arrival time of the ship is obtained by processing the predicted arrival time feature vector through the projection layer in the time series model.

3. The port berth allocation method according to claim 2, characterized in that: The step of using a preset first number of converter modules in the timing model to process the embedded feature time series to obtain a predicted arrival time feature vector comprises: Inputting the embedded feature time series to the adaptive multi-scale dual attention layer of the converter module for processing to obtain a first predicted time feature sequence including attention distribution; Using the instance normalization submodule of the converter module to normalize the first predicted time feature sequence to obtain a first normalized feature sequence; Inputting the first normalized feature sequence into a preset feedforward neural network of the converter module, and receiving a second predicted time feature sequence output by the feedforward neural network; Using the layer normalization submodule of the converter module to normalize the second predicted time feature sequence to obtain a second normalized feature sequence; The step of looping the input of the embedded feature time series to the adaptive multi-scale dual attention submodule of the converter module for processing until the second normalized feature sequence is obtained is repeated a preset number of times to obtain the predicted arrival time feature vector, wherein the preset number of times is equal to the preset first number.

4. The port berth allocation method according to claim 3, characterized in that: The step of inputting the embedded feature time series to the adaptive multi-scale dual attention layer of the converter module for processing to obtain a first predicted time feature series including attention distribution comprises: Routing the embedded feature time series to the time series segmentation submodules of different scales in the multiscale segmentation module of the adaptive multiscale dual attention layer through the multiscale routing of the adaptive multiscale dual attention layer, and generating a sequence weight value of the embedded feature time series; Using the time series segmentation submodule to perform block segmentation processing on the embedded feature time series to obtain a time series segmentation block set of time series blocks of different sizes; Sending the time series segmentation set to the multi-scale converter submodule of the adaptive multi-scale dual attention layer, performing dual attention processing on the time series segmentation set through the multi-scale converter submodel, and extracting a multi-scale feature time series; The multi-scale aggregation submodule of the adaptive multi-scale dual attention layer performs adaptive feature weighted aggregation processing on the sequence weight value and the multi-scale feature time series to obtain the first predicted time feature sequence containing multi-scale features.

5. The port berth allocation method according to claim 1, characterized in that: The step of processing the first multimodal data, the priority coefficient and the second multimodal data by a preset multimodal macro model to obtain a real-time berth allocation result corresponding to the target port comprises: Processing the first multimodal data and the second multimodal data by a multimodal encoder of the multimodal large model to obtain a first multimodal feature representation vector; Inputting the first multimodal feature representation vector after spatial alignment processing to the main network of the multimodal large model, and sequentially performing semantic understanding, semantic reasoning and semantic decision processing through the main network of the multimodal large model to obtain a second multimodal feature representation vector; The real-time berth allocation results corresponding to different modes are generated according to the priority coefficient and the second multimodal feature representation vector through the multimodal generator of the multimodal large model.

6. The port berth allocation method according to claim 1, characterized in that: After the step of processing the first multimodal data, the priority coefficient and the second multimodal data by a preset multimodal macro model to obtain a real-time berth allocation result corresponding to the target port, the method further includes: Sending the real-time berth allocation result to the control center of the target port; The berth allocation execution result of the real-time berth allocation result at the target port is monitored by a preset monitoring method.

7. The port berth allocation method according to claim 6, characterized in that: The step of monitoring the execution result of the real-time berth allocation result at the target port by a preset monitoring method comprises: Acquiring multimodal monitoring data of the target port through the preset monitoring method, converting the multimodal monitoring data into a multimodal monitoring feature vector, wherein the multimodal monitoring feature vector includes a first monitoring image feature vector and a first monitoring text feature vector; Processing the first monitoring image feature vector using a preset image encoder, using a bounding box and a cropped image to represent the berth and the ship, to obtain a second monitoring image feature vector; Processing the second monitoring image feature vector and the first monitoring text feature vector in sequence through at least two alternating cross-attention layers and self-attention layers in a preset guided decoder to obtain an association relationship feature vector between the berth and the ship; The association relationship feature vector is converted into the berth allocation execution result.

8. A port berth allocation device, characterized in that: include: A multimodal data acquisition module, used to acquire first multimodal data of berths and ships of a target port; A predicted arrival time module, used for processing the first multimodal data through a preset time series model to obtain a predicted arrival time of the ship; A priority coefficient module, used to obtain the berthing priority data and the departure time data of the ship, and calculate the priority coefficient corresponding to the ship using the predicted arrival time, the berthing priority data and the departure time data; The berth allocation result module is used to obtain the real-time second multimodal data of the target port, and process the first multimodal data, the priority coefficient and the second multimodal data through a preset multimodal large model to obtain the real-time berth allocation result corresponding to the target port.

9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the port berth allocation method described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the port berth allocation method according to any one of claims 1 to 7 are implemented.

Images