An automatic container terminal loading and unloading time prediction method based on an association graph

Abstract

The application discloses an automatic container terminal loading and unloading time prediction method based on a correlation graph, relates to a neural network and a deep learning algorithm, utilizes correlation data in a terminal correlation graph, and can effectively improve the loading and unloading time prediction accuracy of an automatic container terminal, and belongs to the field of artificial intelligence and control technology, and is characterized in that: step one is neighborhood subgraph construction; step two is forward propagation of a loading and unloading time prediction model; step three is prediction time and real time error calculation; step four is network parameter updating; and step five is model saving and application. Management data of the terminal correlation graph is utilized to construct a neighborhood subgraph facing a target task, structured semantic information in the correlation graph is learned through a loading and unloading time prediction model based on a graph neural network, loading and unloading time is regressed and predicted, the problem of loading and unloading task time prediction of an automatic container terminal is effectively solved, and the loading and unloading time prediction accuracy is higher.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of wharf data mining, and relates to an automatic container wharf loading and unloading time prediction method based on an association graph. BACKGROUND

[0002] With the continuous development of information control technology, automated container wharfs in the port industry are constantly popularized and constructed. Compared with traditional wharfs, automated container wharfs use fully automated or remotely operated loading and unloading mechanical equipment to realize uninterrupted container transportation operations at all times, and are scheduled and managed by a unified wharf control system. Controlling a large number of loading and unloading equipment and ensuring efficiency put high requirements on the intelligent level of the wharf control system. Therefore, how to accurately predict the loading and unloading task time to improve the operation efficiency of the wharf has become a key task in the wharf control system.

[0003] In traditional wharf equipment scheduling algorithms, the equipment loading and unloading time is mostly assumed to follow a normal distribution, and the time expectation is simply used as an algorithm input. Obviously, this estimation method has errors, which are further transmitted and amplified, and has gradually become difficult to meet the high-precision scheduling demand. Another representative method is to calculate the equipment loading and unloading time using a physical model, which is commonly used in simulation models. This method strongly depends on expert knowledge, and when considering the coupling cooperation of multiple devices, the model will become very large and complex, which is not suitable for large-scale application in scheduling algorithms.

[0004] At present, the container wharf loading and unloading operation time prediction methods mainly fall into three categories: physical models, machine learning methods, and deep learning methods. The physical model method uses relevant business logic to model through simulation, which is characterized by fast calculation speed, but requires high modeling of the wharf loading and unloading process, cannot handle sudden situations in the wharf loading and unloading operation, and has large prediction error of the wharf loading and unloading time. The machine learning method learns from historical data and uses optimization algorithms to handle random factors in the wharf loading and unloading process, which does not require accurate modeling of the wharf loading and unloading process, but the utilization of historical data is limited. The deep learning method has the advantages of the machine learning method, and the learning ability of the model is more powerful, which can use massive historical data to more accurately predict the wharf loading and unloading time, but the current method lacks knowledge in the field of wharf loading and unloading operation, and the relationship between loading and unloading operations is not considered.

[0005] Therefore, how to effectively utilize the knowledge in the field of container wharf loading and unloading, its complex business logic and associated processes, and consider the connection between devices in the loading and unloading process to accurately predict the execution time of the loading and unloading task is a key problem that needs to be solved in the process of predicting the loading and unloading time of the automated container wharf. SUMMARY

[0006] In order to solve the problem that it is difficult to effectively utilize the expert knowledge of the terminal and the correlation of the handling process in the automatic container terminal handling time prediction process, the application provides an automatic container terminal handling time prediction method based on a correlation graph, which fuses multi-source terminal big data in a graph structure through an industrial big data-oriented data and knowledge autonomous fusion research paradigm, establishes a terminal correlation graph, and introduces structured semantic information to reflect the container terminal handling field knowledge, so as to realize the autonomous fusion of the numerical deep learning model and the field knowledge and realize the accurate prediction of the terminal handling time.

[0007] The automatic container terminal handling time prediction method based on the correlation graph mainly includes the following steps:

[0008] Step one: neighborhood subgraph construction, on the basis of the existing terminal correlation graph (the related technology is disclosed in Chinese patent application CN113139022A A hybrid rule-based enterprise logistics data on-demand fusion method), the neighborhood subgraph is divided and the training sample is constructed, which is provided to step two;

[0009] Step two: handling time prediction model forward propagation, the neighborhood subgraph provided in step one is used to fuse the correlation between nodes and attribute information through a heterogeneous graph neural network (HGNN), and the handling time of the equipment is predicted, which is provided to step three;

[0010] Step three: prediction time and real time error calculation, the predicted handling time provided in step two is used to calculate the loss of the real handling time through the mean square error loss function, which is provided to step four;

[0011] Step four: network parameter update, the loss provided in step three is used to train the parameters of the handling time prediction model in step two; if the model has not converged, step two is executed; if the model has converged, the converged model is provided to step five;

[0012] Step five: model saving and application, the model structure and parameters provided in step four are saved, and the handling time of the new sample is output by using the model.

[0013] Further, the terminal correlation graph in step one is composed of two elements

[0014] Where O=O e ∪O r ∪O p is the matter layer, which is composed of three types of matter triples,

[0015] O eThis represents a set of node triples, defining the concepts of terminal equipment, including AGV actions, rail-mounted gantry crane actions, gantry cranes (TP), yard seaside TP, yard roadside TP, yard container area, yard container position, yard layout, yard multiple positions, yard seaside support, and automated guided vehicle (PB).

[0016] O r This represents a set of relational triples that describe the business relationships between concepts, including equipment start location, equipment end location, container loading / unloading, operating machinery, yard position, and yard multiple.

[0017] O p The attribute triple represents the set of attributes that describe the specific attributes of the terminal equipment concept, including AGV number, rail gantry number, container weight, container height, relative position, relative angle, relative X / Y coordinate distance, planned upper and lower TP positions, start time, unlocking time, end time, etc.

[0018] where I=I e ∪I r ∪I p At the instance layer, each instance triplet is generated by instantiating the principle triplet.

[0019] I e For node instance triples, O e Instantiated, in the form of (instance IRI, isA, concept name), it indicates that instance IRI is a specific instance of a certain concept (e.g., instance LOC_302 is a specific instance of a yard container location);

[0020] I r For relation instance triples, by O r Instantiated, in the form of (instance IRI, relation name, instance IRI), representing the relationship between instances;

[0021] I p For attribute instance triples, by O p The instantiation is obtained in the form of (instance IRI, attribute name, attribute value), which represents the specific attribute value of the instance.

[0022] Furthermore, the neighborhood subgraph construction process described in step one includes two steps: path search at the principle layer and node search at the instance layer. Path search obtains the set of associated paths from the principle layer of the dock association graph, and node search obtains the neighborhood subgraph from the instance layer of the dock association graph.

[0023] Step 1.1: The path search at the event layer, based on the target task node v cand search the maximum step length s, taking the target task as the center node, and querying the association path P from the code terminal association graph ontology layer by limiting the step length using the Sparql script.

[0024] P is composed of s sequentially connected relationship triples, and the formal expression is P={(v c ,e1|^e1,v1),(v1,e2|^e2,v2),…,(v l-1 ,e l |^e l ,v l )}, wherein the symbol | represents an or relationship, and the symbol ^ represents the exchange of the head and tail entities of the relationship triple, i.e., (v i-1 ,^e i ,v i ) = (v i ,e i ,v i-1 ). Collect all the association paths obtained by querying to obtain the set of association paths

[0025] Step 1.2: The instance layer node search is performed according to the association path The path instance is obtained from the instance layer of the code terminal association graph by Sparql, each path instance contains an instance node v c of a target task, and other nodes v1,…,v n-1 associated under the path, i.e., the neighborhood nodes of the target task node, and a neighborhood subgraph is constructed wherein V={v c ,v1,…,v n-1} represents a node set, E={(v i ,e,v j )|v i ,v j ∈V} represents an edge set, represents a property set, including four types of properties: numerical, categorical, temporal, and numbered.

[0026] Further, in step two, the loading and unloading time prediction model forward propagation inputs the neighborhood subgraph into a heterogeneous graph neural network (HGNN) to obtain the predicted time of the neighborhood subgraph The loading and unloading time prediction model specifically includes the following steps:

[0027] Step 2.1: Node information representation, using a node information representation module, according to the different number and different semantic attribute information set A i of each node, a unified vectorized representation x i, which is provided to step 2.2, and the formal description is The node information represents, first, an independent attribute encoder is constructed for each attribute, and the attribute value of the node is converted into a γ-dimensional vector. The numerical attribute uses a linear mapping layer to convert the normalized real value into a vector, q i,k = w k a i,k ; the categorical attribute uses embedding to assign a vector to each value of the attribute, q i,k = Embedding(w k , a i,k ). For the time attribute, after converting it into a timestamp, it is encoded in the same way as the numerical attribute; for the numbered attribute, if the number of devices is small, the encoding method of the categorical attribute is used, and if the number of devices is large, the character-level encoding and aggregation method is used, wherein is the attribute value of attribute k of node v i , is the jth character in the string of the numbered attribute value k, and the string length is l, is the embedding representation vector of attribute k, is the learnable parameter in the attribute encoder. Then, all attribute vectors A i = {q i,1 , q i,2 , …, q i,k} of node v i are further aggregated to obtain the node representation vector x i , wherein wherein is the learnable attribute type vector of attribute k, which is shared by all nodes.

[0028] Step 2.2: Node feature fusion, using the vector representation X = {x c , x1, x2, …, x n-1} of all nodes in the neighborhood subgraph provided by step 2.1, according to the edge information E of the neighborhood subgraph, the neighbor information of the node is fused through the heterogeneous graph neural network (HGNN) to obtain the hidden layer representation H = {h c , h1, …, h n-1} of the node, which is provided to step 2.3, and the formal description is φ: (X, E) → H. The hidden layer representation of the node is initialized through a fully connected layer j = c, 1, 2, … n-1. The specific calculation method of the lth layer in the heterogeneous graph neural network (HGNN) is as follows:

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] where, is the hidden layer vector of the l-th layer of node v j , is the learnable parameter in node feature fusion, and ρ is the nonlinear activation function. After repeating iteration L times, we get The last iteration result is taken as the output, i.e., the node hidden layer representation fused with neighbor information

[0035] Step 2.3: Information aggregation, including graph pooling, task node attribute mapping, and gated aggregation, uses the node hidden layer representation H to obtain the neighborhood subgraph vector z, which is formally described as ψ: H→z. Graph pooling uses the hidden layer representation H of the nodes in the neighborhood subgraph provided in step 2.2 to output the pooled graph structure vector representation o s , which is calculated as follows:

[0036] η i，t = Softmax(h i ·u t )

[0037]

[0038] u t+1 = LSTM(Concat([u t ; r t ]))

[0039] o s = u T

[0040] Task node attribute mapping uses the hidden layer representation h c of the task node provided in step 2.2 to calculate the graph attribute vector representation o a = MLP(h c ); Gated aggregation fuses the structure vector o s and the attribute vector o a through a gating mechanism to obtain the final neighborhood subgraph vector z = Sigmoid(g)⊙o s +(1-Sigmoid(g))⊙oa , provided to step 2.4, wherein, is a learnable gating vector.

[0041] Step 2.4: Time prediction, the neighborhood subgraph vector z provided in step 2.3 is calculated through a fully connected layer to obtain the predicted loading and unloading time of the model output wherein is a learnable parameter in the time prediction step.

[0042] Further, in step three, the predicted time and real time error calculation, the predicted loading and unloading time obtained by the loading and unloading time prediction model and the real loading and unloading time y, is calculated The mean square error loss function of y

[0043] Further, step four said network parameter θ update, the loading and unloading time prediction model is trained on the training data, according to the mean square error loss Adjust the training parameters in the prediction model. Before the model converges, repeat step two; when the model converges, the converged model M and parameters

[0044] Further, step five said model save and application, save the model M and related parameters provided in step four In application, for new samples Input it into the trained loading and unloading time prediction model M, and use the method of step two to perform forward propagation to obtain The predicted time of BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The figure shows the flowchart of the method of the application.

[0046] Figure 2 The figure shows the overall structure diagram of the automatic container terminal loading and unloading time prediction method based on the association graph of the application.

[0047] Figure 3 The figure shows the structure diagram of the loading and unloading time prediction model in step two of the application (i.e. Figure 2 The detailed structure of the loading and unloading time prediction model M in

[0048] Figure 4 The figure shows the schematic diagram of node information representation in step 2.1 of the application (i.e. Figure 3 The specific structure diagram of the node representation module In

[0049] Figure 5A schematic diagram of information aggregation in step 2.3 of the present application (i.e. Figure 3 A specific structure diagram of the information aggregation module ψ in the present application). DETAILED DESCRIPTION

[0050] To more clearly explain the technical problems, technical solutions and advantages to be solved by the present application, the present application will be further described in detail below with reference to the embodiments.

[0051] The present application proposes an automatic container terminal loading and unloading time prediction method based on an association graph, explores an industrial big data-oriented data and knowledge autonomous integration research paradigm, autonomously discovers associated features using terminal knowledge in the association graph, constructs a neighborhood subgraph centered on a target task, autonomously learns structured semantic information and business connections in the association graph through a heterogeneous graph neural network (HGNN) based on the loading and unloading time prediction model, improves the performance of terminal loading and unloading time prediction, and is conducive to the scheduling management of the automatic terminal management system.

[0052] The basic process of the automatic container terminal loading and unloading time prediction method based on the association graph of the present application is as follows: first, for the problem of associated knowledge mining, based on the rich semantic structured data in the association graph, the neighborhood subgraph is divided by a step-limited traversal search method, and the data features associated with the loading and unloading time are autonomously mined, and the information in the subgraph is the associated knowledge; second, in order to integrate the attribute information of different quantities and different semantics in the neighborhood subgraph, the node encoder is used to encode according to the attribute point type; then, the association relationship and terminal knowledge in the neighborhood subgraph are fused through the method of heterogeneous graph neural network (HGNN); finally, the structure information of the neighborhood subgraph and the attribute information of the task node are integrated to predict the loading and unloading time.

[0053] The specific implementation process of the automatic container terminal loading and unloading time prediction method based on the association graph of the present application is as follows (as shown in Figure 1 、 Figure 2 ):

[0054] Step one: neighborhood subgraph construction

[0055] According to the existing terminal association graph The association relationship of the terminal business can be reflected by the topological structure between the nodes and the relationship semantics on the edge, and the data is expressed on the attribute value of the node, so the neighborhood nodes and their attribute values of the loading and unloading task node are the associated data.

[0056] This invention divides the neighborhood subgraph through a traversal search with a limited step size. Given a search step size of s = 4, with the target task as the central node, nodes reachable within 4 steps are the neighborhood nodes associated with the target loading / unloading task. Since the instance graph is extremely large, directly using Sparql commands for traversal search is inefficient. This invention first searches for associated paths to neighborhood nodes in the event graph, and then queries the associated neighborhood nodes in the instance graph using these associated paths as constraints.

[0057] Furthermore, the associated path search is performed given the context graph O and the target task node v. c (e.g., the concept node of the rail-mounted crane movement), using Sparql query commands to search from v c Starting from, an associated path P of length l (1≤l≤4) = {(v c ,e1|^e1,v1), (v1,e2|^e2,v2),..., (v l-1 e l |^e l v l This yields a set of all associated paths. Neighborhood node search given instance graph I and an associated path Based on the semantics of the associated paths, Sparql is used to obtain the path instances in the instance graph under that path. Each path instance contains an instance node of the target task, as well as other nodes associated with that path. The instance triples containing these nodes form part of the neighborhood subgraph. The set of associated paths... All query results are merged to obtain the neighborhood subgraph of the target task node. Composed of a set of nodes V, a set of edges E, and a set of attributes composition.

[0058] Furthermore, such as Figure 3 , Figure 4 As shown, the node set V = {v c v1, ..., v n-1}, containing the central node v c and neighboring node v 1：n-1 Edge set E = {(v i ,e,v j )|v i v j ∈V}, the semantics of the edges connecting nodes are represented by e. Attribute set set Each node v i Each has a corresponding attribute set A i It contains several attributes and attribute values, namely A i ={(v i ,k,a i，k), a i，k attribute values of k attributes of node v i , including numerical, categorical, temporal and ordinal attributes.

[0059] Step two: forward propagation of the loading and unloading time prediction model

[0060] As shown in Figure 3 , the loading and unloading time prediction model M uses a heterogeneous graph neural network (HGNN) to fuse the node attribute information and the associated interaction information between nodes in the neighborhood subgraph , and calculates the neighborhood subgraph vector z of .

[0061] Specifically, as shown in Figure 3 , the loading and unloading time prediction model M includes four steps: step 2.1 node information representation, step 2.2 node feature fusion, step 2.3 information aggregation, and step 2.4 time prediction. First, the input node V and the attribute information of the node are obtained through node information representation to obtain the vector representation X of the node; then according to the topology structure provided by the neighborhood subgraph and the semantics on the edge, the information transmission is carried out by using the heterogeneous graph neural network (HGNN), and the node hidden layer vector representation H of the fused neighborhood information is obtained; next, the overall representation of the neighborhood subgraph, i.e. the output layer vector z, is obtained through the information aggregation module, and the linear mapping layer and the Sigmoid activation function in the time prediction module are used to calculate the loading and unloading time prediction result of the model

[0062] As shown in Figure 4 : the step 2.1 node information representation needs to use the attribute encoder through the node information representation module to convert the attribute values in the attribute set A i of the node v j into the same dimension vector representation because each node in the neighborhood subgraph may have different number and different semantic attributes, i.e. |A i |≠|A i , which is formalized as The node information representation includes attribute information encoding and node attribute aggregation.

[0063] The attribute information encoding represents the attribute value a i，k of the node as a vector q i，kSince different attributes possess different semantic information and present different data states, a separate attribute encoder needs to be constructed for each attribute. Based on the type of attribute value, attribute information can be categorized into four types: numerical (e.g., stacking height after grabbing a container), categorical (e.g., instruction priority), temporal (e.g., arrival time at the destination), and numbered (e.g., AGV number). For numerical attributes, a linear mapping layer is used to transform the normalized real values ​​into vectors, q... i，k =w k a i，k For categorical attributes, an embedding method is used to assign a vector q to each value of the attribute. i，k =Embedding(w k a i，k For time-based attributes, they are converted to timestamps and then encoded using the same methods as numeric attributes. For number-based attributes, if the number of devices is small, a categorized encoding method is used; if the number of devices is large, a character-level encoding and aggregation method is adopted. in For node v i The attribute value of attribute k. The j-th character in the string containing the k-th attribute value (of type number), and the string length is l. Let k be the embedding representation vector of attribute k. These are the learnable parameters in the attribute encoder.

[0064] The node attribute aggregation for node v i All attributes represent vector A i ={q i，1 q i，2 , ..., q i，k The system uses the concept of attention to aggregate the attribute information of each node. in The attention weights for the attributes are denoted as , where . It is a vector of learnable attribute types for attribute k, which is shared by all nodes.

[0065] Step 2.2, node feature fusion, employs a heterogeneous graph neural network (HGNN). Based on the edge set E in the neighborhood subgraph, it fuses the neighbor information of a node, transforming the vector representation of all nodes in the neighborhood subgraph into X = {x c x1, x2, ..., x n-1} is transformed into a hidden layer representation of nodes H={h c h1, ..., h n-1 The formal expression is φ: (X, E) → H. First, the vector representation of the nodes with γ = 50 dimensions is...j The representation is transformed into a node hidden layer representation of d=60 dimensions through a fully connected layer. j = c, 1, 2, ..., n-1, where c is the central node v c The labels 1, 2, ..., n-1 represent the neighboring nodes v. 1：n-1 The nodes are labeled. Then, a heterogeneous graph neural network (HGNN) is used to perform L=3 iterations on the nodes to fuse the neighbor information of the nodes. The specific calculation method of the l-th iteration is as follows:

[0066]

[0067]

[0068]

[0069]

[0070]

[0071] in, For node v j The hidden layer vector of the l-th layer,

[0072] These are all learnable parameters in node feature fusion.

[0073] σ is a nonlinear activation function. These are all intermediate variables in the HGNN calculation process. After repeated iterations L=3 times, the following results were obtained: The result of the last iteration is used as the output, which is the hidden layer representation of the node that incorporates neighbor information.

[0074] Step 2.3 (as described) Figure 5 , Figure 3 (As shown) Information aggregation includes three components: graph pooling, task node attribute mapping, and gated information aggregation. Based on the input node hidden layer vector representation H, the structure vector representation o of the neighborhood subgraph is obtained through graph pooling. s The hidden representation h of task nodes is obtained through MLP. c Transform into a graph structure vector representation o s Aggregation is performed using a gating mechanism to obtain the output layer vector representation z.

[0075] The graph pooling component employs a Set2Set pooling method to transform the hidden layer representation H of nodes in the neighborhood subgraph into a graph structure vector representation o. s .

[0076] η i，t =Softmax(hi ·u t )

[0077]

[0078] u t+1 =LSTM(Concat([u t ;r t ]))

[0079] o s =u T

[0080] where is an intermediate variable in the Set2Set pooling computation.

[0081] The task node attribute mapping component translates the hidden representation h c of the center node to the same space as the structural representation vector o s , o a =MLP(h c ).

[0082] Since the structural vector representation o s and the attribute vector representation o a reflect different features of the neighborhood subgraph, the gating mechanism gives different weights to each dimension of the two (o s and o a ) through a learnable gating integration function , and the final neighborhood subgraph vector z is obtained after fusion.

[0083] z=Sigmoid(g)⊙o s +(1-Sigmoid(g))⊙o a

[0084] The step 2.4 time prediction uses linear mapping and Sigmoid activation function to calculate the final model output of the loading and unloading time prediction result where is a learnable parameter in the time prediction step.

[0085] Step three: prediction time and real time error calculation

[0086] The predicted loading and unloading time obtained by the loading and unloading time prediction model in step two is used to calculate the loss of the predicted loading and unloading time and the real loading and unloading time y through the mean square error MSE

[0087]

[0088] where For the number of samples.

[0089] Step four: network parameter update

[0090] The loading and unloading time prediction model is trained on the training data according to the mean square error Adjust the training parameters θ in the prediction model. If the model does not reach convergence, repeat step two; if the model converges, obtain the converged model M and parameters

[0091] Step five: model saving and application

[0092] This step first saves the model M and related parameters When applied, for a new sample Input it into the trained loading and unloading time prediction model M, according to the model parameters And use the forward propagation method in step two to calculate, get The predicted loading and unloading time

[0093] Experimental verification

[0094] The present application carries out loading and unloading time prediction experiment on the automatic container terminal loading and unloading record data containing more than 400,000 samples to verify the effectiveness of the application. The opening and closing time of the rail-mounted crane loading and unloading task is taken as the prediction object, and the total node number of the data set reaches 880,000, of which 420,000 are target nodes corresponding to the rail-mounted crane task, containing about 2.4 million triples, with 39 attribute types and 6 relationship types. In order to verify the effectiveness of the present application from multiple dimensions, six indexes are used to evaluate the performance of the model: root mean squared error (RMSE), mean absolute error (MAE), mean squared log error (MSLE), mean absolute percentage error (MAPE), mean relative error (MRE), and R-square score (R2-Score). The calculation method is as follows:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101] Wherein the lower the RMSE, MAE, MSLE, MAPE, MRE index indicates the smaller the prediction error, the better the performance of the model; and the higher the R2 index, the smaller the prediction error, the better the performance of the model. The contrast model selects the history expectation commonly used in industrial production scene, the widely used machine learning models XGBoost, LightGBM, SVM, and the neural network regression model.

[0102] From Table 1 and

[0103] It can be seen from Table 2 that the method of the present application is superior to other methods in various indicators, proving the effectiveness and advancement of the method of the present application.

[0104] Table 1 different model opening time prediction performance

[0105]

[0106] Table 2 different model closing time prediction performance

[0107]

Claims

1. An automatic container terminal handling time prediction method based on an association graph, comprising the following steps: Step one: neighborhood subgraph construction, divide the neighborhood subgraph on the existing terminal association graph and construct the training sample, provide for step two; According to the existing wharf association graph The association relationship of wharf business is embodied by the topological structure between nodes and the relationship semantics on the edges, and the data is expressed on the attribute values of the nodes. The neighborhood nodes of the loading and unloading task nodes and their attribute values are the association data. The neighborhood subgraph is divided by limiting the traversal search, the search step is given, the target task is taken as the center node, and the nodes that can be reached within the step are the neighborhood nodes associated with the target handling task; first search the association path to reach the neighborhood node in the matter graph, and then query the neighborhood association node in the instance graph with the association path as the constraint; Correlation path search given the matter graph O and the target task node v c , using Sparql query command search from v c , the length of the correlation path P = {(v c , e1 | e1, v1), (v1, e2 | e2, v2), …, (v l-1 , e l | e l , v l} of l, get all the correlation path set Neighborhood node search given the instance graph I and a correlation path According to the semantic of the correlation path, get the path instance of the instance graph under the path by Sparql, each path instance contains an instance node of the target task, and other nodes associated under the path, the instance triples containing these nodes are part of the neighborhood subgraph; Merge all the query results in the correlation path set , get the neighborhood subgraph of the target task node composed of node set V, edge set E, attribute set set ; Node set V = {v c ,v1,…,v n-1 }, containing the central node v c and neighboring node v 1:n-1 Edge set E = {(v i ,e,v j )|v i ,v j ∈V}, the semantics of the edges connecting nodes are represented by e; attribute set set Each node v i Each has a corresponding attribute set A i It contains several attributes and attribute values, namely A i ={(v i ,k,a i,k )},a i,k Represents node v i The value of the k attribute; Step two: forward propagation of the handling time prediction model, using the neighborhood subgraph provided in step one, the association relationship and attribute information between nodes are fused by the way of heterogeneous graph neural network to predict the handling time of the equipment, which is provided for step three; The loading and unloading time prediction model M adopts a heterogeneous graph neural network (HGNN) to fuse node attribute information and inter-node association interaction information in a neighborhood subgraph , and calculate a neighborhood subgraph vector z . First, input the node V and the attribute information of the node The vector representation X of the node is obtained through the node information representation Then, according to the topological structure provided by the neighborhood subgraph and the semantics on the edge, information transmission is carried out by using the heterogeneous graph neural network (HGNN) to obtain the node hidden layer vector representation H which fuses the neighborhood information; Next, the overall representation of the neighborhood subgraph, i.e., the output layer vector z, is obtained by the information aggregation module, and the model's loading and unloading time prediction result is calculated using the linear mapping layer and the Sigmoid activation function in the time prediction module The handling time prediction model M includes four steps of step 2.1 node information representation, step 2.2 node feature fusion, step 2.3 information aggregation, and step 2.4 time prediction; The node information in step 2.1 indicates that each node in the neighborhood subgraph has a different number and semantic attributes, i.e., |A i |≠|A j | The node information representation module uses an attribute encoder to represent node v i The attribute set A i The attribute values ​​in the vector are converted into vectors of the same dimension, formalized as follows: Node information representation includes two parts: attribute information encoding and node attribute aggregation; the attribute information encoding converts the node's attribute value a i,k Represented as a γ-dimensional vector q i,k Based on the type of attribute value, attribute information is divided into four categories: numerical, categorical, temporal, and numbered. The node attribute aggregation is applied to node v. i All attributes represent vector A i ={q i,1 ,q i,2 ,…,q i,k The system uses the concept of attention to aggregate the attribute information of each node. in The attention weights for the attributes are denoted as , where . It is a vector of learnable attribute types for attribute k, shared by all nodes; The step 2.2 node feature fusion adopts a heterogeneous graph neural network (HGNN) to fuse neighbor information of nodes based on an edge set E in a neighborhood subgraph, and converts vector representations X = {x c ,x1,x2…,x n-1} of all nodes in the neighborhood subgraph into node hidden layer representations H = {h c ,h1,…,h n-1}, which is formalized as φ:(X,E)→H; Step 2.3, information aggregation, includes three components: graph pooling, task node attribute mapping, and gated information aggregation. Based on the input node hidden layer vector representation H, graph pooling is used to obtain the structure vector representation o of the neighborhood subgraph. s The hidden representation h of task nodes is obtained through MLP. c Transform into a graph structure vector representation o s Aggregation is performed through a gating mechanism to obtain the output layer vector representation z; The step 2.4 time prediction uses linear mapping and Sigmoid activation function to calculate the final model output of the loading and unloading time prediction result wherein are learnable parameters in the time prediction step; Step three: prediction time and real time error calculation, using the predicted handling time provided in step two, the loss with the real handling time is calculated by the mean square error loss function, which is provided for step four; Step four: network parameter update, using the loss provided in step three, the parameters of the handling time prediction model in step two are trained; if the model has not converged, execute step two; if the model has converged, provide the converged model to step five; Step five: model saving and application, save the model structure and parameters provided in step four, and use the model to predict the handling time of new samples and output.

2. The prediction method of claim 1, wherein: Step one: neighborhood subgraph construction Further details: The neighborhood subgraph is divided by limiting the traversal search, the search step is given, the target task is taken as the center node, and the nodes that can be reached within the step are the neighborhood nodes associated with the target handling task; first search the association path to reach the neighborhood node in the matter graph, and then query the neighborhood association node in the instance graph with the association path as the constraint; Correlation path search given the ontology O and the target task node v c , using Sparql query command search from v c , length l (1≤l≤4) correlation path P = {(v c ,e1|^e1,v1),(v1,e2|^e2,v2),…,(v l-1 ,e l |^e l ,v l}, get all the correlation path set Neighborhood node search given the instance graph I and a correlation path According to the semantic of the correlation path, get the path instance of the instance graph under the path by Sparql, each path instance contains an instance node of the target task, and other nodes associated under the path, the instance triples containing these nodes are part of the neighborhood subgraph; merge all the query results in the correlation path set to get the neighborhood subgraph of the target task node composed of node set V, edge set E, attribute set set ​ Node set V = {v c ,v1,…,v n-1 }, containing the central node v c and neighboring node v 1:n-1 Edge set E = {(v i ,e,v j )|v i ,v j ∈V}, the semantics of the edges connecting nodes are represented by e; attribute set set Each node v i Each has a corresponding attribute set A i It contains several attributes and attribute values, namely A i ={(v i ,k,a i,k )},a i,k Represents node v i The k attribute has four types of attribute values: numeric, categorical, time-based, and number-based.

3. The prediction method of claim 1, wherein: First, input the node V and the attribute information of the node The vector representation X of the node is obtained through the node information representation Then, according to the topological structure provided by the neighborhood subgraph and the semantics on the edge, information transmission is carried out by using the heterogeneous graph neural network (HGNN) to obtain the node hidden layer vector representation H which fuses the neighborhood information; Next, the overall representation of the neighborhood subgraph, i.e. the output layer vector z, is obtained by the information aggregation module, and the model's loading and unloading time prediction result is calculated by using the linear mapping layer and the Sigmoid activation function in the time prediction module The handling time prediction model M includes four steps of step 2.1 node information representation, step 2.2 node feature fusion, step 2.3 information aggregation, and step 2.4 time prediction.

4. The prediction method of claim 1, wherein: The step 2.1 node information representation indicates that each node in the neighborhood subgraph has different number and different semantic attributes, that is, |A i |≠|A j |The attribute value in the attribute set A i of the node v i is converted into a vector representation of the same dimension by using an attribute encoder by the node information representation module, which is formalized as The node information representation includes attribute information encoding and node attribute aggregation. The attribute information encoding encodes the attribute value a i,k of a node as a vector q i,k of dimension γ=50 i,k ; an independent attribute encoder is constructed for each attribute; according to the type of the attribute value, the attribute information is divided into four categories: numerical type, categorical type, time type, and number type; for the numerical type attribute, a linear mapping layer is used to convert the normalized real value into a vector q i = w i,k a i,k ; for the categorical type attribute, an embedding method is used to assign a vector q i,k = Embedding(wk,a i ) to each value of the attribute; for the time type attribute, after being converted into a time stamp, it is encoded in the same way as the numerical type attribute; for the number type attribute, if the number of devices is small, the encoding method of the categorical type attribute is used, and if the number of devices is large, the character level encoding and aggregation method is used wherein a is the attribute value of attribute k of node v i , a is the jth character in the string of the attribute value of the number type k, the length of the string is l, q is the embedding representation vector of attribute k, and w is the learnable parameter in the attribute encoder; The node attribute aggregation for all attributes of node v i is a vector A i = {q i,1 , q i,2 , …, q i,k}, which aggregates the attribute information of each node using the attention idea, where is the attention weight of the attribute, and where is a learnable attribute type vector of k attributes, which is shared by all nodes.

5. The prediction method of claim 1, wherein: The step 2.2 node feature fusion adopts a heterogeneous graph neural network (HGNN) to fuse the neighbor information of the nodes based on the edge set E in the neighborhood subgraph, and converts the vector representation X = {x c , x1, x2, …, x n-1} of all nodes in the neighborhood subgraph into node hidden layer representation H = {h c , h1, …, h n-1}, which is formalized as φ: (X, E) → H; first, the 50-dimensional vector representation x j of the node is converted into a 60-dimensional node hidden layer representation h c is the label of the center node v c , and 1, 2, … n-1 are the labels of the neighborhood nodes v 1:n-1 ; then the heterogeneous graph neural network (HGNN) is used for L = 3 iterations to fuse the neighbor information of the nodes, and the specific calculation method of the lth iteration is as follows: wherein, is the hidden layer vector of the l-th layer for node v j , are learnable parameters in node feature fusion, σ is a nonlinear activation function, are intermediate variables in the HGNN computation process; after repeating the iteration L = 3 times, we obtain The last iteration result is taken as the output, i.e., the node hidden layer representation fused with neighbor information 6. The prediction method of claim 1, wherein: The step 2.3 information aggregation includes three components of graph pooling, task node attribute mapping and gated information aggregation; according to the input node hidden layer vector representation H, the structure vector representation o of the neighborhood subgraph is obtained through graph pooling s ; the hidden layer representation h c of the task node is converted into the graph structure vector representation o s by the MLP; the output layer vector representation z is obtained by the aggregation through the gating mechanism, The graph pooling component adopts a Set2Set pooling method to convert the node hidden layer representation H in the neighborhood subgraph into a graph structure vector representation o s ; η i,t = Softmax(h i ·u t ) wherein is an intermediate variable in the Set2Set pooling computation process; The task node property mapping component maps the hidden layer representation h c into the same space as the structural representation vector o s , a = MLP(h c ); Since the structural vector representation o s and the attribute vector representation o a reflects different features of the neighborhood subgraph, the gating mechanism integrates the two vectors through a learnable gating function Different weights are assigned to each dimension of the two vectors (o s and o a ), and the final neighborhood subgraph vector z is obtained after fusion. z = Sigmoid(g) 0 o s + (1 - Sigmoid(g)) 0 o a .

7. The prediction method of claim 1, wherein: The step 2.4 time prediction is calculated using a linear mapping and a sigmoid activation function to obtain the final model output of the time prediction result of the loading and unloading time wherein are learnable parameters in the time prediction step.

8. The prediction method of claim 1, wherein: Step three: prediction time and real time error calculation predicted handling time obtained using the second step two handling time prediction model predicted handling time calculated by mean square error, MSE loss with respect to true handling time y wherein is the number of samples; Step four: Network parameter update training the loading and unloading time prediction model on the training data according to the mean square error adjusting the training parameters θ in the prediction model; if the model does not reach convergence, repeating step two; if the model converges, obtaining the converged model M and parameters Step five: Model saving and application This step first saves the model M and related parameters provided in step four At the time of application, for a new sample Input it into the trained loading and unloading time prediction model M, and calculate according to the model parameters And use the forward propagation method in step two to calculate, get the predicted loading and unloading time of the sample

Images