Container data processing method and system based on big data

Abstract

The present invention relates to the technical field of data processing, and discloses a container data processing method and system based on big data, comprising: generating an undirected graph according to a yard layout diagram; screening optional yard nodes according to load attribute data and task attribute data; calculating the average spatial distance between each optional yard node and all exit nodes, and screening candidate yard nodes according to the size of the average spatial distance; obtaining a set of transportation paths between each candidate yard node and a target entry node, and screening out an optimal transportation path corresponding to each candidate yard node by using a genetic algorithm; screening out a target yard node according to the size of fitness, and generating a transportation scheduling plan according to the target yard node and the optimal transportation path corresponding to the target yard node; the present invention can optimize the allocation of yard resources while improving the unloading efficiency, ensure the scientific connection between the unloading operation and the subsequent logistics links, and significantly improve the overall operation efficiency of the port.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a container data processing method and system based on big data. Background Art

[0002] As an important hub in the logistics chain, the overall operational efficiency of ports depends to a large extent on the scientificity and efficiency of unloading operations. With the continuous growth of global logistics demand, existing unloading solutions are difficult to fully adapt to dynamic and multi-dimensional operational needs. Traditional static planning methods are obviously insufficient in integrating ship mission attributes and yard resource status, and fail to optimize unloading paths and resource allocation from a global perspective, resulting in low unloading efficiency, uneven utilization of yard resources and increased transportation costs. At the same time, there are also great limitations in the connection of subsequent logistics links.

[0003] The key to unloading operations is to achieve accurate matching of task attributes with yard resources, and through scientific path planning and resource allocation, enable the unloading process to dynamically adapt to task changes and effectively reduce operation and transportation costs; however, existing technologies lack the ability to comprehensively analyze and dynamically adjust multi-dimensional data, and are unable to reasonably optimize yard resource allocation while improving unloading efficiency, and it is also difficult to ensure the scientific connection between unloading operations and subsequent logistics links.

[0004] Based on this, there is an urgent need for a technical method that can dynamically optimize unloading efficiency from a global level, rationally allocate yard resources, reduce transportation costs, and ensure efficient connection between unloading operations and subsequent logistics links through scientific planning, thereby significantly improving the overall operational efficiency of the port and providing efficient and intelligent solutions for modern port logistics management. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a container data processing method and system based on big data.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A container data processing method based on big data, the method comprising:

[0008] Obtaining task attribute data of the ship to be unloaded and a yard layout diagram in the port, and generating an undirected graph according to the yard layout diagram; the undirected graph contains multiple edges and multiple nodes, and the nodes are divided into a target entry node, P yard nodes and Q exit nodes, where P and Q are both integers greater than zero;

[0009] Obtain the load attribute data of each yard node, and select S optional yard nodes from P yard nodes according to the load attribute data and task attribute data, where S is an integer greater than zero;

[0010] Calculate the average spatial distance between each optional yard node and all exit nodes, and select K candidate yard nodes from the S optional yard nodes according to the average spatial distance, where K is an integer greater than zero;

[0011] In the undirected graph, a set of transportation paths between each candidate yard node and the target entrance node is obtained, and a preconfigured genetic algorithm is used to screen out the optimal transportation path corresponding to each candidate yard node from the transportation path set to obtain multiple optimal transportation paths;

[0012] The fitness of each optimal transportation path is obtained, and the target yard node is selected from the K candidate yard nodes according to the fitness size. Then, the transportation scheduling plan for the ship to be unloaded is generated according to the target yard node and the optimal transportation path corresponding to the target yard node.

[0013] Furthermore, the task attribute data includes first task attribute data and second task attribute data, the first task attribute data is the number of containers to be unloaded by the ship to be unloaded, and the second task attribute data includes multiple attributes of the containers to be unloaded; the carrying attribute data includes first carrying attribute data and second carrying attribute data, the first carrying attribute data is the number of containers that the yard node can carry, and the second carrying attribute data includes multiple attributes of the containers that can be carried, and the attributes include type, size and weight.

[0014] Further, the selecting S optional storage yard nodes from the P storage yard nodes includes:

[0015] Calculate the difference between the number of containers to be unloaded and the number of containers that can be unloaded according to the first load attribute data and the first task attribute data;

[0016] The quantity difference is compared with a preset quantity difference threshold. If the quantity difference is greater than or equal to the quantity difference threshold, the corresponding yard node is marked as an alternative yard node. If the quantity difference is less than the quantity difference threshold, the corresponding yard node is marked as a non-alternative yard node.

[0017] Count all candidate storage yard nodes whose quantity difference is greater than or equal to the quantity difference threshold, and obtain multiple candidate storage yard nodes;

[0018] Calculate the matching degree between each candidate yard node and the ship to be unloaded according to the second load-bearing attribute data and the second task attribute data;

[0019] The matching degree is compared with a preset matching degree threshold. If the matching degree is greater than or equal to the matching degree threshold, the corresponding candidate yard node is marked as an optional yard node. If the matching degree is less than the matching degree threshold, the corresponding candidate yard node is marked as a non-optional yard node.

[0020] All optional yard nodes whose matching degree is greater than or equal to the matching degree threshold are counted to obtain S optional yard nodes.

[0021] Further, the calculation of the average spatial distance between each optional storage yard node and all exit nodes includes:

[0022] a1: Propose the i-th optional yard node from the S optional yard nodes, and select the j-th exit node from the Q exit nodes, where i and j are integers greater than zero;

[0023] a2: Calculate the spatial distance between the i-th optional yard node and the j-th exit node using the Euclidean distance, set j=j+1, and return to step a1;

[0024] a3: Repeat the above steps a1 to a2 until j=Q, and the loop ends, and the spatial distance between the i-th optional yard node and all exit nodes is obtained;

[0025] a4: Calculate the average of the spatial distances between the i-th optional storage yard node and all exit nodes to obtain the average spatial distance between the i-th optional storage yard node and all exit nodes, set i=i+1, and return to step a1;

[0026] a5: Repeat the above steps a1 to a4 until the loop ends when i=P, and obtain the average spatial distance between each optional yard node and all exit nodes.

[0027] Further, the step of selecting K candidate storage yard nodes from the S optional storage yard nodes includes:

[0028] Extract the average spatial distance between each optional yard node and all exit nodes, and compare the average spatial distance with the preset average spatial distance;

[0029] If the average spatial distance is less than or equal to the average spatial distance, the corresponding optional yard node is used as a candidate yard node;

[0030] If the average spatial distance is greater than the average spatial distance, the corresponding optional yard node is regarded as a non-candidate yard node;

[0031] Count all candidate yard nodes and obtain K candidate yard nodes.

[0032] Further, the transport path set includes L transport paths, where L is an integer greater than zero;

[0033] The step of selecting the optimal transportation path corresponding to each candidate storage yard node from the transportation path set includes:

[0034] b1: extract the transport path set of the h-th candidate yard node, where h is an integer greater than zero;

[0035] b2: Initialize the population: Generate an original population according to the transportation path set of the hth candidate yard node, wherein the original population contains X individuals, each of which represents a transportation path, and X is an integer greater than zero;

[0036] b3: Fitness evaluation: Under each individual, obtain the required unloading time of the ship to be unloaded; input the required unloading time into the pre-built fitness function to calculate the fitness of each individual;

[0037] b4: Selection: Use the roulette method to select two individuals with high fitness in the original population as the father and mother;

[0038] b5: Crossover: Perform crossover operation on the father and mother to produce new individuals;

[0039] b6: Mutation: Perform mutation operation on the new individuals to obtain Y new individuals, combine the Y new individuals into a new population, replace the original population with the new population, and return to step b3;

[0040] b7: Repeat steps b2 to b6 until the fitness of the individuals in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, output the transportation path represented by the corresponding individual as the optimal transportation path, set h=h+1, and return to step b1;

[0041] b8: Repeat the above steps b1 to b7 until the loop ends when h=L, and obtain the optimal transportation path corresponding to each candidate yard node.

[0042] Furthermore, the method for obtaining the required uninstallation time is as follows:

[0043] Acquire the path length of the transport path, the first task attribute data and the attribute data of the unloading equipment, wherein the attribute data of the unloading equipment includes the type of unloading equipment, the number of each type of unloading equipment and the average operating speed, and the types of the unloading equipment include quay cranes, AGV transport vehicles and cranes;

[0044] The path length of the transport path, the first task attribute data and the attribute data of the unloading equipment are integrated into unloading feature data, and the unloading feature data is input into a pre-trained machine learning model to obtain the required unloading time of the ship to be unloaded;

[0045] The training method of the machine learning model is as follows:

[0046] Acquire historical unloading time training data, and divide the historical unloading time training data into an unloading time training set and an unloading time test set, wherein the historical unloading time training data includes unloading feature data and its corresponding required unloading duration;

[0047] A regression network is constructed, unloading feature data in the unloading time training set is used as input of the regression network, and the required unloading duration in the unloading time training set is used as output of the regression network, and the regression network is trained to obtain an initial regression network;

[0048] The initial regression network is model verified using the unloading time test set, and the initial regression network with a value less than or equal to the preset test error threshold is output as the trained machine learning model.

[0049] Furthermore, the calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is a natural constant, The required uninstallation time.

[0050] Furthermore, the step of selecting a target storage yard node from the K candidate storage yard nodes includes:

[0051] Sort the fitness of the optimal transportation path corresponding to each candidate yard node from small to large;

[0052] The candidate yard node that has the highest fitness ranking is marked as the target yard node.

[0053] A container data processing system based on big data, comprising:

[0054] A data processing module is used to obtain the task attribute data of the ship to be unloaded and the yard layout diagram in the port, and generate an undirected graph according to the yard layout diagram; the undirected graph contains multiple edges and multiple nodes, and the nodes are divided into a target entry node, P yard nodes and Q exit nodes, and P and Q are both integers greater than zero;

[0055] The first screening module is used to obtain the load attribute data of each yard node, and screen S optional yard nodes from P yard nodes according to the load attribute data and task attribute data, where S is an integer greater than zero;

[0056] The second screening module is used to calculate the average spatial distance between each optional yard node and all exit nodes, and screen out K candidate yard nodes from the S optional yard nodes according to the average spatial distance, where K is an integer greater than zero;

[0057] The path optimization module is used to obtain a set of transportation paths between each candidate yard node and the target entry node in the undirected graph, and use a pre-configured genetic algorithm to filter out the optimal transportation path corresponding to each candidate yard node from the transportation path set to obtain multiple optimal transportation paths;

[0058] The scheme generation module is used to obtain the fitness of each optimal transportation path, select the target yard node from K candidate yard nodes according to the fitness, and generate the transportation scheduling scheme for the ship to be unloaded based on the target yard node and the optimal transportation path corresponding to the target yard node.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] The present application discloses a container data processing method and system based on big data, including: generating an undirected graph according to a yard layout diagram; selecting optional yard nodes according to load-bearing attribute data and task attribute data; calculating the average spatial distance between each optional yard node and all exit nodes, and selecting candidate yard nodes according to the average spatial distance; obtaining a set of transportation paths between each candidate yard node and a target entry node, and selecting the optimal transportation path corresponding to each candidate yard node by using a genetic algorithm; selecting a target yard node according to the fitness value, and generating a transportation scheduling plan according to the target yard node and the optimal transportation path corresponding to the target yard node; based on the above technical features, the present invention can dynamically respond to changes in unloading demand, comprehensively consider the ship's task attributes, yard resource distribution and transportation path characteristics, and solve the problems of static planning, unreasonable resource utilization and low subsequent transfer efficiency in traditional unloading plans; while improving unloading efficiency, it can optimize yard resource allocation, reduce transportation costs, ensure the scientific connection between unloading operations and subsequent logistics links, significantly improve the overall port operation efficiency, and provide an intelligent and dynamic solution for port logistics management. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A flowchart of a container data processing method based on big data provided by the present invention;

[0062] Figure 2 A schematic diagram of the module structure of a container data processing system based on big data provided by the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example 1

[0064] See also Figure 1 As shown, this embodiment discloses a container data processing method based on big data, and the method includes:

[0065] S101: Obtain task attribute data of the ship to be unloaded and a yard layout diagram in the port, and generate an undirected graph according to the yard layout diagram; the undirected graph contains multiple edges and multiple nodes, and the nodes are divided into a target entry node, P yard nodes and Q exit nodes, and P and Q are both integers greater than zero;

[0066] Specifically, the task attribute data includes first task attribute data and second task attribute data, the first task attribute data is the number of containers to be unloaded by the ship to be unloaded, and the second task attribute data includes multiple attributes of the containers to be unloaded, including but not limited to type, size and weight, etc.;

[0067] For example, the type is one of a refrigerated container, a dangerous goods container or a normal container, the size is one of 20 feet or 40 feet, and the weight is one of 5 tons, 10 tons or 15 tons;

[0068] It should be noted that: the task attribute data of the ship to be unloaded is obtained through Internet transmission or registration records, and the yard layout diagram in the port is pre-stored in the system database, and the yard layout diagram includes P yard nodes, (yard) entrance nodes and (yard) exit nodes, etc., and discloses information such as the number and location of each node; among which, the (yard) entrance node is the berthing terminal of the ship to be unloaded;

[0069] It should be understood that: the undirected graph is generated based on the graph theory method, and its prior art has the following generation process: extract all key positions from the yard layout diagram, which are defined as the node set V of the graph, wherein each node is represented by a unique identifier (such as v1, v2, ...), wherein the key positions include: entry nodes, representing the target point for ship unloading, yard nodes, representing the storage area, and exit nodes, representing the transportation destination of the goods; extract all paths between the nodes from the layout diagram, which are defined as the edge set E, and if there is a path between any two nodes, a corresponding edge is created in the undirected graph, wherein the edge is an undirected edge, indicating that there is bidirectional traffic between the two nodes, and weight information is attached to each edge according to information such as the distance between the nodes; finally, the topological structure of the undirected graph is constructed according to the node set V, the edge set E and the connection relationship between the nodes to obtain the undirected graph.

[0070] S102: Obtaining the load attribute data of each yard node, and selecting S optional yard nodes from P yard nodes according to the load attribute data and the task attribute data, where S is an integer greater than zero;

[0071] Specifically, the load-bearing attribute data includes first load-bearing attribute data and second load-bearing attribute data, wherein the first load-bearing attribute data is the number of containers that the yard node can carry, and the second load-bearing attribute data includes but is not limited to multiple attributes of the containers that can be carried, including but not limited to type, size, and weight, etc.; it is worth noting that the data types of the load-bearing attribute data and the task attribute data are completely consistent;

[0072] In implementation, the step of selecting S optional storage yard nodes from P storage yard nodes includes:

[0073] Calculate the difference between the number of containers to be unloaded and the number of containers that can be unloaded according to the first load attribute data and the first task attribute data;

[0074] The quantity difference is compared with a preset quantity difference threshold. If the quantity difference is greater than or equal to the quantity difference threshold, the corresponding yard node is marked as an alternative yard node. If the quantity difference is less than the quantity difference threshold, the corresponding yard node is marked as a non-alternative yard node.

[0075] Count all candidate storage yard nodes whose quantity difference is greater than or equal to the quantity difference threshold, and obtain multiple candidate storage yard nodes;

[0076] Calculate the matching degree between each candidate yard node and the ship to be unloaded according to the second load-bearing attribute data and the second task attribute data;

[0077] The matching degree is calculated using the cosine similarity function; the calculation formula is: ; In the formula, For the matching degree, is the attribute value of the rth attribute in the second attribute data in vector form, is the attribute value of the rth attribute in vector form in the second task attribute data, Represents the norm of a vector. For text attribute data such as type, the attribute value in vector form is obtained by converting it in advance through one-hot encoding or Word2Vec.

[0078] The matching degree is compared with a preset matching degree threshold. If the matching degree is greater than or equal to the matching degree threshold, the corresponding candidate yard node is marked as an optional yard node. If the matching degree is less than the matching degree threshold, the corresponding candidate yard node is marked as a non-optional yard node.

[0079] All optional yard nodes whose matching degree is greater than or equal to the matching degree threshold are counted to obtain S optional yard nodes.

[0080] S103: Calculate the average spatial distance between each optional yard node and all exit nodes, and select K candidate yard nodes from the S optional yard nodes according to the average spatial distance, where K is an integer greater than zero;

[0081] In implementation, the calculation of the average spatial distance between each optional yard node and all exit nodes includes:

[0082] a1: Propose the i-th optional yard node from the S optional yard nodes, and select the j-th exit node from the Q exit nodes, where i and j are integers greater than zero;

[0083] a2: Calculate the spatial distance between the i-th optional yard node and the j-th exit node using the Euclidean distance, set j=j+1, and return to step a1;

[0084] It should be noted that the spatial distance between the i-th optional storage yard node and the j-th exit node is obtained by using the Euclidean distance calculation after obtaining the spatial coordinates of the i-th optional storage yard node and the j-th exit node, and the calculation formula is: , where: are the spatial coordinates of the i-th optional storage yard node and the j-th exit node, is the spatial coordinate of the i-th optional yard node, is the spatial coordinate of the jth exit node;

[0085] a3: Repeat the above steps a1 to a2 until j=Q, and end the loop to obtain the spatial distance between the i-th optional yard node and all exit nodes;

[0086] a4: Calculate the average of the spatial distances between the i-th optional storage yard node and all exit nodes to obtain the average spatial distance between the i-th optional storage yard node and all exit nodes, set i=i+1, and return to step a1;

[0087] It should be noted that the average spatial distance between the i-th optional storage yard node and all exit nodes is obtained by mean calculation, and the calculation formula is: , where is the average spatial distance between the i-th optional yard node and all exit nodes, is the number of spatial distances, is the value of the g-th spatial distance;

[0088] a5: Repeat steps a1 to a4 above until the loop ends when i=P, and obtain the average spatial distance between each optional yard node and all exit nodes;

[0089] In implementation, the step of selecting K candidate storage yard nodes from the S optional storage yard nodes includes:

[0090] Extract the average spatial distance between each optional yard node and all exit nodes, and compare the average spatial distance with the preset average spatial distance;

[0091] If the average spatial distance is less than or equal to the average spatial distance, the corresponding optional yard node is used as a candidate yard node;

[0092] If the average spatial distance is greater than the average spatial distance, the corresponding optional yard node is regarded as a non-candidate yard node;

[0093] Count all candidate yard nodes and obtain K candidate yard nodes;

[0094] It should be understood that in existing container unloading scenarios, in order to improve unloading efficiency, the principle of proximity is usually adopted, that is, the containers to be unloaded on the ship to be unloaded are placed close to the nearest yard node, or the technical staff places the containers to be unloaded at the manually designated yard node according to the actual yard layout. No matter which of the above situations is used, this will lead to greater transshipment efficiency and cost issues for the unloaded containers during transshipment. A further explanation is that, since both the principle of proximity and the manually designated yard nodes lack scientific planning, the distances between the unloaded containers and various export nodes are different. Therefore, although the above two methods improve the unloading efficiency in the short term, they ignore the needs of the subsequent transshipment links of the containers, resulting in the lack of scientificity and global optimization of the overall process.

[0095] S104: in the undirected graph, a set of transportation paths between each candidate storage yard node and the target entry node is obtained, and an optimal transportation path corresponding to each candidate storage yard node is screened out from the transportation path set using a preconfigured genetic algorithm to obtain multiple optimal transportation paths;

[0096] Wherein, the transport path set includes L transport paths, and L is an integer greater than zero;

[0097] In implementation, the step of selecting the optimal transportation path corresponding to each candidate yard node from the transportation path set includes:

[0098] b1: extract the transport path set of the h-th candidate yard node, where h is an integer greater than zero;

[0099] b2: Initialize the population: Generate an original population according to the transportation path set of the hth candidate yard node, wherein the original population contains X individuals, each of which represents a transportation path, and X is an integer greater than zero;

[0100] b3: Fitness evaluation: Under each individual (i.e., each transport path in the original population), obtain the required unloading time of the ship to be unloaded; input the required unloading time into the pre-built fitness function to calculate the fitness of each individual;

[0101] The method for obtaining the required uninstallation time is as follows:

[0102] Acquire the path length of the transport path, the first task attribute data and the attribute data of the unloading equipment, wherein the attribute data of the unloading equipment includes the type of unloading equipment, the number of each type of unloading equipment and the average operating speed, and the types of the unloading equipment include quay cranes, AGV transport vehicles and cranes;

[0103] It should be understood that: the quay crane is used to transfer the containers to be unloaded from the ship to be unloaded to the AGV transport vehicle, the AGV transport vehicle is used to carry the containers to be unloaded to the yard node, and the crane is used to place the containers to be unloaded in the yard node; wherein, the path length of the transportation path is obtained through actual collection; the type of unloading equipment and the number of each unloading equipment are manually selected and determined according to actual conditions, and the average operating speed of each unloading equipment is determined according to historical actual records;

[0104] The path length of the transport path, the first task attribute data and the attribute data of the unloading equipment are integrated into unloading feature data, and the unloading feature data is input into a pre-trained machine learning model to obtain the required unloading time of the ship to be unloaded;

[0105] Specifically, the training method of the machine learning model is as follows:

[0106] Acquire historical unloading time training data, and divide the historical unloading time training data into an unloading time training set and an unloading time test set, wherein the historical unloading time training data includes unloading feature data and its corresponding required unloading duration;

[0107] It should be noted that the unloading characteristic data in the historical unloading time training data, including the path length of the transportation path, the first task attribute data and the attribute data of the unloading equipment, are all collected by technicians according to actual scenarios or experiments, and the required unloading duration in the historical unloading time training data is manually recorded according to the actual unloading duration in the actual scenarios or experiments;

[0108] A regression network is constructed, unloading feature data in the unloading time training set is used as input of the regression network, and the required unloading duration in the unloading time training set is used as output of the regression network, and the regression network is trained to obtain an initial regression network;

[0109] The regression network is specifically one of the regression algorithms such as linear regression, random forest regression, gradient boosted decision tree (GBDT) or neural network;

[0110] The initial regression network is validated using the unloading time test set, and the initial regression network with a value less than or equal to the preset test error threshold is output as the trained machine learning model.

[0111] The calculation formula of the pre-constructed fitness function is: ; Where: For fitness, is a natural constant, The required uninstallation time;

[0112] b4: Selection: Use the roulette method to select two individuals with high fitness in the original population as the father and mother;

[0113] The roulette wheel method is a commonly used selection method, which is used in genetic algorithms to select individuals with higher fitness to enter the next generation. It simulates the process of roulette, and each individual obtains a corresponding "roulette wheel" area according to its fitness. The higher the fitness of the individual, the larger the corresponding area, and the higher the probability of being selected.

[0114] b5: Crossover: Perform crossover operation on the father and mother to produce new individuals;

[0115] It should be noted that the crossover operation on the paternal parent and the maternal parent is implemented based on a crossover operation, and the crossover operation includes but is not limited to one of a single-point crossover, a uniform crossover or a sequential crossover, etc.;

[0116] b6: Mutation: Perform mutation operation on the new individuals to obtain Y new individuals, combine the Y new individuals into a new population, replace the original population with the new population, and return to step b3;

[0117] In genetic algorithms, mutation operations are used to introduce genetic diversity and prevent the algorithm from falling into a local optimum. The mutation operation on new individuals is implemented by uniform mutation or Gaussian mutation.

[0118] b7: Repeat steps b2 to b6 until the fitness of the individuals in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, output the transportation path represented by the corresponding individual as the optimal transportation path, set h=h+1, and return to step b1;

[0119] For example: Assume that the maximum number of iterations is 100 times, and record the individual with the highest fitness in the current population and its fitness value after each iteration; if it is found that the fitness value has not changed significantly in a certain generation, it is considered that the convergence condition is met, the iteration is stopped, and the transportation path represented by the corresponding individual is output as the optimal transportation path;

[0120] b8: Repeat the above steps b1 to b7 until the loop ends when h=L, and obtain the optimal transportation path corresponding to each candidate yard node.

[0121] S105: Obtain the fitness of each optimal transportation path, select the target yard node from the K candidate yard nodes according to the fitness, and generate a transportation scheduling plan for the ship to be unloaded according to the target yard node and the optimal transportation path corresponding to the target yard node;

[0122] In implementation, the step of selecting a target storage yard node from K candidate storage yard nodes includes:

[0123] Sort the fitness of the optimal transportation path corresponding to each candidate yard node from small to large;

[0124] The candidate yard node that has the highest ranking fitness is marked as the target yard node;

[0125] It should be understood that: when the target yard node is screened out, the optimal transportation path corresponding to the target yard node is extracted, and finally, the transportation scheduling plan for the ship to be unloaded is generated according to the target yard node and the optimal transportation path corresponding to the target yard node, and the plan is fed back to the unloading scheduling personnel and unloading equipment to enable them to execute the unloading process of the container of the ship to be unloaded;

[0126] By acquiring the task attribute data of the ship to be unloaded and the layout information of the port yard, an undirected graph model based on graph theory is constructed, and the optional yard nodes are screened in combination with the yard load attribute data, and the transportation path is further optimized by average spatial distance calculation and genetic algorithm to generate an efficient unloading scheduling plan; the method of the present invention can dynamically respond to changes in unloading demand, comprehensively consider the task attributes of the ship, the distribution of yard resources and the characteristics of the transportation path, and solve the problems of static planning, unreasonable resource utilization and low subsequent transportation efficiency in traditional unloading plans; through the technical solution of the present invention, the unloading efficiency can be improved while optimizing the yard resource allocation, reducing transportation costs, ensuring the scientific connection between unloading operations and subsequent logistics links, significantly improving the overall operation efficiency of the port, and providing an intelligent and dynamic solution for port logistics management. Example 2

[0127] See also Figure 2 As shown, based on the same inventive concept, this embodiment discloses a container data processing system based on big data. For details not provided in this embodiment, please refer to the description of the relevant parts in Embodiment 1. The system includes:

[0128] The data processing module 210 is used to obtain the task attribute data of the ship to be unloaded and the yard layout diagram in the port, and generate an undirected graph according to the yard layout diagram; the undirected graph contains multiple edges and multiple nodes, and the nodes are divided into a target entry node, P yard nodes and Q exit nodes, and P and Q are both integers greater than zero;

[0129] The first screening module 220 is used to obtain the load attribute data of each yard node, and screen S optional yard nodes from the P yard nodes according to the load attribute data and the task attribute data, where S is an integer greater than zero;

[0130] The second screening module 230 is used to calculate the average spatial distance between each optional yard node and all exit nodes, and screen out K candidate yard nodes from the S optional yard nodes according to the average spatial distance, where K is an integer greater than zero;

[0131] The path optimization module 240 is used to obtain a set of transportation paths between each candidate yard node and the target entry node in the undirected graph, and to select an optimal transportation path corresponding to each candidate yard node from the transportation path set by using a preconfigured genetic algorithm to obtain multiple optimal transportation paths;

[0132] The solution generation module 250 is used to obtain the fitness of each optimal transportation path, select the target yard node from the K candidate yard nodes according to the fitness, and generate a transportation scheduling solution for the ship to be unloaded based on the target yard node and the optimal transportation path corresponding to the target yard node.

[0133] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters, weights and thresholds in the formula are set by technicians in this field according to actual conditions.

[0134] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired network or a wireless network. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media sets. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD) or a semiconductor medium. The semiconductor medium may be a solid-state hard disk.

[0135] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0137] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0140] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0141] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A container data processing method based on big data, characterized in that: The method comprises: Obtaining the task attribute data of the ship to be unloaded and the yard layout diagram in the port, and generating an undirected graph according to the yard layout diagram; the undirected graph contains multiple edges and multiple nodes, and the nodes are divided into a target entry node, P yard nodes and Q exit nodes, where P and Q are both integers greater than zero; wherein the entry node represents the target point for ship unloading, the yard node represents the storage area, and the exit node represents the transportation destination of the goods; Obtain the load attribute data of each yard node, and select S optional yard nodes from P yard nodes according to the load attribute data and task attribute data, where S is an integer greater than zero; The step of selecting S optional storage yard nodes from P storage yard nodes includes: Calculate the difference between the number of containers to be unloaded and the number of containers that can be unloaded according to the first load attribute data and the first task attribute data; The quantity difference is compared with a preset quantity difference threshold. If the quantity difference is greater than or equal to the quantity difference threshold, the corresponding yard node is marked as an alternative yard node. If the quantity difference is less than the quantity difference threshold, the corresponding yard node is marked as a non-alternative yard node. Count all candidate storage yard nodes whose quantity difference is greater than or equal to the quantity difference threshold, and obtain multiple candidate storage yard nodes; Calculate the matching degree between each candidate yard node and the ship to be unloaded according to the second load-bearing attribute data and the second task attribute data; The matching degree is compared with a preset matching degree threshold. If the matching degree is greater than or equal to the matching degree threshold, the corresponding candidate yard node is marked as an optional yard node. If the matching degree is less than the matching degree threshold, the corresponding candidate yard node is marked as a non-optional yard node. Count all optional yard nodes whose matching degree is greater than or equal to the matching degree threshold, and obtain S optional yard nodes; Calculate the average spatial distance between each optional yard node and all exit nodes, and select K candidate yard nodes from the S optional yard nodes according to the average spatial distance, where K is an integer greater than zero; In the undirected graph, a set of transportation paths between each candidate yard node and the target entrance node is obtained, and a preconfigured genetic algorithm is used to screen out the optimal transportation path corresponding to each candidate yard node from the transportation path set to obtain multiple optimal transportation paths; The fitness of each optimal transportation path is obtained, and the target yard node is selected from the K candidate yard nodes according to the fitness size. Then, the transportation scheduling plan for the ship to be unloaded is generated according to the target yard node and the optimal transportation path corresponding to the target yard node.

2. The container data processing method based on big data according to claim 1 is characterized in that: The task attribute data includes first task attribute data and second task attribute data, the first task attribute data is the number of containers to be unloaded by the ship to be unloaded, and the second task attribute data includes multiple attributes of the containers to be unloaded; the carrying attribute data includes first carrying attribute data and second carrying attribute data, the first carrying attribute data is the number of containers that the yard node can carry, and the second carrying attribute data includes multiple attributes of the containers that can be carried, and the attributes include type, size and weight.

3. The container data processing method based on big data according to claim 2 is characterized in that: The calculation of the average spatial distance between each optional storage yard node and all exit nodes includes: a1: Propose the i-th optional yard node from the S optional yard nodes, and select the j-th exit node from the Q exit nodes, where i and j are integers greater than zero; a2: Calculate the spatial distance between the i-th optional storage yard node and the j-th exit node using the Euclidean distance, set j=j+1, and return to step a1; a3: Repeat the above steps a1 to a2 until j = Q, and the loop ends, and the spatial distance between the i-th optional yard node and all exit nodes is obtained; a4: Calculate the average of the spatial distances between the i-th optional storage yard node and all exit nodes to obtain the average spatial distance between the i-th optional storage yard node and all exit nodes, set i=i+1, and return to step a1; a5: Repeat the above steps a1 to a4 until the loop ends when i=P, and obtain the average spatial distance between each optional yard node and all exit nodes.

4. The container data processing method based on big data according to claim 3 is characterized in that: The step of selecting K candidate storage yard nodes from the S optional storage yard nodes includes: Extract the average spatial distance between each optional yard node and all exit nodes, and compare the average spatial distance with the preset average spatial distance; If the average spatial distance is less than or equal to the average spatial distance, the corresponding optional yard node is used as a candidate yard node; If the average spatial distance is greater than the average spatial distance, the corresponding optional yard node is regarded as a non-candidate yard node; Count all candidate yard nodes and obtain K candidate yard nodes.

5. The container data processing method based on big data according to claim 4 is characterized in that: The transport path set includes L transport paths, where L is an integer greater than zero; The step of selecting the optimal transportation path corresponding to each candidate storage yard node from the transportation path set includes: b1: extract the transport path set of the h-th candidate yard node, where h is an integer greater than zero; b2: Initialize the population: Generate an original population according to the transportation path set of the hth candidate yard node, wherein the original population contains X individuals, each of which represents a transportation path, and X is an integer greater than zero; b3: Fitness evaluation: Under each individual, obtain the required unloading time of the ship to be unloaded; input the required unloading time into the pre-built fitness function to calculate the fitness of each individual; b4: Selection: Use the roulette method to select two individuals with high fitness in the original population as the father and mother; b5: Crossover: Perform crossover operation on the father and mother to produce new individuals; b6: Mutation: Perform mutation operation on the new individuals to obtain Y new individuals, combine the Y new individuals into a new population, replace the original population with the new population, and return to step b3; b7: Repeat the above steps b2 to b6 until the fitness of the individuals in the original population or the new population is greater than or equal to the preset fitness threshold, or the number of iterations is greater than or equal to the preset maximum number of iterations threshold, output the transportation path represented by the corresponding individual as the optimal transportation path, set h=h+1, and return to step b1; b8: Repeat the above steps b1 to b7 until the loop ends when h=L, and obtain the optimal transportation path corresponding to each candidate yard node.

6. The container data processing method based on big data according to claim 5 is characterized in that: The method for obtaining the required uninstallation time is as follows: Acquire the path length of the transport path, the first task attribute data and the attribute data of the unloading equipment, wherein the attribute data of the unloading equipment includes the type of unloading equipment, the number of each type of unloading equipment and the average operating speed, and the types of the unloading equipment include quay cranes, AGV transport vehicles and cranes; The path length of the transport path, the first task attribute data and the attribute data of the unloading equipment are integrated into unloading feature data, and the unloading feature data is input into a pre-trained machine learning model to obtain the required unloading time of the ship to be unloaded; The training method of the machine learning model is as follows: Acquire historical unloading time training data, and divide the historical unloading time training data into an unloading time training set and an unloading time test set, wherein the historical unloading time training data includes unloading feature data and its corresponding required unloading duration; A regression network is constructed, unloading feature data in the unloading time training set is used as input of the regression network, and the required unloading duration in the unloading time training set is used as output of the regression network, and the regression network is trained to obtain an initial regression network; The initial regression network is model verified using the unloading time test set, and the initial regression network with a value less than or equal to the preset test error threshold is output as the trained machine learning model.

7. The container data processing method based on big data according to claim 6 is characterized in that: The calculation formula of the pre-constructed fitness function is: Fitness=e-α; wherein: Fitness is fitness, e is a natural constant, and α is the required unloading time.

8. The container data processing method based on big data according to claim 7 is characterized in that: The step of selecting a target storage yard node from the K candidate storage yard nodes includes: Sort the fitness of the optimal transportation path corresponding to each candidate yard node from small to large; The candidate yard node that has the highest fitness ranking is marked as the target yard node.

9. A container data processing system based on big data, implemented based on the container data processing method based on big data according to any one of claims 1 to 8, characterized in that: include: A data processing module is used to obtain the task attribute data of the ship to be unloaded and the yard layout diagram in the port, and generate an undirected graph according to the yard layout diagram; the undirected graph contains multiple edges and multiple nodes, and the nodes are divided into a target entry node, P yard nodes and Q exit nodes, and P and Q are both integers greater than zero; The first screening module is used to obtain the load attribute data of each yard node, and screen S optional yard nodes from P yard nodes according to the load attribute data and task attribute data, where S is an integer greater than zero; The second screening module is used to calculate the average spatial distance between each optional yard node and all exit nodes, and screen out K candidate yard nodes from the S optional yard nodes according to the average spatial distance, where K is an integer greater than zero; The path optimization module is used to obtain a set of transportation paths between each candidate yard node and the target entry node in the undirected graph, and use a pre-configured genetic algorithm to filter out the optimal transportation path corresponding to each candidate yard node from the transportation path set to obtain multiple optimal transportation paths; The scheme generation module is used to obtain the fitness of each optimal transportation path, select the target yard node from K candidate yard nodes according to the fitness, and generate the transportation scheduling scheme for the ship to be unloaded based on the target yard node and the optimal transportation path corresponding to the target yard node.

Images