Container intelligent transportation path prediction planning method based on YOLO-World and Transform

Abstract

The invention discloses a container intelligent transportation path predicting and planning method based on YOLO-World and Transform. The method comprises the steps of optimizing the position of a camera, collecting and preprocessing data in real time, detecting the YOLO-World, tracking through Kalman filtering, calculating the speed and the direction through an RAFT optical flow algorithm, predicting the path through the Transform, and planning and optimizing the path through a D * algorithm. According to the method, the intelligence of the container transportation path is realized, the prediction accuracy and the path optimization are excellent, the logistics transportation efficiency is improved, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of container transport path planning, and in particular to a container intelligent transport path prediction and planning method based on YOLO-World and Transformer. Background Art

[0002] Intelligent container transportation path prediction and planning is a topic of great significance in the field of logistics and transportation. With the growth of global trade and the increase in transportation demand, how to effectively manage and optimize the transportation path of containers has become the key. At present, container transportation path planning mainly relies on manual experience and traditional algorithms, but this method is often inefficient and cannot fully adapt to the real-time changing transportation needs. YOLO-World is an advanced target detection algorithm with real-time processing capabilities and high accuracy. It is able to identify and detect the location and status of containers at the transportation site. This provides strong support for real-time data collection and monitoring. On the other hand, Transformer is a deep learning architecture for sequence modeling that can process complex time series data and is suitable for predicting the future movement path of containers. Kalman filtering and RAFT optical flow algorithms are technologies used for tracking and speed calculation. Through these algorithms, the movement of containers can be stably tracked and their speed and direction can be accurately calculated. The combination of these technologies lays the foundation for intelligent path prediction and planning. The D* algorithm is a flexible path planning algorithm that can dynamically adjust the optimal path based on real-time data. It can plan the optimal transportation route based on the predicted path of the container and the current traffic conditions to ensure the efficiency and flexibility of the transportation process. By integrating these technologies, the intelligent container transportation path prediction and planning solution based on YOLO-World and Transformer can significantly improve transportation efficiency, reduce costs, and reduce negative impacts on the environment. The application of this technology will drive the logistics and transportation industry towards a more intelligent and sustainable development path.

[0003] Therefore, the present invention provides a container intelligent transportation path prediction and planning method based on YOLO-World and Transformer for real-time monitoring and prediction of the movement path of the container. Summary of the invention

[0004] The present invention provides a container intelligent transportation path prediction and planning method based on YOLO-World and Transformer for real-time monitoring and prediction of the movement path of a container.

[0005] The purpose of the present invention is to provide a container intelligent transportation path prediction and planning method based on YOLO-World and Transformer, the method comprising the following steps:

[0006] Step 1: Optimize camera positions: Measure the best coverage of cameras through triangulation methods and verify the layout to obtain clear images of container transportation scenes;

[0007] Step 2, data acquisition and preprocessing: Use the optimized camera to collect data and perform preprocessing steps such as denoising and brightness adjustment;

[0008] Step 3: Real-time detection: Use YOLO-World to detect containers in real time;

[0009] Step 4: Tracking: Use the Kalman filter to stably track the position and trajectory of the container, so that the container can be continuously tracked in a complex environment;

[0010] Step 5: Calculate speed and direction: Calculate the speed and direction of movement of the container using the RAFT optical flow algorithm to provide a basis for subsequent path prediction;

[0011] Step 6: Path prediction: Use Transformer to predict the future movement path of the container and provide accurate path prediction based on its historical data and current status;

[0012] Step 7: Path planning and optimization: Based on the D* method, combined with prediction results and real-time data, the best transportation path is provided for the container, and the plan is dynamically adjusted to adapt to the changing environment.

[0013] Furthermore, the specific steps of step one are as follows:

[0014] Step 1: Scenario analysis: In the container transportation scenario, analyze the site layout, mark key areas, including container storage areas, loading and unloading areas, and traffic routes, and identify potential obstructions, including buildings and vehicles;

[0015] Step 2: Triangulation: According to the results of scene analysis, select the preliminary camera position, consider the distance and relative angle between cameras, use triangulation method to calculate the ideal spacing and angle between cameras, cover the entire transportation area, calculate the camera layout, first calculate the distance between the two cameras, through the Euclidean distance formula, when the coordinates of the camera are (x1, y1, z1) and (x2, y2, z2), then the distance can be calculated as:

[0016]

[0017] Triangulation calculation, when the distance d between the two cameras and the angles θ1 and θ2 are known, the included angle is calculated using the inverse tangent function. The calculation formula is:

[0018]

[0019] Step 3: Camera installation: According to the results of triangulation, install the camera, adjust the camera angle, cover the entire container transportation scene, calculate the camera's viewing angle, and use the camera's focal length and the size of the photosensitive element. The focal length of the camera is f, and the width of the photosensitive element is w. The viewing angle can be calculated as follows:

[0020]

[0021] Step 4: Camera calibration and verification: After installing the camera, calibrate it, verify the camera coverage by testing the video stream, and check the overlapping area between cameras. The camera coverage area is calculated based on the viewing angle and distance. Based on the camera height h and viewing angle α, the camera coverage calculation formula is:

[0022]

[0023] Furthermore, the specific steps of step 2 are as follows:

[0024] Step 1: Data collection preparation: Check the camera installation and calibration status;

[0025] Step 2: Data collection: Start the data collection system, collect image data according to the designed frequency and resolution, and check the camera status regularly during the collection process;

[0026] Step 3: Preliminary preprocessing: Use a denoising filter to remove random noise from the image;

[0027] Step 4. Brightness adjustment: Check the brightness and contrast of the image and adjust the level;

[0028] Step 5: Image enhancement: Perform sharpening processing as needed to improve the image clarity and edge contrast;

[0029] Step 6. Data verification: After preprocessing, perform a quality check on the image data and use sample data for detection testing.

[0030] Furthermore, the specific steps of step three are as follows:

[0031] Step 1: Model initialization: load the pre-trained YOLO-World model and check the model parameters;

[0032] Step 2: Input preprocessing: resize the preprocessed image data to meet the input requirements of the YOLO-World model, and adjust the image data to the specific input size W×H required by the YOLO-World model, 416×416 pixels;

[0033] Step 3: Real-time detection: pass the preprocessed image data to the YOLO-World model to perform real-time target detection, set the detection threshold T to 0.5, so that the model can accurately identify container targets, filter out unnecessary detection results, and record the detected container location and size;

[0034] Step 4: Result verification: Verify the test results by visualizing the test results and comparing them with the known container locations. If the test results are erroneous or unstable, adjust the model parameters and preprocessing methods, and re-test.

[0035] Step 5: Processing of detection results: Based on the location and size of the container detected by YOLO-World, the coordinates and size information of the target are extracted and passed to the subsequent tracking stage so that the location of the container can be accurately tracked.

[0036] Furthermore, the specific steps of step 4 are as follows:

[0037] Step 1: Initialize the Kalman filter: Before tracking the container, first initialize the Kalman filter. This stage includes setting the initial state and covariance matrix, determining the state transition model and observation model, and defining the process noise and observation noise. The initial state is generally based on the detection results, including the container position and speed provided by YOLO-World.

[0038] Step 2: State prediction: At each time step, the Kalman filter predicts the next position and motion state of the container based on the state transition model. The prediction step uses the previous state and process noise estimate to generate a predicted state vector and covariance matrix. The goal of this step is to provide an estimate of the future position of the container. The state prediction uses the following formula:

[0039] x k+1 =Fx k +u k

[0040] Among them, x k is the current state vector, F is the state transfer matrix, u is the process noise,

[0041] Covariance forecasting uses the following formula:

[0042] P k+1 =FP k F T +Q

[0043] Among them, P k is the current covariance matrix, and Q is the process noise covariance matrix.

[0044] Step 3: Observation update: When new observation data is available, the Kalman filter corrects the predicted state through observation update. This step includes calculating the Kalman gain and updating the state vector and covariance matrix according to the new observation data. The Kalman gain calculation formula is:

[0045] K=P k H T (HP k H T +R) -1

[0046] Where H is the observation model matrix, R is the observation noise covariance matrix,

[0047] The update status formula is:

[0048] x k+1 =x k +K(z-Hx k )

[0049] Among them, z is the observed data,

[0050] The covariance update formula is:

[0051] P k+1 =(KH)P k

[0052] Among them, P k is the updated covariance matrix, K is the Kalman gain, and H is the observation matrix;

[0053] Step 4: Tracking calibration: In the real-time tracking process, there will be problems such as camera jitter and light changes, which will lead to unstable observation data. In order to maintain stable tracking, the Kalman filter needs to be calibrated to reduce errors caused by external factors, which involves adjusting process noise and observation noise, as well as introducing environmental adaptability constraints and prediction consistency constraints in state prediction, so that the Kalman filter can maintain high tracking accuracy and stability in a dynamic and changing environment.

[0054] Step 5: Trajectory output: The final result of the Kalman filter is a stable container trajectory output. This trajectory information, including position, speed, and direction, is used for subsequent path prediction and planning.

[0055] Furthermore, the specific steps of step five are as follows:

[0056] Step 1: Prepare image sequences: Prepare continuous sequences of image frames from real-time cameras, fixed surveillance cameras, container transport vehicle cameras, and portable cameras of field workers so that the image sequences have sufficient temporal resolution. This stage involves adjusting the image size to meet the input requirements of the RAFT algorithm.

[0057] Step 2: Optical flow calculation: pass the image sequence to the RAFT optical flow algorithm to calculate the motion vector of each pixel. The result of the optical flow calculation is a vector field of the same size as the input image, indicating the moving direction and distance of each pixel between the previous and next frames.

[0058] Step 3: Extract the speed and direction of the container: After the optical flow calculation is completed, extract the speed and direction information related to the container. Through the previous Kalman filter tracking results, determine the area of ​​the container in the optical flow vector field. Then, by calculating the average vector in this area, the speed and direction of the container are obtained. The specific calculation of speed and direction is as follows:

[0059] Optical flow field:

[0060] u(x, y) = u x (x,y),u y (x, y)

[0061] Among them, u x and u y represent the movement along the x-axis and y-axis respectively,

[0062] Optical flow equation:

[0063]

[0064] Where I is the image intensity, and is the gradient of the image on the x-axis and y-axis, is the time gradient,

[0065] speed:

[0066]

[0067] direction:

[0068]

[0069] Overall container speed:

[0070]

[0071]

[0072] Where n is the number of pixels in the container area;

[0073] Step 4: Data smoothing: To improve the stability and reliability of speed and direction data, the Kalman filter will continue to be used for data smoothing to reduce the impact of noise and outliers.

[0074] Step 5: Result verification: After calculating the speed and direction, the results need to be verified by comparing the calculated results with actual observations and testing the accuracy of the optical flow calculation through simulation;

[0075] Step 6: Pass to path prediction: The final speed and direction data will be passed to the subsequent path prediction stage so that the data format and accuracy meet the requirements of path prediction and accurately predict the future movement trajectory of the container.

[0076] Furthermore, the specific steps of step six are as follows:

[0077] Step 1: Data preparation: Obtain historical data of the container from the previous stage, including location, speed, and direction, and convert this data into a format suitable for Transformer input. When the data needs to be converted into a time series format, each data point represents a time step.

[0078] Step 2: Model initialization: Initialize the Transformer model, set model parameters, including the number of layers, heads, and hidden units, so that the model configuration matches the path prediction task, load the pre-trained model, and retrain the model;

[0079] Step 3: Multi-head attention: Use the multi-head attention mechanism to capture important patterns and relationships in the data. Multi-head attention determines which data points are important in the sequence by calculating attention weights.

[0080] Step 4: Position encoding: In order to make the model understand the temporal relationship in the time series data, position encoding is applied. Position encoding adds unique time information to each time step through sine and cosine functions, allowing Transformer to recognize the temporal relationship of the data;

[0081] Step 5: Training and reasoning: Pass the data to the Transformer model for model training and reasoning. During the training process, the loss function is used to measure the accuracy of the prediction results. For path prediction, the goal of the model is to minimize the error between the predicted path and the actual path. The loss function is calculated as follows:

[0082] Mean Squared Error (MSE):

[0083]

[0084] Among them, y i is the actual path data, is the predicted path data, and n is the number of data points.

[0085] Mean Absolute Error (MAE):

[0086]

[0087] Cross Entropy Loss:

[0088]

[0089] Among them, y i is the actual category, is the predicted probability.

[0090] Synthetic loss:

[0091] TotalLOSSS = 0·MSE + β·MAE

[0092] Among them, α and β are weights that adjust the contribution of different loss functions;

[0093] Step 6: Result output: When the model training or inference is completed, the predicted container path is output. The prediction result is the location information of several future time steps, indicating the movement trajectory of the container in the future. The prediction result is used for subsequent path planning and optimization;

[0094] Step 7. Result verification: In order to ensure the accuracy of the prediction, it is necessary to verify the prediction results by comparing them with the actual data and testing the accuracy of the model through simulation.

[0095] Furthermore, the specific steps of step seven are as follows:

[0096] Step 1: Initialize path planning: According to the predicted container movement path and the current environment status, initialize the path planning, including the starting point, end point and passable area of ​​the container, and consider the obstacles and traffic conditions in the environment;

[0097] Step 2: Set planning parameters: Set the parameters of the D* algorithm, including the heuristic estimation function and the search depth. The heuristic estimation is usually based on the straight-line distance of the path and other metrics to guide the algorithm to find the path.

[0098] Step 3: Path planning: Use the D* algorithm to plan the path. The algorithm calculates the shortest path from the starting point to the end point based on heuristic estimation and the location of known obstacles. The planning result is a sequence of path nodes. The path cost is the cost from the starting point to the previous node. The specific calculation is as follows:

[0099] Path cost:

[0100] g(x)=g(xprev)+cost(xprev,x)

[0101] Among them, g(x prev ) is the path cost to the previous node, cost(x prev , x) is the cost from the previous node to the current node.

[0102] Total path cost:

[0103] f(x)=g(x)+h(x)

[0104] Where g(x) is the path cost and h(x) is the heuristic estimate;

[0105] Step 4: Dynamic Adjustment: The key feature of the D* algorithm is the dynamic adjustment of the path. When real-time data indicates changes in the environment, including new obstacles and traffic congestion, the algorithm will recalculate the best path.

[0106] Step 5: Path optimization: Based on the planned path, further optimize the path. The optimization process includes smoothing the path, reducing the number of turns, and avoiding congested areas. The optimized path should improve transportation efficiency.

[0107] Step 6: Output the best path: When the path planning and optimization are completed, the path is output. This path will be used to guide the transportation of containers and obtain the route from the starting point to the end point.

[0108] Step 7: Real-time monitoring: After the path planning is completed, continue to monitor the transportation status and environmental changes of the container. When unexpected situations occur, including traffic jams or equipment failures, the D* algorithm replans the path to maintain the flexibility of the transportation process.

[0109] The present invention has the following advantages: By using YOLO-World, the present invention shows high accuracy in target detection in container transportation scenarios, especially when dealing with complex scenarios and targets of different sizes. In addition, YOLO-World provides real-time processing capabilities to ensure high efficiency in dynamic environments. After combining with Transformer, it is also more adaptable in predicting the future movement path of the container. The multi-head self-attention mechanism of Transformer can effectively capture important patterns in time series and help predict the movement trajectory of the container. Through Kalman filtering and RAFT optical flow algorithm, the system can stably track the position and movement of the container, provide accurate speed and direction information, and provide reliable data for subsequent path prediction. The D* algorithm is used in path planning and optimization, which can dynamically adjust the container's transportation path. This method can re-plan the optimal path when encountering environmental changes or obstacles, ensuring the flexibility and efficiency of the container transportation process. By combining YOLO-World, Transformer and D* algorithms, the prediction accuracy and planning efficiency of the container transportation path are significantly improved, and it is suitable for complex logistics environments such as ports and freight stations. Overall, the scheme shows unique advantages in stably and efficiently predicting and planning container transportation paths. BRIEF DESCRIPTION OF THE DRAWINGS

[0110] Figure 1 It is a flowchart of the present invention. DETAILED DESCRIPTION

[0111] The present invention provides a container intelligent transportation path prediction and planning method based on YOLO-World and Transformer, and the method comprises the following steps:

[0112] Step 1: Optimize camera positions: Measure the best coverage of cameras through triangulation methods and verify the layout to obtain clear images of container transportation scenes;

[0113] Step 2, data acquisition and preprocessing: Use the optimized camera to collect data and perform preprocessing steps such as denoising and brightness adjustment;

[0114] Step 3: Real-time detection: Use YOLO-World to detect containers in real time;

[0115] Step 4: Tracking: Use the Kalman filter to stably track the position and trajectory of the container, so that the container can be continuously tracked in a complex environment;

[0116] Step 5: Calculate speed and direction: Calculate the speed and direction of movement of the container using the RAFT optical flow algorithm to provide a basis for subsequent path prediction;

[0117] Step 6: Path prediction: Use Transformer to predict the future movement path of the container and provide accurate path prediction based on its historical data and current status;

[0118] Step 7: Path planning and optimization: Based on the D* method, combined with prediction results and real-time data, the best transportation path is provided for the container, and the plan is dynamically adjusted to adapt to the changing environment.

[0119] In this embodiment, the specific steps of step one are as follows:

[0120] Step 1: Scenario analysis: In the container transportation scenario, analyze the site layout, mark key areas, including container storage areas, loading and unloading areas, and traffic routes, and identify potential obstructions, including buildings and vehicles;

[0121] Step 2: Triangulation: According to the results of scene analysis, select the preliminary camera position, consider the distance and relative angle between cameras, use triangulation method to calculate the ideal spacing and angle between cameras, cover the entire transportation area, calculate the camera layout, first calculate the distance between the two cameras, through the Euclidean distance formula, when the coordinates of the camera are (x1, y1, z1) and (x2, y2, z2), then the distance can be calculated as:

[0122]

[0123] Triangulation calculation, when the distance d between the two cameras and the angles θ1 and θ2 are known, the included angle is calculated using the inverse tangent function. The calculation formula is:

[0124]

[0125] Step 3: Camera installation: According to the results of triangulation, install the camera, adjust the camera angle, cover the entire container transportation scene, calculate the camera's viewing angle, and use the camera's focal length and the size of the photosensitive element. The focal length of the camera is f, and the width of the photosensitive element is w. The viewing angle can be calculated as follows:

[0126]

[0127] Step 4: Camera calibration and verification: After installing the camera, calibrate it, verify the camera coverage by testing the video stream, and check the overlapping area between cameras. The camera coverage area is calculated based on the viewing angle and distance. Based on the camera height h and viewing angle α, the camera coverage calculation formula is:

[0128]

[0129] In this embodiment, the specific steps of step 2 are as follows:

[0130] Step 1: Data collection preparation: Check the camera installation and calibration status;

[0131] Step 2: Data collection: Start the data collection system, collect image data according to the designed frequency and resolution, and check the camera status regularly during the collection process;

[0132] Step 3: Preliminary preprocessing: Use a denoising filter to remove random noise from the image;

[0133] Step 4. Brightness adjustment: Check the brightness and contrast of the image and adjust the level;

[0134] Step 5: Image enhancement: Perform sharpening processing as needed to improve the image clarity and edge contrast;

[0135] Step 6. Data verification: After preprocessing, perform a quality check on the image data and use sample data for detection testing.

[0136] In this embodiment, the specific steps of step three are as follows:

[0137] Step 1: Model initialization: load the pre-trained YOLO-World model and check the model parameters;

[0138] Step 2: Input preprocessing: resize the preprocessed image data to meet the input requirements of the YOLO-World model, and adjust the image data to the specific input size W×H required by the YOLO-World model, 416×416 pixels;

[0139] Step 3: Real-time detection: pass the preprocessed image data to the YOLO-World model to perform real-time target detection, set the detection threshold T to 0.5, so that the model can accurately identify container targets, filter out unnecessary detection results, and record the detected container location and size;

[0140] Step 4: Result verification: Verify the test results by visualizing the test results and comparing them with the known container locations. If the test results are erroneous or unstable, adjust the model parameters and preprocessing methods, and re-test.

[0141] Step 5: Processing of detection results: Based on the location and size of the container detected by YOLO-World, the coordinates and size information of the target are extracted and passed to the subsequent tracking stage so that the location of the container can be accurately tracked.

[0142] In this embodiment, the specific steps of step 4 are as follows:

[0143] Step 1: Initialize the Kalman filter: Before tracking the container, first initialize the Kalman filter. This stage includes setting the initial state and covariance matrix, determining the state transition model and observation model, and defining the process noise and observation noise. The initial state is generally based on the detection results, including the container position and speed provided by YOLO-World.

[0144] Step 2: State prediction: At each time step, the Kalman filter predicts the next position and motion state of the container based on the state transition model. The prediction step uses the previous state and process noise estimate to generate a predicted state vector and covariance matrix. The goal of this step is to provide an estimate of the future position of the container. The state prediction uses the following formula:

[0145] x k+1 =Fx k +u k

[0146] Among them, x k is the current state vector, F is the state transfer matrix, u is the process noise,

[0147] Covariance forecasting uses the following formula:

[0148] P k+1 =FP k FT +Q

[0149] Among them, P k is the current covariance matrix, Q is the process noise covariance matrix;

[0150] Step 3: Observation update: When new observation data is available, the Kalman filter corrects the predicted state through observation update. This step includes calculating the Kalman gain and updating the state vector and covariance matrix according to the new observation data. The Kalman gain calculation formula is:

[0151] K=P k H T (HP k H T +R) -1

[0152] Where H is the observation model matrix, R is the observation noise covariance matrix,

[0153] The update status formula is:

[0154] x k+1 =x k +K(z-Hx k )

[0155] Among them, z is the observed data,

[0156] The covariance update formula is:

[0157] P k+1 =(KH)P k

[0158] Among them, P k is the updated covariance matrix, K is the Kalman gain, and H is the observation matrix;

[0159] Step 4: Tracking calibration: In the real-time tracking process, there will be problems such as camera jitter and light changes, which will lead to unstable observation data. In order to maintain stable tracking, the Kalman filter needs to be calibrated to reduce errors caused by external factors, which involves adjusting process noise and observation noise, as well as introducing environmental adaptability constraints and prediction consistency constraints in state prediction, so that the Kalman filter can maintain high tracking accuracy and stability in a dynamic and changing environment;

[0160] Step 5: Trajectory output: The final result of the Kalman filter is a stable container trajectory output. This trajectory information, including position, speed, and direction, is used for subsequent path prediction and planning.

[0161] In this embodiment, the specific steps of step five are as follows:

[0162] Step 1: Prepare image sequences: Prepare continuous sequences of image frames from real-time cameras, fixed surveillance cameras, container transport vehicle cameras, and portable cameras of field workers so that the image sequences have sufficient temporal resolution. This stage involves adjusting the image size to meet the input requirements of the RAFT algorithm.

[0163] Step 2: Optical flow calculation: pass the image sequence to the RAFT optical flow algorithm to calculate the motion vector of each pixel. The result of the optical flow calculation is a vector field of the same size as the input image, indicating the moving direction and distance of each pixel between the previous and next frames.

[0164] Step 3: Extract the speed and direction of the container: After the optical flow calculation is completed, extract the speed and direction information related to the container. Through the previous Kalman filter tracking results, determine the area of ​​the container in the optical flow vector field. Then, by calculating the average vector in this area, the speed and direction of the container are obtained. The specific calculation of speed and direction is as follows:

[0165] Optical flow field:

[0166] u(x, y) = u x (x,y),u y (x, y)

[0167] Among them, u x and u y represent the movement along the x-axis and y-axis respectively,

[0168] Optical flow equation:

[0169]

[0170] Where I is the image intensity, and is the gradient of the image on the x-axis and y-axis, is the time gradient,

[0171] speed:

[0172]

[0173] direction:

[0174]

[0175] Overall container speed:

[0176]

[0177]

[0178] Where n is the number of pixels in the container area;

[0179] Step 4: Data smoothing: To improve the stability and reliability of speed and direction data, the Kalman filter will continue to be used for data smoothing to reduce the impact of noise and outliers.

[0180] Step 5: Result verification: After calculating the speed and direction, the results need to be verified by comparing the calculated results with actual observations and testing the accuracy of the optical flow calculation through simulation;

[0181] Step 6: Pass to path prediction: The final speed and direction data will be passed to the subsequent path prediction stage so that the data format and accuracy meet the requirements of path prediction and accurately predict the future movement trajectory of the container.

[0182] In this embodiment, the specific steps of step six are as follows:

[0183] Step 1: Data preparation: Obtain historical data of the container from the previous stage, including location, speed, and direction, and convert this data into a format suitable for Transformer input. When the data needs to be converted into a time series format, each data point represents a time step.

[0184] Step 2: Model initialization: Initialize the Transformer model, set model parameters, including the number of layers, heads, and hidden units, so that the model configuration matches the path prediction task, load the pre-trained model, and retrain the model;

[0185] Step 3: Multi-head attention: Use the multi-head attention mechanism to capture important patterns and relationships in the data. Multi-head attention determines which data points are important in the sequence by calculating attention weights.

[0186] Step 4: Position encoding: In order to make the model understand the temporal relationship in the time series data, position encoding is applied. Position encoding adds unique time information to each time step through sine and cosine functions, allowing Transformer to recognize the temporal relationship of the data;

[0187] Step 5: Training and reasoning: Pass the data to the Transformer model for model training and reasoning. During the training process, the loss function is used to measure the accuracy of the prediction results. For path prediction, the goal of the model is to minimize the error between the predicted path and the actual path. The loss function is calculated as follows:

[0188] Mean Squared Error (MSE):

[0189]

[0190] Among them, y i is the actual path data, is the predicted path data, and n is the number of data points.

[0191] Mean Absolute Error (MAE):

[0192]

[0193] Cross Entropy Loss:

[0194]

[0195] Among them, y i is the actual category, is the predicted probability.

[0196] Synthetic loss:

[0197] TotalLoss = α·MSE + β·MAE

[0198] Among them, α and β are weights that adjust the contribution of different loss functions;

[0199] Step 6: Result output: When the model training or inference is completed, the predicted container path is output. The prediction result is the location information of several future time steps, indicating the movement trajectory of the container in the future. The prediction result is used for subsequent path planning and optimization;

[0200] Step 7. Result verification: In order to ensure the accuracy of the prediction, it is necessary to verify the prediction results by comparing them with the actual data and testing the accuracy of the model through simulation.

[0201] In this embodiment, the specific steps of step seven are as follows:

[0202] Step 1: Initialize path planning: According to the predicted container movement path and the current environment status, initialize the path planning, including the starting point, end point and passable area of ​​the container, and consider the obstacles and traffic conditions in the environment;

[0203] Step 2: Set planning parameters: Set the parameters of the D* algorithm, including the heuristic estimation function and the search depth. The heuristic estimation is usually based on the straight-line distance of the path and other metrics to guide the algorithm to find the path.

[0204] Step 3: Path planning: Use the D* algorithm to plan the path. The algorithm calculates the shortest path from the starting point to the end point based on heuristic estimation and the location of known obstacles. The planning result is a sequence of path nodes. The path cost is the cost from the starting point to the previous node. The specific calculation is as follows:

[0205] Path cost:

[0206] g(x)=g(xprev)+cost(xprev,x)

[0207] Among them, g(x prev ) is the path cost to the previous node, cost(x prev , x) is the cost from the previous node to the current node.

[0208] Total path cost:

[0209] f(x)=g(x)+h(x)

[0210] Where g(x) is the path cost and h(x) is the heuristic estimate;

[0211] Step 4: Dynamic Adjustment: The key feature of the D* algorithm is the dynamic adjustment of the path. When real-time data indicates changes in the environment, including new obstacles and traffic congestion, the algorithm will recalculate the best path.

[0212] Step 5: Path optimization: Based on the planned path, further optimize the path. The optimization process includes smoothing the path, reducing the number of turns, and avoiding congested areas. The optimized path should improve transportation efficiency.

[0213] Step 6: Output the best path: When the path planning and optimization are completed, the path is output. This path will be used to guide the transportation of containers and obtain the route from the starting point to the end point.

[0214] Step 7: Real-time monitoring: After the path planning is completed, continue to monitor the transportation status and environmental changes of the container. When unexpected situations occur, including traffic jams or equipment failures, the D* algorithm replans the path to maintain the flexibility of the transportation process.

[0215] In this example, camera position optimization is integrated into data collection and preprocessing: After triangulation is completed, the best coverage of the camera layout is ensured and the layout is verified. Next, data collection starts and the cameras collect raw data. During the data collection phase, preprocessing work begins. This includes denoising, brightness adjustment, and image enhancement to ensure that subsequent detection and analysis are based on high-quality data.

[0216] In this example, data collection and preprocessing to real-time detection: The preprocessed image data will be passed to the detection stage. Use the YOLO-World model to perform object detection, such as detecting the location and size of the container. If the model needs to be retrained or fine-tuned, ensure that the preprocessed data meets the input requirements of the detection model.

[0217] In this embodiment, real-time detection to tracking: After the target container is detected, real-time tracking needs to start. The Kalman filter provides the initial state through the detection results and continues to track the movement of the target. The tracking stage needs to ensure the consistency of detection to avoid tracking interruptions caused by camera shaking or light changes. The prediction mechanism of the Kalman filter helps to maintain stable tracking.

[0218] In this example, tracking to speed and direction calculation: After tracking the position and movement of the container, the data is passed to the speed and direction calculation stage. The RAFT optical flow algorithm is used to analyze the speed and direction of the container's movement. RAFT uses the optical flow information in the image sequence to ensure that accurate speed and direction are obtained from the tracking data. This information will be used for path prediction.

[0219] In this example, speed and direction are calculated to path prediction: Once the speed and direction are calculated, the data is passed to the path prediction stage. The Transformer model is used to predict the future movement path of the container. The Transformer uses the tracked time series data and speed information to predict the future movement trajectory. Ensure that the model is fully trained and takes into account changes in the external environment.

[0220] In this embodiment, path prediction to path planning and optimization: After predicting the future path of the container, the D* algorithm is used for path planning and optimization. D* plans the best route based on the path prediction results and real-time data, taking into account obstacles and other changes in the dynamic environment.

[0221] Although the specific embodiments of the present invention are described in detail in conjunction with the accompanying drawings, it should not be understood as limiting the scope of protection of the present invention. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still belong to the scope of protection of the present invention.

Claims

1. A container intelligent transportation path prediction and planning method based on YOLO-World and Transformer, characterized by: The method comprises the following steps: Step 1: Optimize camera positions: Measure the best coverage of cameras through triangulation methods and verify the layout to obtain clear images of container transportation scenes; Step 2, data acquisition and preprocessing: Use the optimized camera to collect data and perform preprocessing steps such as denoising and brightness adjustment; Step 3: Real-time detection: Use YOLO-World to detect containers in real time; Step 4: Tracking: Use the Kalman filter to stably track the position and trajectory of the container, so that the container can be continuously tracked in a complex environment; Step 5: Calculate speed and direction: Calculate the speed and direction of movement of the container using the RAFT optical flow algorithm to provide a basis for subsequent path prediction; Step 6: Path prediction: Use Transformer to predict the future movement path of the container and provide accurate path prediction based on its historical data and current status; Step 7: Path planning and optimization: Based on the D* method, combined with prediction results and real-time data, the best transportation path is provided for the container, and the plan is dynamically adjusted to adapt to the changing environment.

2. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer as claimed in claim 1, characterized in that: The specific steps of step one are as follows: Step 1: Scenario analysis: In the container transportation scenario, analyze the site layout, mark key areas, including container storage areas, loading and unloading areas, and traffic routes, and identify potential obstructions, including buildings and vehicles; Step 2: Triangulation: Based on the results of the scene analysis, select the preliminary camera position, consider the distance and relative angle between the cameras, use the triangulation method to calculate the ideal spacing and angle between the cameras, cover the entire transportation area, and calculate the camera layout. First, calculate the distance between the two cameras. Using the Euclidean distance formula, when the coordinates of the cameras are (x1, y1, z1) and (x2, y2, z2), the distance can be calculated as: Triangulation calculation, when the distance d between the two cameras and the angles θ1 and θ2 are known, the included angle is calculated using the inverse tangent function. The calculation formula is: Step 3: Camera installation: According to the results of triangulation, install the camera, adjust the camera angle, cover the entire container transportation scene, calculate the camera's viewing angle, and use the camera's focal length and the size of the photosensitive element. The focal length of the camera is f, and the width of the photosensitive element is w. The viewing angle can be calculated as follows: Step 4: Camera calibration and verification: After installing the camera, calibrate it, verify the camera coverage by testing the video stream, and check the overlapping area between cameras. The camera coverage area is calculated based on the viewing angle and distance. Based on the camera height h and viewing angle α, the camera coverage calculation formula is:

3. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer according to claim 1 is characterized by: The specific steps of step 2 are as follows: Step 1: Data collection preparation: Check the camera installation and calibration status; Step 2: Data collection: Start the data collection system, collect image data according to the designed frequency and resolution, and check the camera status regularly during the collection process; Step 3: Preliminary preprocessing: Use a denoising filter to remove random noise from the image; Step 4. Brightness adjustment: Check the brightness and contrast of the image and adjust the level; Step 5: Image enhancement: Perform sharpening processing as needed to improve the image clarity and edge contrast; Step 6. Data verification: After preprocessing, perform a quality check on the image data and use sample data for detection testing.

4. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer as claimed in claim 1, characterized in that: The specific steps of step three are as follows: Step 1: Model initialization: load the pre-trained YOLO-World model and check the model parameters; Step 2: Input preprocessing: resize the preprocessed image data to meet the input requirements of the YOLO-World model, and adjust the image data to the specific input size W×H required by the YOLO-World model, 416×416 pixels; Step 3: Real-time detection: pass the preprocessed image data to the YOLO-World model to perform real-time target detection, set the detection threshold T to 0.5, so that the model can accurately identify container targets, filter out unnecessary detection results, and record the detected container location and size; Step 4: Result verification: Verify the test results by visualizing the test results and comparing them with the known container locations. If the test results are erroneous or unstable, adjust the model parameters and preprocessing methods, and re-test. Step 5: Processing of detection results: Based on the location and size of the container detected by YOLO-World, the coordinates and size information of the target are extracted and passed to the subsequent tracking stage so that the location of the container can be accurately tracked.

5. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer as claimed in claim 1, characterized in that: The specific steps of step 4 are as follows: Step 1: Initialize the Kalman filter: Before tracking the container, first initialize the Kalman filter. This stage includes setting the initial state and covariance matrix, determining the state transition model and observation model, and defining the process noise and observation noise. The initial state is generally based on the detection results, including the container position and speed provided by YOLO-World. Step 2: State prediction: At each time step, the Kalman filter predicts the next position and motion state of the container based on the state transition model. The prediction step uses the previous state and process noise estimate to generate a predicted state vector and covariance matrix. The goal of this step is to provide an estimate of the future position of the container. The state prediction uses the following formula: x k+1 =Fx k +u k Among them, x k is the current state vector, F is the state transfer matrix, u is the process noise, Covariance forecasting uses the following formula: P k+1 =FP k F T +Q Among them, P k is the current covariance matrix, Q is the process noise covariance matrix; Step 3: Observation update: When new observation data is available, the Kalman filter corrects the predicted state through observation update. This step includes calculating the Kalman gain and updating the state vector and covariance matrix according to the new observation data. The Kalman gain calculation formula is: K=P k H T (HP k H T +R) -1 Where H is the observation model matrix, R is the observation noise covariance matrix, The update status formula is: x k+1 =x k +K(z-Hx k ) Among them, z is the observed data, The covariance update formula is: P k+1 =(KH)P k Among them, P k is the updated covariance matrix, K is the Kalman gain, and H is the observation matrix; Step 4: Tracking calibration: In the real-time tracking process, there will be problems such as camera jitter and light changes, which will lead to unstable observation data. In order to maintain stable tracking, the Kalman filter needs to be calibrated to reduce errors caused by external factors, which involves adjusting process noise and observation noise, as well as introducing environmental adaptability constraints and prediction consistency constraints in state prediction, so that the Kalman filter can maintain high tracking accuracy and stability in a dynamic and changing environment. Step 5: Trajectory output: The final result of the Kalman filter is a stable container trajectory output. This trajectory information, including position, speed, and direction, is used for subsequent path prediction and planning.

6. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer as claimed in claim 1, characterized in that: The specific steps of step five are as follows: Step 1: Prepare image sequences: Prepare continuous sequences of image frames from real-time cameras, fixed surveillance cameras, container transport vehicle cameras, and portable cameras of field workers so that the image sequences have sufficient temporal resolution. This stage involves adjusting the image size to meet the input requirements of the RAFT algorithm. Step 2: Optical flow calculation: pass the image sequence to the RAFT optical flow algorithm to calculate the motion vector of each pixel. The result of the optical flow calculation is a vector field of the same size as the input image, indicating the moving direction and distance of each pixel between the previous and next frames. Step 3: Extract the speed and direction of the container: After the optical flow calculation is completed, extract the speed and direction information related to the container. Through the previous Kalman filter tracking results, determine the area of ​​the container in the optical flow vector field. Then, by calculating the average vector in this area, the speed and direction of the container are obtained. The specific calculation of speed and direction is as follows: Optical flow field: u(x,y)=u x (x,y),u y (x,y) Among them, u x and u y represent the movement along the x-axis and y-axis respectively, Optical flow equation: Where I is the image intensity, and is the gradient of the image on the x-axis and y-axis, is the time gradient, speed: direction: Overall container speed: Where n is the number of pixels in the container area; Step 4: Data smoothing: To improve the stability and reliability of speed and direction data, the Kalman filter will continue to be used for data smoothing to reduce the impact of noise and outliers. Step 5: Result verification: After calculating the speed and direction, the results need to be verified by comparing the calculated results with actual observations and testing the accuracy of the optical flow calculation through simulation; Step 6: Pass to path prediction: The final speed and direction data will be passed to the subsequent path prediction stage so that the data format and accuracy meet the requirements of path prediction and accurately predict the future movement trajectory of the container.

7. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer as claimed in claim 1, characterized in that: The specific steps of step six are as follows: Step 1: Data preparation: Obtain historical data of the container from the previous stage, including location, speed, and direction, and convert this data into a format suitable for Transformer input. When the data needs to be converted into a time series format, each data point represents a time step. Step 2: Model initialization: Initialize the Transformer model, set model parameters, including the number of layers, heads, and hidden units, so that the model configuration matches the path prediction task, load the pre-trained model, and retrain the model; Step 3: Multi-head attention: Use the multi-head attention mechanism to capture important patterns and relationships in the data. Multi-head attention determines which data points are important in the sequence by calculating attention weights. Step 4: Position encoding: In order to make the model understand the temporal relationship in the time series data, position encoding is applied. Position encoding adds unique time information to each time step through sine and cosine functions, allowing Transformer to recognize the temporal relationship of the data; Step 5: Training and reasoning: Pass the data to the Transformer model for model training and reasoning. During the training process, the loss function is used to measure the accuracy of the prediction results. For path prediction, the goal of the model is to minimize the error between the predicted path and the actual path. The loss function is calculated as follows: Mean Squared Error (MSE): Among them, y i is the actual path data, is the predicted path data, and n is the number of data points. Mean Absolute Error (MAE): Cross Entropy Loss: Among them, y i is the actual category, is the predicted probability. Synthetic loss: TotalLoss = α·MSE + β·MAE Among them, α and β are weights that adjust the contribution of different loss functions; Step 6: Result output: When the model training or inference is completed, the predicted container path is output. The prediction result is the location information of several future time steps, indicating the movement trajectory of the container in the future. The prediction result is used for subsequent path planning and optimization; Step 7. Result verification: In order to ensure the accuracy of the prediction, it is necessary to verify the prediction results by comparing them with the actual data and testing the accuracy of the model through simulation.

8. The container intelligent transportation path prediction and planning method based on YOLO-World and Transformer as claimed in claim 1, characterized in that: The specific steps of step seven are as follows: Step 1: Initialize path planning: According to the predicted container movement path and the current environment status, initialize the path planning, including the starting point, end point and passable area of ​​the container, and consider the obstacles and traffic conditions in the environment; Step 2: Set planning parameters: Set the parameters of the D* algorithm, including the heuristic estimation function and the search depth. The heuristic estimation is usually based on the straight-line distance of the path and other metrics to guide the algorithm to find the path. Step 3: Path planning: Use the D* algorithm to plan the path. The algorithm calculates the shortest path from the starting point to the end point based on heuristic estimation and the location of known obstacles. The planning result is a sequence of path nodes. The path cost is the cost from the starting point to the previous node. The specific calculation is as follows: Path cost: g(x)=g(x prev )+cost(x prev ,x) Among them, g(x prev ) is the path cost to the previous node, cost(x prev ,x) is the cost from the previous node to the current node. Total path cost: f(x)=g(x)+h(x) Where g(x) is the path cost and h(x) is the heuristic estimate; Step 4: Dynamic Adjustment: The key feature of the D* algorithm is the dynamic adjustment of the path. When real-time data indicates changes in the environment, including new obstacles and traffic congestion, the algorithm will recalculate the best path. Step 5: Path optimization: Based on the planned path, further optimize the path. The optimization process includes smoothing the path, reducing the number of turns, and avoiding congested areas. The optimized path should improve transportation efficiency. Step 6: Output the best path: When the path planning and optimization are completed, the path is output. This path will be used to guide the transportation of containers and obtain the route from the starting point to the end point. Step 7: Real-time monitoring: After the path planning is completed, continue to monitor the transportation status and environmental changes of the container. When unexpected situations occur, including traffic jams or equipment failures, the D* algorithm replans the path to maintain the flexibility of the transportation process.

Images