Visual-based automatic container depositing method and system for container yard

Abstract

The present application belongs to the technical field of visual detection, and discloses a kind of yard container automatic box method and system based on vision.The present application is respectively installed on the four corners of spreader image acquisition device, and the image containing container under spreader and target container is obtained by real-time acquisition using image acquisition device;The bottom corner point of container under spreader and the top corner point of target container are identified and labeled using key point detection model, and detection information is obtained;Wherein, the key point detection model contains boundary perception attention module;Finally, according to detection information and spreader information, yard container automatic box is placed.This application can improve the positioning accuracy of yard container automatic box, and reduce the cost.

Description

Technical Field

[0001] This invention belongs to the field of visual inspection technology, and more specifically, relates to a vision-based automatic container placement method and system in a container yard. Background Technology

[0002] In traditional port operations, gantry crane operators need to manually control the spreader, relying entirely on visual observation to align the container openings with the truck's locking pins. This method is not only time-consuming and inefficient, but also prone to safety accidents due to fatigue or misoperation. With the growth of global trade and the continuous increase in port cargo throughput, the traditional manual operation mode can no longer meet the demands for efficient and rapid loading and unloading.

[0003] To achieve automated container placement, researchers in this field have proposed several solutions. However, current automated container placement by yard cranes mainly relies on laser scanning installed on the trolley frame or spreader. These solutions suffer from high hardware costs, poor positioning accuracy, and numerous other problems. For example, when positioning is based on identifying and detecting lock holes, the lock holes are obstructed when the container under the spreader approaches the target container, making it impossible to provide positional offset. Other issues include high computational complexity, slow inference speed, and poor recognition and positioning performance of existing algorithms due to the distance and blurriness of the target in the container placement scenario. Summary of the Invention

[0004] This invention provides a vision-based automatic container placement method and system for yards, which solves the problems of poor positioning accuracy and high cost in the existing automatic container placement technology.

[0005] This invention provides a vision-based automated container placement method for container yards, comprising the following steps:

[0006] Image acquisition devices are installed at the four corners of the spreader to capture images of the container below the spreader and the target container in real time.

[0007] The key point detection model is used to identify and label the bottom corner of the container under the spreader and the top corner of the target container to obtain detection information; the key point detection model includes a boundary awareness attention module.

[0008] The container yard is automatically placed based on the detection information and the lifting equipment information.

[0009] Preferably, after the boundary-aware attention module is applied to each C2f module of the backbone, the model's attention to the boundary region is enhanced by the boundary-aware attention module.

[0010] The boundary awareness attention module performs multi-directional edge perception and integrates multi-directional edge detection information into the attention weights. The boundary awareness attention module combines spatial attention and channel attention and adds an edge guidance mechanism. It uses spatial attention to enhance the response of edge regions and uses channel attention to filter boundary-related feature channels.

[0011] Preferably, after the original feature map is input into the boundary-aware attention module, the original feature map first goes through multiple parallel convolutional layers, with different convolutional layers using edge detection kernels in different directions. Then, these edge feature maps in different directions are added together and fused through a convolutional layer to generate an edge saliency map.

[0012] Spatial attention is applied to the edge saliency map to generate a spatial weight matrix; channel attention is applied to the edge saliency map to generate channel weights.

[0013] The original feature map is multiplied by the spatial weight matrix and the channel weights to obtain the enhanced feature map;

[0014] The original feature map and the enhanced feature map are added together or concatenated along the channel dimension, and then subjected to a 1x1 convolution to obtain the final output feature map of the boundary-aware attention module.

[0015] Preferably, the boundary-aware attention module adopts a lightweight design and uses depthwise separable convolution for channel compression.

[0016] Preferably, the spreader information includes some or all of the following information: the position of the spreader trolley, the position of the spreader carriage, the lifting height of the spreader, the lifting speed of the spreader, the height of the target container, the open / closed status of the spreader, and the dimensions of the container; when automatically placing containers in the yard, the positions of the trolley and / or carriage, as well as the rotation angle and height of the spreader are adjusted.

[0017] Preferably, before using the key point detection model for identification and annotation, the method further includes: determining whether the trigger detection condition is met based on the lifting equipment information; if it is met, the detection process is triggered.

[0018] Preferably, after the image is acquired in real time but before the detection information is obtained, the method further includes: preprocessing the image; the preprocessing includes selecting the ROI region and performing noise reduction and geometric correction on the image.

[0019] Preferably, after obtaining the detection information using the key point detection model, the method further includes: smoothing the detection information using an extended Kalman filter.

[0020] Preferably, the offset of a single image acquisition device is calculated based on the detection information and the spreader information, the offset calculation results of multiple image acquisition devices are fused to obtain offset information, and the container in the yard is automatically placed in combination with the offset information.

[0021] On the other hand, the present invention provides a vision-based automated container placement system for container yards, comprising:

[0022] An image acquisition device is used to acquire images in real time, including the container below the spreader and the target container.

[0023] The detection unit is used to identify and mark the bottom corners of the container under the spreader and the top corners of the target container to obtain detection information;

[0024] The control unit is used to automatically place containers in the yard based on detection information and spreader information;

[0025] The vision-based automated container placement system for yards is used to perform the steps in the vision-based automated container placement method for yards described above.

[0026] One or more technical solutions provided in this invention have at least the following technical effects or advantages:

[0027] (1) This invention installs image acquisition devices at the four corners of the spreader to acquire images of the container below the spreader and the target container in real time. A key point detection model is used to identify and label the bottom corner of the container below the spreader and the top corner of the target container to obtain detection information. The key point detection model includes a boundary-aware attention module. Finally, the container is automatically placed in the yard based on the detection information and the spreader information. This invention targets multi-layered container stacking scenarios. Considering the strong features of container corners, a key point detection method is adopted, using point labeling (directly labeling the bottom corner of the container below the spreader and the top corner of the target container). Compared to existing identification and labeling schemes, this method has advantages such as faster inference speed, no need for complex post-processing logic, simple labeling, and low labeling cost. The identification target of this invention is the corner inflection point of the container; therefore, the edge or boundary information of the object is crucial for accurate key point localization. Considering this, this invention proposes a boundary-aware attention module (BAAM), which simultaneously considers edge-guided spatial attention and channel attention, effectively improving the model's attention to boundary features and enhancing the accuracy of key point localization. This invention, by identifying the target container and the corners of the container under the spreader, provides stable data on trolley offset, gantry offset, and rotation angle even when the lock hole is obscured, compared to existing methods. This results in a higher container placement success rate, exceeding 98% in actual measurements, and high alignment accuracy (less than 3cm in actual measurements), significantly improving operational safety and reliability. Furthermore, the port operating environment is complex and frequently affected by weather and lighting conditions. The vision-based automated container placement solution proposed in this invention can operate normally in adverse environments such as rain, fog, snow, and nighttime, ensuring efficient, year-round port operations.

[0028] (2) The boundary awareness attention module in this invention is designed for convolutional neural networks and can be further lightweighted to suit scenarios with real-time requirements. It is very suitable for the application scenario of automatic container placement in yards.

[0029] (3) The present invention utilizes extended Kalman filtering to make the detected key points smoother, reduce the impact of fluctuations in detection results, and obtain more stable data. Combined with image preprocessing and data fusion of multiple image acquisition devices, the overall scheme can further improve the accuracy of monitoring, thereby achieving more precise automatic box placement. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating the algorithm processing of a vision-based automated container placement method for container yards, as provided in Embodiment 1 of the present invention.

[0031] Figure 2 The images are obtained in real time by four image acquisition devices in a vision-based automatic container placement method for yards provided in Embodiment 1 of the present invention.

[0032] Figure 3 This is a labeling example of a vision-based automated container placement method for container yards provided in Embodiment 1 of the present invention;

[0033] Figure 4 This is a schematic diagram of the boundary perception attention module in a vision-based automated container placement method for yards provided in Embodiment 1 of the present invention.

[0034] Figure 5 This is a schematic diagram illustrating the calculation of rotation angles in a vision-based automated container placement method for yards provided in Embodiment 1 of the present invention. Detailed Implementation

[0035] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0036] Example 1:

[0037] Example 1 provides a vision-based automated container placement method for container yards, which mainly includes the following steps:

[0038] Image acquisition devices are installed at the four corners of the spreader to capture images of the container below the spreader and the target container in real time.

[0039] The key point detection model is used to identify and label the bottom corner of the container under the spreader and the top corner of the target container to obtain detection information; the key point detection model includes a boundary awareness attention module.

[0040] The container yard is automatically placed based on the detection information and the lifting equipment information.

[0041] Specifically, the boundary-aware attention module is applied after each C2f module of the backbone to enhance the model's attention to boundary regions. The boundary-aware attention module performs multi-directional edge perception, fusing multi-directional edge detection information into the attention weights. It combines spatial attention and channel attention, incorporating an edge-guiding mechanism. Spatial attention strengthens the response to edge regions, while channel attention filters boundary-related feature channels.

[0042] Specifically, after the original feature map is input into the boundary-aware attention module, it first passes through multiple parallel convolutional layers, each using edge detection kernels in different directions. These edge feature maps from different directions are then added together and fused through a convolutional layer to generate an edge saliency map. Spatial attention is applied to the edge saliency map to generate a spatial weight matrix. Channel attention is then applied to the edge saliency map to generate channel weights. The original feature map is multiplied by the spatial weight matrix and the channel weights to obtain an enhanced feature map. The original feature map and the enhanced feature map are then added together or concatenated along the channel dimension, and finally subjected to a 1x1 convolution to obtain the final output feature map of the boundary-aware attention module.

[0043] Furthermore, the boundary-aware attention module can be further optimized using a lightweight design and channel compression using depthwise separable convolution.

[0044] The spreader information includes some or all of the following: the position of the spreader trolley, the position of the spreader carriage, the lifting height of the spreader, the lifting speed of the spreader, the height of the target container, the open / closed status of the spreader, and the dimensions of the container; when automatically placing containers in the yard, the positions of the trolley and / or carriage, as well as the rotation angle and height of the spreader are adjusted.

[0045] The offset of a single image acquisition device is calculated based on the detection information and the spreader information. The offset calculation results of multiple image acquisition devices are fused to obtain offset information. The container is then automatically placed in the yard based on the offset information.

[0046] In a preferred embodiment, before using the keypoint detection model for identification and annotation, the method may further include: determining whether the trigger detection condition is met based on the lifting equipment information; if met, triggering the detection process. After real-time image acquisition but before obtaining detection information, the method may further include: preprocessing the image; the preprocessing includes selecting the ROI region and performing noise reduction and geometric correction on the image. After obtaining detection information using the keypoint detection model, the method may further include: smoothing the detection information using an extended Kalman filter.

[0047] The following example uses an image acquisition device that employs a camera and is controlled by a programmable logic controller (PLC) to illustrate a scheme that includes several preferred designs.

[0048] See Figure 1After acquiring images of the container below the spreader and the target container in real time using a camera, the algorithm processing flow of this invention mainly includes the following steps: reading PLC data and video frame data (containing several real-time images), determining whether the trigger detection condition is met, if met, preprocessing the image using the image preprocessing module, then performing key point detection and extended Kalman filtering, then calculating the offset using the offset calculation module, and fusing the multi-camera structure to determine whether the container placement confirmation condition is met, if not met, then fine-tuning the spreader rotation, trolley position, and trolley position, etc., if met, then completing the automatic container placement.

[0049] In this invention, image acquisition modules are installed at the four corners of the spreader to capture real-time images of the container below the spreader and the target container. The images acquired in real time by the four image acquisition devices are as follows: Figure 2 As shown. Simultaneously, the control unit reads information such as the position of the spreader trolley, the position of the main trolley, the spreader lifting height, the spreader lifting speed, the target container height, the spreader's open / closed status, and the container dimensions to determine trigger detection conditions, container landing confirmation conditions, and pixel-to-actual distance conversion calculations. When the height difference between the spreader and the target container is less than a given threshold (e.g., 5m), and the spreader is in a locked state, the detection process is triggered.

[0050] Specifically, the image preprocessing in this invention mainly includes the following steps:

[0051] 1. Select ROI: Preset ROI areas based on the corner positions of the target container and the corner positions of the container under the spreader, and then process the ROI areas from the original image.

[0052] 2. Image Denoising: Adaptive histogram equalization is used to improve image contrast, and median filtering is used to denoise the image.

[0053] 3. Distortion correction: Geometric image correction based on camera calibration parameters (intrinsic parameter matrix, distortion coefficients).

[0054] This invention employs key point detection. The annotation scheme used in this invention involves annotating the top corner of the target container and the bottom corner of the container below the spreader, such as... Figure 3 As shown. To improve the generalization ability of the keypoint detection model, this invention can also perform data augmentation processing. Specifically, it enriches the diversity of training data through data augmentation strategies such as random brightness, random saturation, random hue, random flipping, random rotation, random scaling, and random Gaussian blur, thereby improving the model's generalization ability.

[0055] To meet real-time requirements, this invention can select the YOLOv8 keypoint detection model for training and make improvements based on it. Of course, the improvements proposed in this invention are not limited to the YOLOv8 keypoint detection model and can also be extended to other models.

[0056] Since the target of the proposed solution is the corner of a container, the edge or boundary information of the object is crucial for accurate location of key points. Considering this, the present invention proposes a boundary-aware attention module and applies it after each C2f module of the backbone, thereby enhancing the model's attention to boundary regions.

[0057] Specifically, the boundary-aware attention module proposed in this invention has the following characteristics.

[0058] 1. Multi-directional edge sensing:

[0059] Apply edge detection convolution kernels, such as those for horizontal, vertical, and diagonal edge detection, to the feature map, and then fuse this edge information into the attention weights.

[0060] In order for the model to adaptively learn good edge features, the parameters of the convolution kernel are learnable and can be initialized using common edge detection operators such as Sobel.

[0061] To keep the module lightweight, depthwise separable convolutional kernel channel compression is used to maintain computational efficiency.

[0062] 2. Dual-path attention mechanism.

[0063] It combines spatial attention and channel attention, but incorporates an edge-oriented mechanism.

[0064] Among them, spatial attention enhances the response of edge regions, while channel attention filters boundary-related feature channels.

[0065] 3. Lightweight design.

[0066] Use depthwise separable convolutions (reducing computation to 1 / 8 to 1 / 9 of standard convolutions) and channel compression (reduction_ratio=4) to reduce fully connected layer parameters.

[0067] Specifically, the structure of the boundary-aware attention module proposed in this invention is shown in [reference needed]. Figure 4 Its working principle includes:

[0068] 1. The input feature map is processed through several parallel convolutional layers, each using edge detection kernels with different orientations (e.g., horizontal, vertical, 45-degree, 135-degree). The parameters of these kernels are learnable to adapt to the edge features. These edge feature maps with different orientations are then summed and fused through a single convolutional layer to generate an edge saliency map. To keep the module lightweight, depthwise separable convolutions are used to reduce the number of parameters.

[0069] 2. Apply spatial attention to the edge saliency map to generate a spatial weight matrix, enhancing the features of the edge regions. Simultaneously, apply channel attention (such as the SE module) to the edge saliency map to generate channel weights and filter the channel dimensions.

[0070] 3. Multiply the original feature map with the spatial weight matrix and channel weights to obtain the enhanced features.

[0071] 4. Add the original feature map and the enhanced feature map together or concatenate them along the channel dimension, and then perform a 1x1 convolution to obtain the final output feature of the BAAM module.

[0072] As can be seen from the above description, the boundary-aware attention module proposed in this invention considers both edge-guided spatial attention and channel attention, which can effectively enhance the model's attention to boundary features and improve the accuracy of key point localization.

[0073] The extended Kalman filter (EKF) and offset calculation used in this invention will be explained below.

[0074] Since the detection results for each frame are independent, and the model's single-frame detection results may exhibit noise fluctuations, this invention employs extended Kalman filtering to smooth out the detected keypoints in order to reduce the impact of these fluctuations.

[0075] 1. Definition of state variables.

[0076] State vector:

[0077]

[0078] Where (x, y) are the pixel coordinates of the keypoint. This represents the rate of change of pixel position.

[0079] 2. Nonlinear state transition model.

[0080] Kinematic model:

[0081]

[0082] Assume that the speed of the spreader is approximately constant over a short period of time, where w is the process noise and Δt is the time step.

[0083] State transition function f(x):

[0084]

[0085] Taking the partial derivative of the state transition function, we obtain the Jacobian matrix F, as follows:

[0086]

[0087] 3. Visual observation model.

[0088] Observation parameters:

[0089] The key point detection model detects the top corner of the target container and the bottom corner of the container under the spreader, and outputs the corner coordinates (x, y, y). obs ,y obs ).

[0090] Observation vector:

[0091] z = [x obs ,y obs ] T

[0092] Observation model:

[0093] h(x) = [x, y] Τ

[0094] Observational Jacobian matrix H:

[0095]

[0096] 4. EKF algorithm flow.

[0097] 1) Initialization:

[0098] Initial state: x0 = [0,0,0,0] T ;

[0099] Initial covariance matrix: P0 = diag([1,1,0.1,0.1]);

[0100] Process noise covariance: Q = diag([0.1, 0.1, 0.5, 0.5]);

[0101] Observation noise covariance: R = diag([0.5, 0.5]).

[0102] 2) Forecasting phase:

[0103] Calculate state prediction: x k|k-1 =f(x) k-1 );

[0104] Calculate covariance prediction: Pk|k-1 =FP k-1 F T +Q.

[0105] 3) Update phase:

[0106] Obtain visual observations: z k ;

[0107] Calculate the observation residual: y = z k -h(x k|k-1 );

[0108] Calculate the Kalman gain: K k =P k|k-1 H T HP k|k-1 H T +R) -1 ;

[0109] Update status: x k =x k|k-1 +K k y;

[0110] Update covariance: P k =(IK k H)P k|k-1 .

[0111] Specifically, the single-camera offset calculation in this invention includes:

[0112] (1) The conversion relationship between pixel distance and actual distance.

[0113] 1. Place a checkerboard calibration plate of known size on the ground below the hoist, ensuring it is within the camera's field of view.

[0114] 2. Gradually adjust the height of the spreader from the lowest to the highest, and take an image of the calibration plate at each height. After each height adjustment, record the actual distance h between the spreader and the calibration plate.

[0115] 3. Use image processing algorithms to detect the corners or center of the calibration board and obtain pixel coordinates. Based on the detected points, calculate the pixel distance d between two points on the calibration board whose actual distance is known.

[0116] 4. Calculate the scaling factor: Based on the actual distance d′ and the pixel distance d, calculate the scaling factor s = d′ / d.

[0117] 5. Fitting Relationship Model: The scaling factor at different heights is fitted to the lifting device height H to obtain the conversion relationship model between pixel distance and actual distance. This is usually a linear relationship, as shown below:

[0118] s x =ax H+b x

[0119] s y =a y H+b y

[0120] Among them, s x s is the scaling factor of the image along the x-axis. y This is the scaling factor of the image along the y-axis.

[0121] (2) The actual offset distances of the small car and the large car are calculated based on the pixel distance.

[0122] By subtracting the corner coordinates of the target container from the corner coordinates of the container below the spreader, the pixel offsets d in the direction of the trolley (corresponding to the x-axis direction of the image coordinate system) and the direction of the trailer (corresponding to the y-axis direction of the image coordinate system) are obtained. x and d y (That is, obtain the offset of two pixels in mutually perpendicular directions), and then substitute them into the transformation relationship model obtained above to obtain the actual offset distance of a single camera in the direction of the small car and the direction of the large car, as shown below:

[0123] d x ′=s x ·d x

[0124] d y ′=s y ·d y

[0125] (3) Calculate the rotation angle between the container below the spreader and the target container, with counterclockwise as positive.

[0126] Establish a coordinate system with the target container's location as a reference, such as Figure 5 As shown, L is the length of the container, and W is the width of the container. Using the formula above, the offsets of the four corners of the container below the spreader relative to the four corners of the target container can be calculated, and further, the coordinates of the container below the spreader in this coordinate system can be obtained. Then, the rotation angle is calculated by taking one edge of the target container and one edge of the container below the spreader. The calculation process is now illustrated using line segments AB and A′B′. In practice, there may be corners where the offset value cannot be provided due to obstruction; in this case, other edges need to be used for calculation.

[0127] a) Vector definition.

[0128] Line segment A: Assume the starting point is (0,0) and the ending point is (L,0), and its vector is

[0129] Line segment B: Its starting point is offset from the starting point of line segment A by (dx1, dy1), and its ending point is offset from the ending point of line segment A by (dx2, dy2). Its vector is:

[0130] b) Calculate the angle using the dot product.

[0131] Calculate the cosine value using the dot product formula:

[0132]

[0133] c) Determine the direction using the cross product.

[0134]

[0135] Counterclockwise (positive angle);

[0136] Clockwise (negative angle).

[0137] d) Obtain the radian value through the inverse cosine function and convert it into an angle.

[0138] Specifically, the principle of multi-camera data fusion in this invention is as follows: Different weights are assigned to each camera based on its confidence level or measurement accuracy, and a weighted average is then performed. Specifically, multi-camera data fusion mainly includes the following steps:

[0139] 1. Calculate the confidence level of each camera measurement (e.g., the confidence level of the detection box).

[0140] 2. Assign weights based on confidence level, with cameras having higher confidence levels receiving greater weights.

[0141] 3. Take a weighted average of the measurements from the four cameras to obtain the final distance.

[0142] The formula used is as follows:

[0143]

[0144] Among them, D final It is the final distance, D i w is the offset distance obtained from the i-th camera. i It is the weight corresponding to the i-th camera.

[0145] The present invention controls the spreader by: when the distance between the container below the spreader and the target container is less than a given threshold (e.g., 10cm), and the trolley deviates, the trolley deviates, and the rotation angle is less than the given threshold, the container placement operation is performed; otherwise, the spreader is rotated, the trolley moves, and the trolley moves, so that the container placement confirmation conditions are finally met.

[0146] In summary, the vision-based automated container placement method for container yards provided in Example 1 can achieve automated operations, improve placement success rate, alignment accuracy and stability, accelerate reasoning speed, and reduce costs.

[0147] Example 2:

[0148] Example 2 provides a vision-based automated container placement system for container yards, comprising:

[0149] An image acquisition device is used to acquire images in real time, including the container below the spreader and the target container.

[0150] The detection unit is used to identify and mark the bottom corners of the container under the spreader and the top corners of the target container to obtain detection information;

[0151] The control unit is used to automatically place containers in the yard based on detection information and spreader information;

[0152] The vision-based automated container placement system for yards is used to perform the steps in the vision-based automated container placement method for yards as described in Example 1.

[0153] The image acquisition device can be a high-definition camera, a camera, etc.

[0154] The detection unit includes a key point detection model, and the key point detection model includes a boundary-aware attention module.

[0155] The control unit may be a programmable logic controller.

[0156] Since the functions of each device or unit in the system provided in Embodiment 2 correspond to the steps in the method provided in Embodiment 1, they can be understood by referring to the description of Embodiment 1, and will not be repeated here.

[0157] Finally, it should be noted that the above specific embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A vision-based automated container placement method for container yards, characterized in that, Includes the following steps: Image acquisition devices are installed at the four corners of the spreader to capture images of the container below the spreader and the target container in real time. A key point detection model is used to identify and label the bottom corners of the container below the spreader and the top corners of the target container to obtain detection information. Extended Kalman filtering is used to smooth the detection information. The key point detection model includes a boundary-aware attention module. This boundary-aware attention module is applied after each C2f module of the backbone to enhance the model's attention to boundary regions. The boundary-aware attention module performs multi-directional edge perception, fusing multi-directional edge detection information into the attention weights. The boundary-aware attention module combines spatial attention and channel attention, and incorporates an edge-guided mechanism. It uses spatial attention to enhance the response of edge regions and channel attention to filter boundary-related feature channels. In this process, after the original feature map is input into the boundary-aware attention module, it first passes through multiple parallel convolutional layers, each using edge detection kernels in different directions. These edge feature maps from different directions are then added together and fused through a single convolutional layer to generate an edge saliency map. Spatial attention is applied to the edge saliency map to generate a spatial weight matrix. Channel attention is then applied to the edge saliency map to generate channel weights. The original feature map is multiplied by the spatial weight matrix and the channel weights to obtain an enhanced feature map. Finally, the original feature map and the enhanced feature map are added together or concatenated along the channel dimension, and then subjected to a 1x1 convolution to obtain the final output feature map of the boundary-aware attention module. The offset of a single image acquisition device is calculated based on the detection information and the lifting equipment information. The offset calculation results of the four image acquisition devices are fused to obtain offset information. The container is then automatically placed in the yard based on the offset information. The fusion process includes assigning different weights to each image acquisition device based on its confidence level or measurement accuracy, and then performing a weighted average.

2. The vision-based automated container placement method for container yards according to claim 1, characterized in that, The boundary-aware attention module adopts a lightweight design and uses depthwise separable convolution for channel compression.

3. The vision-based automated container placement method for container yards according to claim 1, characterized in that, The spreader information includes some or all of the following: the position of the spreader trolley, the position of the spreader carriage, the lifting height of the spreader, the lifting speed of the spreader, the height of the target container, the open / closed status of the spreader, and the dimensions of the container; when automatically placing containers in the yard, the positions of the trolley and / or carriage, as well as the rotation angle and height of the spreader are adjusted.

4. The vision-based automated container placement method for container yards according to claim 1, characterized in that, Before using the key point detection model for identification and annotation, the process also includes: determining whether the triggering detection conditions are met based on the lifting equipment information; if they are met, the detection process is triggered.

5. The vision-based automated container placement method for container yards according to claim 1, characterized in that, After the image is acquired in real time but before the detection information is obtained, the process also includes: image preprocessing; the preprocessing includes selecting the ROI region and performing noise reduction and geometric correction on the image.

6. A vision-based automated container placement system for container yards, characterized in that, include: An image acquisition device is used to acquire images in real time, including the container below the spreader and the target container. The detection unit is used to identify and mark the bottom corners of the container under the spreader and the top corners of the target container to obtain detection information; The control unit is used to automatically place containers in the yard based on detection information and spreader information; The vision-based automated container placement system for yards is used to perform the steps in the vision-based automated container placement method for yards as described in any one of claims 1-5.

Images