Container lifting spreader container detection method and device, electronic equipment and storage medium

Abstract

The application provides a container lifting lug container detection method and device, electronic equipment and storage medium, the method comprises the following steps: collecting current frame three-dimensional point cloud data covering the area below the lifting lug and above the trailer; screening the point cloud of the region of interest from the current frame three-dimensional point cloud data; dividing the point cloud of the region of interest into a plurality of point cloud cells along the height direction; determining the reference width of the container to be grabbed according to the point cloud in the at least one point cloud cell with the lowest height position; determining the width difference according to the difference between the width of the point cloud in the point cloud cell where the box top height is located and its adjacent upper point cloud cell in the horizontal direction and the reference width; determining the height difference representing the spacing between the lower edge of the lifting lug and the upper edge of the box top of the container to be grabbed according to the maximum height of the point cloud in the point cloud cell where the box top height is located and its adjacent upper at least one point cloud cell; and determining the container detection state according to the width difference and the height difference. The method improves the accuracy of container detection.

Description

Technical Field

[0001] This application relates to the field of automated loading and unloading technology, and in particular to a container spreader landing detection method, device, electronic equipment and storage medium. Background Technology

[0002] In container handling scenarios such as ports, railway ports, and freight yards, lifting and unloading equipment such as reach stackers, yard cranes, and quay cranes use spreaders to grab and move containers. A spreader is a specialized crossbeam suspended below the lifting wire ropes of the lifting and unloading equipment, with rotatable locking heads at its four corners. Container engagement refers to the state where, during the descent of the spreader, the locking heads or the lower edge of the spreader are stably and securely aligned with the top corner fittings of the container. Container engagement detection is a crucial trigger signal in the automated loading and unloading process; only after a stable engagement is confirmed can the lifting and unloading equipment perform subsequent actions such as locking and lifting. If a missed or false engagement occurs, it can lead to minor issues like operational interruptions and reduced efficiency, or more serious safety accidents such as the spreader locking while suspended in mid-air, the container falling, or damaging the trailer.

[0003] For box inspection, existing technologies mainly fall into the following categories: For enclosure detection based on contact or mechanical switches, specifically, limit switches or pressure sensors are installed on the locking head or lower edge of the lifting device. A signal is triggered when the locking head presses against the top of the enclosure. However, the mechanical components in the switches or sensors are easily damaged or prone to false triggers due to long-term exposure to dust, salt spray, and impact environments, resulting in high maintenance costs.

[0004] Image-based or deep learning-based container detection uses a vehicle-mounted camera to detect or classify targets at the interface between the spreader and the container. However, this method is significantly affected by environmental factors such as changes in lighting, backlighting, nighttime, rain, fog, and differences in container color. Furthermore, it requires a large amount of labeled data for training and has poor cross-scene generalization ability.

[0005] The landing detection method based on threshold comparison of single-frame laser 3D point clouds involves filtering the single-frame laser 3D point cloud and then directly comparing the Z-coordinate of a reference point on the lifting device with the Z-coordinate of the top of the box. However, for this method to be accurate, the lower edge of the lifting device and the top edge of the box must first be precisely separated in the single-frame laser 3D point cloud. But when noise, occlusion, and jitter exist in the single-frame 3D point cloud, it is difficult to reliably separate the two points, and directly comparing height thresholds can easily lead to misjudgments. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a container spreader landing detection method, device, electronic equipment and storage medium to improve the accuracy of landing detection.

[0007] In a first aspect, embodiments of this application provide a method for detecting container spreader attachment, including: Collect current frame 3D point cloud data covering the area below the spreader and above the trailer; the area above the trailer is used to place the container to be grabbed. When it is determined that the preset preconditions for container grabbing detection are met, the three-dimensional point cloud data of the current frame is selected as the region of interest point cloud, which is based on the predetermined position of the front surface of the container to be grabbed, extends forward and backward by a first preset distance in the driving direction, extends to the left and right sides by a second preset distance in the horizontal direction perpendicular to the driving direction, and is within a preset height range. Along the height direction, the point cloud of the region of interest is divided into multiple point cloud cells at preset intervals; The reference width of the container to be grabbed is determined based on the point cloud in at least one point cloud cell with the lowest height position; and the width difference is determined based on the difference between the width of the point cloud in the point cloud cell where the container top height is located and the width of the point cloud in the adjacent upper point cloud cell in the horizontal direction and the reference width. Based on the maximum height of the point cloud cell containing the container top height and at least one adjacent point cloud cell above it, determine the height difference used to characterize the distance between the lower edge of the spreader and the upper edge of the container top to be grabbed. Based on the width difference and the height difference, it is determined whether the spreader and the container to be grabbed are in a docking state.

[0008] In conjunction with the first aspect, this application provides a first possible implementation of the first aspect, wherein, after acquiring the current frame 3D point cloud data covering the area below the spreader and above the trailer, the method further includes: Based on the current frame's 3D point cloud data, determine the height of the spreader, the yaw angle of the trailer, and whether there is a container to be grabbed currently on the trailer; When a container to be grabbed is placed on the trailer, the top height of the container to be grabbed is determined based on the current frame's 3D point cloud data; When the height of the lifting device is less than the first preset height, the yaw angle is less than the preset angle, and the height of the top of the container is determined, it indicates that the preset preconditions for container placement detection have been met.

[0009] In conjunction with the first possible implementation of the first aspect, this application provides a second possible implementation of the first aspect, wherein determining the reference width of the container to be grabbed based on the point cloud within at least one point cloud cell with the lowest height position includes: Select the two non-empty point cloud cells with the lowest height from a set of point cloud cells; Calculate the width of the point cloud in the horizontal direction within each non-empty point cloud cell, and use the average of the two widths as the baseline width.

[0010] In conjunction with the first aspect, this application provides a third possible implementation of the first aspect, wherein determining the width difference based on the difference between the width of the point cloud in the point cloud cell where the box top height is located and the width of the point cloud in the adjacent upper point cloud cell in the horizontal direction and the reference width includes: For the point cloud cell containing the height of the box top and its adjacent upper point cloud cells, the point cloud cells that are more than a third preset distance from the center line in the horizontal direction are determined as outer edge points; Calculate the first width of the outer edge point in the horizontal direction of the point cloud cell containing the box top height, and the second width of the outer edge point in the horizontal direction of the adjacent upper point cloud cell; The maximum of the difference between the first width and the reference width, and the difference between the second width and the reference width, is taken as the width difference.

[0011] In conjunction with the first aspect, this application provides a fourth possible implementation of the first aspect, wherein determining the height difference used to characterize the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height value of the point cloud cell containing the container top height and at least one adjacent point cloud cell above it, includes: For the point cloud cell at the height of the box top and its adjacent first and second point cloud cells above it, the point cloud cells that are less than a fourth preset distance from the center line in the horizontal direction are determined as inner cavity points; wherein, among the adjacent first and second point cloud cells, the second point cloud cell is located above the first point cloud cell; Obtain the maximum height of the inner cavity point in the point cloud cell containing the top height of the box; Obtain the maximum and minimum heights of the inner cavity points in the first point cloud cell; Obtain the minimum height of the inner cavity point in the second point cloud cell; Arrange the maximum height of the point cloud cell containing the top height of the box, the minimum height of the first point cloud cell, the maximum height of the second point cloud cell, and the minimum height of the second point cloud cell in order from low to high to construct a height sequence. Find the first maximum height value in the height sequence whose difference from the top height of the box is greater than a preset threshold, and use the difference between the maximum height value and the top height of the box as the height difference.

[0012] In conjunction with the first aspect, this application provides a fifth possible implementation of the first aspect, wherein determining whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference includes: When the width difference is greater than a preset width threshold and the height difference is less than a first preset height threshold, or when the height difference is less than a second preset height threshold, it indicates that the spreader and the container to be grabbed are in a docking state; wherein, the first preset height threshold is greater than the second preset height threshold.

[0013] In conjunction with the first aspect, this application provides a sixth possible implementation of the first aspect, wherein determining whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference includes: The width difference and the height difference are mapped to width confidence and height confidence using the following formulas:

[0014]

[0015] in, For width confidence; High confidence level; The width difference; The width difference; The preset width threshold; The first preset height threshold; and The preset gain coefficient; The confidence level of a strongly boxed item is calculated using the following formula:

[0016] in, To strengthen the confidence level of a single item in the box; The preset gain coefficient; The second preset height threshold is set, and the first preset height threshold is greater than the second preset height threshold. The total confidence level is calculated using the following formula. :

[0017] When the total confidence level is greater than a preset probability threshold, it is determined that the spreader and the container to be grabbed are in a docking state.

[0018] Secondly, embodiments of this application also provide a container spreader landing detection device, comprising: The acquisition module is used to acquire current frame 3D point cloud data covering the area below the spreader and above the trailer; the area above the trailer is used to place the container to be grabbed. The filtering module is used to filter out the three-dimensional point cloud data of the current frame when it is determined that the preset preconditions for container grabbing detection are met. The three-dimensional point cloud data is selected as the region of interest point cloud based on the preset position of the front surface of the container to be grabbed as a reference, which is extended forward and backward by a first preset distance in the driving direction, and extended to the left and right sides by a second preset distance in the horizontal direction perpendicular to the driving direction based on the center line of the trailer, and the height is within a preset height range. The segmentation module is used to divide the region of interest point cloud into multiple point cloud cells along the height direction at preset intervals; The first determining module is used to determine the reference width of the container to be grabbed based on the point cloud in at least one point cloud cell with the lowest height position; and to determine the width difference based on the difference between the width of the point cloud in the point cloud cell where the top of the container is located and the width of the point cloud in the adjacent upper point cloud cell in the horizontal direction and the reference width. The second determining module is used to determine the height difference that characterizes the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height of the point cloud cell where the top height of the container is located and at least one adjacent point cloud cell above it. The judgment module is used to determine whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference.

[0019] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps in any of the possible implementations of the first aspect described above are performed.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps in any of the possible implementations of the first aspect described above.

[0021] This application provides a container spreader landing detection method, device, electronic equipment, and storage medium. It utilizes a non-contact vehicle-mounted LiDAR sensing method to collect current-frame 3D point cloud data covering the area below the spreader and above the trailer, replacing mechanical switches and fundamentally avoiding wear and tear, false triggering, and maintenance issues associated with mechanical components. Furthermore, the acquisition of 3D point cloud data is unaffected by environmental factors such as lighting changes, backlighting, nighttime, rain, fog, and container color differences, eliminating the need for extensive training with labeled data and demonstrating excellent cross-scene adaptability. Furthermore, when the preset preconditions for container detection are met, the region of interest (ROI) point cloud is selected from the current frame's 3D point cloud data, thus limiting the analysis scope to the critical spatial area where the spreader and container may come into contact. Based on this, the ROI point cloud is divided into multiple point cloud cells along the height direction at preset intervals. Through gridding, the discrete 3D point cloud data is transformed into structured cell information, making the statistical features within a single point cloud cell more robust and stable than those in a single frame of point cloud data. This eliminates the need for precise separation of the spreader's lower edge point cloud and the container's top edge point cloud in a single frame of point cloud data. Subsequently, the width representing the top position of the container to be grabbed and the reference width are determined. The width difference and the height difference, which characterize the distance between the lower edge of the spreader and the upper edge of the container top, are used to determine the fit between the spreader and the container. Since the spreader's locking head and base frame structure extend beyond the container edge when the container is in the grabbing state, the width difference increases and the distance between the lower edge of the spreader and the upper edge of the container top decreases, thus reducing the height difference. Therefore, by jointly determining the width difference and the height difference, two independent geometric features with clear physical meaning, the degree of fit between the spreader and the container can be reflected without separating the spreader point cloud and the container point cloud. This effectively avoids misjudgments caused by the difficulty in reliably separating the lower edge of the spreader and the top edge of the container in a single frame of 3D point cloud due to noise, obstruction, and vibration, and direct height comparison. This achieves real-time container grabbing detection with low false positives.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of a container spreader landing detection method provided in an embodiment of this application is shown; Figure 2This illustration shows a schematic diagram of a region of interest point cloud in a suspended, non-coffin state provided by an embodiment of this application; Figure 3 This illustration shows a schematic diagram of a region of interest point cloud with the lifting device attached to the container, as provided in an embodiment of this application. Figure 4 This illustration shows a schematic diagram of a lifting device whose lower edge and top edge are located within the same point cloud cell, according to an embodiment of this application. Figure 5 This illustration shows a schematic diagram of a lifting device provided in an embodiment of this application, in which the lower edge of the lifting device is located in the point cloud cell adjacent to the upper point cloud cell at the height of the box top, and is close to the lower boundary of the point cloud cell; Figure 6 This illustration shows a schematic diagram of a container top edge spanning the cell boundary and a lifting device bottom edge located within a first point cloud cell, as provided in an embodiment of this application. Figure 7 This illustration shows a point cloud cell where the top edge of the box spans the cell boundary, and the bottom edge of the hanger is completely empty between the top edge of the box and the top edge of the box, according to an embodiment of this application. Figure 8 This illustration shows a structural schematic diagram of a container spreader landing detection device provided in an embodiment of this application; Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0026] For box inspection, existing technologies mainly fall into the following categories: For enclosure detection based on contact or mechanical switches, specifically, limit switches or pressure sensors are installed on the locking head or lower edge of the lifting device. A signal is triggered when the locking head presses against the top of the enclosure. However, the mechanical components in the switches or sensors are easily damaged or prone to false triggers due to long-term exposure to dust, salt spray, and impact environments, resulting in high maintenance costs.

[0027] Image-based or deep learning-based container detection uses a vehicle-mounted camera to detect or classify targets at the interface between the spreader and the container. However, this method is significantly affected by environmental factors such as changes in lighting, backlighting, nighttime conditions, rain, fog, and differences in container color. Furthermore, it requires a large amount of labeled data for training and has poor cross-scene generalization ability.

[0028] The landing detection method based on threshold comparison of single-frame laser 3D point clouds involves filtering the single-frame laser 3D point cloud and then directly comparing the Z-coordinate of a reference point on the lifting device with the Z-coordinate of the top of the box. However, for this method to be accurate, the lower edge of the lifting device and the top edge of the box must first be precisely separated in the single-frame laser 3D point cloud. But when noise, occlusion, and jitter exist in the single-frame 3D point cloud, it is difficult to reliably separate the two points, and directly comparing height thresholds can easily lead to misjudgments.

[0029] Based on this, embodiments of this application provide a container spreader landing detection method, apparatus, electronic device, and storage medium. It uses only the current frame's 3D point cloud data for landing detection, eliminating the need for additional sensors and avoiding the problems associated with mechanical contact. Furthermore, the acquisition of the current frame's 3D point cloud data is unaffected by environmental factors, requiring no large amount of labeled data for training, exhibiting good cross-scene adaptability. Moreover, it eliminates the need to separate the spreader point cloud from the container point cloud, reducing misjudgments. The following is a description through embodiments.

[0030] To facilitate understanding of this embodiment, a container spreader landing detection method disclosed in this application will first be described in detail. This method can be applied to container operation scenarios such as ports, railway ports, and freight yards. Figure 1 As shown, the process includes the following steps S101-S106: S101: Collects current frame 3D point cloud data covering the area below the spreader and above the trailer; the area above the trailer is used to place the container to be grabbed.

[0031] In this embodiment, the spreader refers to the lifting device on the lifting and loading equipment, suspended below the hoisting wire rope. Its main body is a beam structure, with rotatable locks at the four corners of the beam, used to cooperate with the corner fittings on the top of the container to be grabbed to achieve grabbing and release. The lifting and loading equipment includes, for example, a reach stacker, a yard crane, or a quay crane.

[0032] A trailer is a transport vehicle platform used to carry containers; the area above the trailer is where the container to be grabbed is placed. The trailer's tractor unit constitutes the front towing part of the trailer, used to connect to the trailer chassis and provide traction power.

[0033] The towing head is equipped with a lidar unit to collect real-time 3D point cloud data of the area below the spreader and above the trailer. The current frame 3D point cloud data refers to the 3D point cloud data collected at the current moment.

[0034] The current frame of 3D point cloud data consists of multiple spatial points with 3D coordinates (x, y, z). The coordinate system used in this embodiment is the trailer body coordinate system, that is, the forward direction of the trailer's tractor (truck head) is the positive x-axis (driving direction), the leftward direction perpendicular to the x-axis is the positive y-axis (horizontal direction), and the upward direction perpendicular to the xoy plane is the positive z-axis (height direction).

[0035] In one possible implementation, after step S101 and before step S102, multi-level validity pre-screening can be performed according to the following steps S1001-S1003 to ensure that subsequent box-mounting detection has reliable judgment criteria: S1001: Based on the current frame's 3D point cloud data, determine the height of the spreader, the yaw angle of the trailer, and whether there is a container to be grabbed on the trailer.

[0036] In this step, using the acquired 3D point cloud data of the current frame, existing point cloud processing algorithms are used to determine the height of the spreader, the yaw angle of the trailer, and the cargo status of the trailer. The height of the spreader refers to its position relative to the origin of the vehicle coordinate system in the vertical direction. The yaw angle of the trailer refers to the deflection angle between the trailer and the tractor, used to characterize whether the trailer's parking posture is normal. Determining whether a container to be grabbed is placed on the trailer can be achieved, for example, by detecting the presence of container feature point clouds above the trailer's load-bearing surface in the point cloud data.

[0037] S1002: When there is a container to be grabbed on the trailer, determine the top height of the container to be grabbed based on the current frame's 3D point cloud data.

[0038] In this step, the top height of the container refers to the coordinate value of the plane containing the top corner piece of the container to be grabbed in the height direction (z-axis direction).

[0039] When there is no container to be grabbed on the trailer, there is no target to grab, so the subsequent container grabbing detection has no judgment benchmark and is directly judged as not being grabbed.

[0040] S1003: When the height of the spreader is less than the first preset height, the yaw angle is less than the preset angle, and the height of the top of the container is determined, it means that the preset preconditions for container landing detection have been met.

[0041] In this step, when the following three conditions are met simultaneously, the preset prerequisites for the box-mounting detection are determined to be met, and the process can proceed to the subsequent step S102 to determine the box-mounting status: The spreader height is less than the first preset height: The first preset height is a preset spreader height threshold. When the spreader height is less than the first preset height, it indicates that the spreader has descended to a height range where it may contact the container to be grabbed, rather than being in a high-positioned suspended state. If the spreader height is greater than or equal to the first preset height, it means that the spreader is still operating at a high position and cannot be in a container-attaching state, and is directly judged as not in a container-attaching state.

[0042] The trailer's yaw angle is less than the preset angle: The preset angle is a preset threshold for trailer attitude deviation. When the trailer's yaw angle is less than the preset angle, it indicates that the trailer's parking attitude is basically normal and loading and unloading operations can be carried out. If the yaw angle is too large, it means that the trailer's attitude is abnormal. In this state, normal loading and unloading operations will not be carried out and it will be directly judged as a non-loading state.

[0043] The top height of the container to be grabbed has been determined: Determining the top height is a necessary prerequisite for subsequent calculation of the height difference and width difference. If the top height cannot be successfully determined, it means that there is currently no judgment benchmark, and it is directly judged as a non-grabbing state.

[0044] Through the above multi-level validity pre-screening, 3D point cloud data that clearly do not meet the conditions for box detection are filtered out before entering the point cloud screening of the region of interest and the calculation of box features, thus avoiding invalid calculations and ensuring that subsequent processing steps have a reliable data foundation.

[0045] S102: When it is determined that the preset preconditions for container grabbing detection are met, select from the current frame of 3D point cloud data the 3D point cloud data that are extended forward and backward by a first preset distance in the driving direction based on the predetermined front surface position of the container to be grabbed, and extended to the left and right sides by a second preset distance in the horizontal direction perpendicular to the driving direction based on the center line of the trailer, and whose height is within a preset height range, as the region of interest point cloud.

[0046] The purpose of this step is to filter out the region of interest point cloud from the current frame's 3D point cloud data in order to narrow down the data range for subsequent processing and eliminate interference from irrelevant point clouds.

[0047] Specifically, the selection of point clouds for regions of interest is based on the following three limiting conditions: Along the trailer's travel direction (x-axis direction), using the predetermined position of the front surface of the container to be grabbed as a reference, the distance is extended forward and backward by a first preset distance. The front surface of the container refers to the vertical surface of the container facing the tractor, and its position is the x-axis coordinate of this vertical surface in the travel direction. The first preset distance is a preset extension amount in the travel direction, for example, it can be set to 20 centimeters, ensuring that the region of interest covers a 20-centimeter area in front of and behind the front surface in the travel direction. This ensures that the critical area where the lower edge of the spreader may contact the top edge of the container is included, while excluding irrelevant areas other than the sides of the container.

[0048] In the horizontal direction (y-axis direction) perpendicular to the trailer's travel direction, a second preset distance is extended to the left and right sides, using the trailer's centerline as a reference. The trailer's centerline refers to the central axis extending horizontally along the travel direction. The second preset distance is a preset horizontal extension amount, and its value should be greater than half the width of the spreader to ensure that the region of interest (ROI) covers the outer edges of the spreader's sides in the horizontal direction; for example, it can be set to 1.5 meters. Thus, the ROI covers the area within the second preset distance to the left and right of the trailer's centerline in the horizontal direction, including portions of the spreader that may extend beyond the container to be grabbed, while excluding irrelevant objects far from the trailer's sides.

[0049] In the vertical direction (z-axis), the height of the point cloud of the region of interest (ROI) lies within a preset height range. The preset height range refers to a pre-defined height limit. The lower limit of the preset height range is higher than the trailer's load-bearing surface but lower than the top height of the container to be grabbed. The upper limit of the preset height range is higher than the container's top height but lower than the lowest height of the spreader when not in operation. This ensures that the preset height range simultaneously includes the point cloud along both the top edge of the container and the bottom edge of the spreader. For example, the lower limit of the preset height range can be set to be offset downwards from the container's top height by a certain distance (e.g., 0.4 meters), and the upper limit can be set to be offset upwards from the container's top height by a certain distance (e.g., 0.8 meters). This ensures that, even with different container heights and a certain degree of height measurement error, the ROI still completely covers the key point cloud required for container identification.

[0050] By filtering based on the above three dimensions, a three-dimensional region of interest (ROI) is extracted from the current frame's 3D point cloud data. The point cloud within this ROI is the ROI point cloud. The ROI point cloud limits the analysis scope for container landing determination, focusing subsequent processing on the key spatial area where the spreader and the container to be grabbed may come into contact, effectively eliminating interference from irrelevant point clouds such as those from the ground, other parts of the trailer, and yard facilities.

[0051] like Figure 2 and Figure 3 As shown, Figure 2A schematic diagram of the region of interest point cloud is shown in the state where the spreader is suspended and not attached to the container. Figure 3 A schematic diagram of the region of interest point cloud with the spreader attached to the container is shown.

[0052] S103: Divide the point cloud of the region of interest into multiple point cloud cells along the height direction at preset intervals.

[0053] In this step, the region of interest (ROI) point cloud is divided into multiple point cloud cells along the height direction (z-axis) at preset intervals. The preset interval refers to a pre-defined cell height size, such as 0.2 meters. Each point cloud cell corresponds to a height layer in the height direction, containing all ROI point clouds falling within that height layer.

[0054] By dividing the point cloud of the region of interest into multiple point cloud cells, the originally discrete 3D point cloud data of the current frame is transformed into a structured sequence of cells arranged hierarchically along the height direction. The point cloud cells are arranged in ascending order of height, with the point cloud cell at the lowest height corresponding to the lowest position in the region of interest, and the point cloud cell at the highest height corresponding to the highest position in the region of interest.

[0055] S104: Determine the reference width of the container to be grabbed based on the point cloud in at least one point cloud cell with the lowest height position; and determine the width difference based on the difference between the width of the point cloud in the horizontal direction of the point cloud cell where the container top height is located and the width of the point cloud cell adjacent to it above the container top height and the reference width.

[0056] In this embodiment, the width difference is used to characterize the extent to which the spreader extends beyond the side edge of the container to be grabbed at the corresponding height position. When the container is in the grabbing state, the spreader's lock head and base frame structure will extend beyond the edge of the container body to be grabbed, making the width difference larger.

[0057] In one possible implementation, when performing step S104 to determine the reference width of the container to be grabbed based on the point cloud in at least one point cloud cell with the lowest height position, the specific steps S1041-S1042 can be performed as follows: S1041: Select the two non-empty point cloud cells with the lowest height position among multiple point cloud cells.

[0058] In step S1041, from multiple point cloud cells, two point cloud cells with the lowest height and containing point cloud data are selected in ascending order of height. A non-empty point cloud cell is one containing at least one point cloud data point. The point cloud cell with the lowest height corresponds to the lowest position in the region of interest. Since this position is lower than the top height of the container to be grabbed, the point cloud at this position mainly originates from the side wall of the container, accurately reflecting the container's lateral dimensions.

[0059] S1042: Calculate the width of the point cloud in the horizontal direction in each non-empty point cloud cell, and take the average of the two widths as the base width.

[0060] In step S1042, for each of the two non-empty point cloud cells selected in step S1041, the width of all point clouds within that cell in the horizontal direction (y-axis direction) is calculated. This width is specifically the difference between the maximum and minimum y-coordinate values ​​of all point clouds within that cell, reflecting the lateral dimension of the container to be grabbed in the horizontal direction at that height level. After calculating the widths corresponding to the two non-empty point cloud cells, the arithmetic mean of the two widths is taken as the reference width of the container to be grabbed. The reference width represents the reference lateral dimension of the container itself in the horizontal direction, used for subsequent comparison with the lateral width of the height level where the spreader is located.

[0061] In one possible implementation, when performing step S104 to determine the width difference based on the difference between the horizontal width of the point cloud in the point cloud cell where the box top height is located and the width of the point cloud in the adjacent upper point cloud cell and the reference width, the specific steps S1043-S1045 can be performed as follows: S1043: For the point cloud cell at the height of the box top and its adjacent point cloud cells above it, the point cloud cells that are more than three preset distances from the center line in the horizontal direction are identified as outer edge points.

[0062] In step S1043, the point cloud cell containing the container top height refers to the point cloud cell into which the container top height determined in step S1002 falls, i.e., the point cloud cell corresponding to the height of the top corner fitting of the container to be grabbed. The adjacent upper point cloud cell refers to the point cloud cell immediately above the point cloud cell containing the container top height in the height direction. For these two point cloud cells, with the centerline of the trailer as the reference, the horizontal distance of each point cloud in the point cloud cell from the centerline is calculated, and the point cloud with a distance greater than a third preset distance is determined as the outer edge point. The third preset distance is a preset horizontal distance threshold. The outer edge point corresponds to the area far from the centerline of the trailer. These point clouds mainly come from the grippers, edges, and other structures on both sides of the spreader, and can reflect the lateral extension width of the spreader at the corresponding height.

[0063] S1044: Calculate the first width of the outer edge point in the horizontal direction of the point cloud cell containing the top height of the box, and the second width of the outer edge point in the horizontal direction of the adjacent upper point cloud cell.

[0064] In step S1044, for each point cloud cell containing the top height of the container, the width of all outer edge points within that cell in the horizontal direction (y-axis direction) is calculated and denoted as the first width. Specifically, the first width is the difference between the maximum and minimum y-coordinate values ​​of all outer edge points within that point cloud cell, reflecting the lateral extension width of the spreader at the top height level of the container. Similarly, for each adjacent upper point cloud cell, the width of all outer edge points within that cell in the horizontal direction is calculated and denoted as the second width, reflecting the lateral extension width of the spreader at the adjacent height level above the container top.

[0065] S1045: Take the largest of the differences between the first width and the reference width, and the differences between the second width and the reference width, as the width difference.

[0066] In step S1045, the difference between the first width and the reference width, and the difference between the second width and the reference width are calculated respectively. The difference between the first width and the reference width reflects the extent to which the lateral width of the spreader exceeds the reference width of the container to be grabbed at the top height level, and the difference between the second width and the reference width reflects the extent to which the lateral width of the spreader exceeds the reference width at the adjacent height level above the top of the container. The larger of these two differences is taken as the width difference to ensure that the widest part of the spreader at different height positions can be captured.

[0067] When the spreader is not in contact with the container, it has not yet descended to the vicinity of the top height of the container. There may be no outer edge point in the point cloud cell where the top height of the container is located and the point cloud cell above it, or the lateral width corresponding to the outer edge point may be similar to the reference width with a small width difference. After the container is in contact, the lock head and base frame structure of the spreader extend beyond the edge of the container, and the lateral width corresponding to the outer edge point is significantly greater than the reference width, and the width difference increases accordingly.

[0068] S105: Determine the height difference used to characterize the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height of the point cloud cell where the container top height is located and at least one adjacent point cloud cell above it.

[0069] In this embodiment, the height difference is used to quantitatively describe the vertical distance between the lower edge of the spreader and the upper edge of the container top to be grabbed. When the container is being grabbed, the lower edge of the spreader is close to the upper edge of the container top, and the height difference is close to zero or very small; when the spreader is suspended in the air, the height difference is larger.

[0070] Ideally, if the lidar can accurately measure the three-dimensional coordinates of each point, and the boundary between the lower edge of the lifting device and the upper edge of the box is clear, the height difference can be obtained simply by subtracting the height coordinate of the lower edge of the lifting device from the height of the box. However, in actual operation, the lidar experiences ranging jitter and inaccuracies at the top edge of the box. The point cloud at the lower edge of the lifting device and the point cloud at the top edge of the box are spatially very close and mixed together, making it impossible to reliably separate them in a single frame of point cloud. Furthermore, because step S103 discretizes the point cloud of the region of interest into multiple point cloud cells along the height direction at preset intervals (e.g., 0.2 meters), the actual distance between the lower edge of the lifting device and the upper edge of the box may be less than the height of one point cloud cell, or it may span one or more point cloud cells. This results in the precise position of the lower edge of the lifting device potentially appearing in different point cloud cells under different operating conditions.

[0071] To address the aforementioned issues, this embodiment does not attempt to precisely separate the lower edge of the spreader and the top edge of the container from the point cloud. Instead, it employs an indirect measurement method based on a height sequence. The core idea is to extract the maximum and minimum values ​​in the height direction of the point cloud data in the region near the trailer centerline (i.e., the region where the lower edge of the spreader and the top corner fittings of the container are located) from the point cloud cell containing the container's top height. This constructs a height sequence reflecting the geometric relationship between the lower edge of the spreader and the upper edge of the container. Since this sequence contains multiple candidate height values ​​that could represent the lower edge of the spreader, arranged in ascending order of height, it is sufficient to find the first value in the sequence that is significantly higher than the top height of the container. The difference between this value and the top height can then be used as a reliable estimate of the actual distance.

[0072] The following combination Figures 4 to 7 The four typical operating conditions shown explain in detail why the height difference can be accurately calculated by constructing a height sequence and finding the first threshold exceeding the threshold. Figures 4 to 7 In the diagram, five black dashed lines are arranged along the height direction, with each adjacent dashed line representing a point cloud cell. Green-filled cells represent the point cloud cells where the container top height falls (i.e., the point cloud cells where the container top height determined in step S1002 falls), red diagonal lines indicate the upper edge of the container top to be grabbed, and blue diagonal lines indicate the lower edge of the spreader.

[0073] Operating condition one, such as Figure 4 As shown, a schematic diagram is presented where the lower edge of the spreader and the upper edge of the box are located within the same point cloud cell.

[0074] In this operating condition, both the upper edge of the container top and the lower edge of the spreader are located within the green grid, with a small actual distance between them, meaning the spreader is very close to the container top. At this point, the maximum height of the inner cavity point within the green grid (i.e., the uppermost point in that grid) is contributed by the lower edge of the spreader, and this value directly approximates the height position of the lower edge of the spreader. Subtracting this maximum height from the height of the container top gives the height difference. Although approximating the lowest point of the spreader's lower edge with the highest point of the cell introduces a slight error, this approximation error is negligible because the lower edge of the spreader is usually close to horizontal, and the distance between them is very small in this condition.

[0075] Operating condition two, such as Figure 5 As shown, a schematic diagram is shown where the lower edge of the spreader is located in the point cloud cell adjacent to the top of the box, and is close to the lower boundary of that point cloud cell.

[0076] In this condition, the upper edge of the box top is located within the green grid, and the lower edge of the spreader is located within the point cloud cell immediately above the green grid (i.e., the first point cloud cell), with the lower edge of the spreader just close to the bottom boundary of this first point cloud cell. At this point, the point cloud within the green grid primarily reflects the box structure, and the minimum height of the inner cavity points in the first point cloud cell (i.e., the lowest point within that cell) is contributed by the lower edge of the spreader. Subtracting this minimum height from the box top height yields the actual distance between the two.

[0077] Operating condition three, such as Figure 6 As shown, a schematic diagram is presented where the top edge of the box crosses the cell boundary and the bottom edge of the hanger is located within the first point cloud cell.

[0078] In this scenario, because the upper edge of the container is relatively intact in this frame's point cloud image, the point cloud over the top edge of the container crosses the upper boundary of the green grid, extending partially to the lower part of the first point cloud cell. At this point, the maximum height of the green grid does not represent the lower edge of the spreader (it may be contributed by the container top), and the minimum height of the first point cloud cell also does not represent the lower edge of the spreader (it may be contributed by the container top that has crossed over). What truly represents the lower edge of the spreader is the maximum height of the internal points within the first point cloud cell, because the top point cloud of the container that has crossed over cannot reach the upper part of this cell. Therefore, subtracting the maximum height of the first point cloud cell from the height of the container top is the only way to accurately reflect the distance between the lower edge of the spreader and the upper edge of the container top.

[0079] Operating condition four, such as Figure 7 As shown, a schematic diagram is presented where the top edge of the box crosses the cell boundary, and a point cloud cell is completely left empty between the bottom edge of the hanger and the top edge of the box.

[0080] In this condition, similar to condition three, the point cloud at the top edge of the container extends beyond the upper boundary of the green grid. The difference is that the lower edge of the spreader is higher, leaving a completely empty cell in the vertical direction between it and the top edge of the container. At this point, the maximum height of the green grid, the minimum height of the first point cloud cell, and the maximum height of the first point cloud cell cannot accurately reflect the lower edge of the spreader, because these values ​​are either contributed by the top edge of the container or are located in the empty area. The value that truly represents the lower edge of the spreader is the minimum height of the inner cavity points in the second point cloud cell (i.e., the point cloud cell immediately above the first point cloud cell). Subtracting this minimum height from the height of the container top is the only way to accurately reflect the gap between the two.

[0081] As can be seen from the four working conditions described above, the actual distance between the lower edge of the spreader and the upper edge of the box always corresponds to the first value in the height sequence that is significantly higher than the height of the box top. Under all four conditions, the truly effective height value is precisely the first value in the sequence whose difference from the box top height exceeds a reasonable threshold (e.g., 0.8 meters, which is set based on the lidar top-edge ranging error and the spreader's structural dimensions). Therefore, by constructing a height sequence and searching sequentially, the correct representative height value can be adaptively selected under various complex working conditions without needing to pre-determine which working condition is currently being used.

[0082] In one possible implementation, when performing step S105, the following steps S1051-S1056 can be specifically performed: S1051: For the point cloud cell at the height of the box top and the first and second point cloud cells above it, the point cloud cells that are less than the fourth preset distance from the center line in the horizontal direction are determined as inner cavity points; wherein, among the adjacent first and second point cloud cells, the second point cloud cell is located above the first point cloud cell.

[0083] In step S1051, the point cloud cell containing the container top height refers to the point cloud cell into which the container top height determined in step S1002 falls. The first point cloud cell refers to the point cloud cell immediately above the point cloud cell containing the container top height in the height direction. The second point cloud cell refers to another point cloud cell immediately above the first point cloud cell. For the above three point cloud cells, using the trailer's centerline as a reference, the distance of each point cloud from the centerline in the horizontal direction (y-axis direction) is calculated, and point clouds with distances less than a fourth preset distance are identified as inner cavity points. The fourth preset distance is a preset horizontal distance threshold, the value of which is less than the third preset distance used to determine the outer edge points. The inner cavity points correspond to the area near the trailer's centerline, located directly above the container. These point clouds mainly come from the bottom crossbeam structure of the spreader's lower edge and the top surface of the top corner fittings of the container to be grabbed, and can reflect the height distribution in this area. The annular region between the centerline and the inner cavity point may contain structures such as the lifting device lock head. This part of the point cloud is not included in the height difference calculation to avoid interfering with the determination of the lowest point of the lower edge of the lifting device and the upper edge of the top of the box.

[0084] S1052: Get the maximum height of the inner cavity point in the point cloud cell containing the top height of the box.

[0085] In this embodiment, the coordinates of all internal cavity points within the point cloud cell containing the box top height are extracted along the height direction (z-axis), and the maximum value is recorded as the maximum height of the current cell. This value corresponds to... Figure 4 The height of the lower edge of the spreader when it is close to the top of the box during the working condition, or as the first candidate value in the height sequence.

[0086] S1053: Get the maximum and minimum heights of the inner cavity points in the first point cloud cell.

[0087] Extract the maximum and minimum height values ​​(maximum and minimum height values) of all internal cavity points within the first point cloud cell, and denote them as the maximum and minimum height values ​​of the first cell, respectively. These two values ​​correspond to... Figure 6 Operating conditions and Figure 5 The possible height position of the lower edge of the lifting device during operation.

[0088] S1054: Get the minimum height of the inner cavity point in the second point cloud cell.

[0089] Extract the minimum height value (minimum height value) of all internal cavity points within the second point cloud cell, and record it as the minimum height value of the second cell. This value corresponds to... Figure 7 The height position of the lower edge of the spreader when there is a complete gap of one point cloud cell between the lower edge of the spreader and the upper edge of the box during operation.

[0090] S1055: Arrange the maximum height of the point cloud cell containing the top height of the box, the minimum height of the first point cloud cell, the maximum height of the first point cloud cell, and the minimum height of the second point cloud cell in order from low to high to construct a height sequence.

[0091] In this embodiment, the four maximum and minimum height values ​​obtained in steps S1052 to S1054 are arranged sequentially according to their height positions from low to high, forming a height sequence containing four elements. This sequence covers all possible height layers where the lower edge of the spreader may appear, and each element in the sequence represents a potential candidate point for the lower edge of the spreader.

[0092] S1056: Find the first maximum height value in the height sequence whose difference from the top height of the container is greater than a preset threshold, and use the difference between the maximum height value and the top height of the container as the height difference.

[0093] Starting from the first element of the height sequence (the maximum height of the current cell), the difference between each maximum height and the box top height determined in step S1002 is calculated sequentially. After finding the first maximum height where this difference exceeds a preset threshold, the difference between this maximum height and the box top height is taken as the height difference for the current frame. The preset threshold needs to be determined by considering the ranging error of the LiDAR at the top edge of the box and the possible actual distance between the lower edge of the lifting device and the upper edge of the box; for example, it can be set to 0.8 meters.

[0094] The correct height values ​​for each of the four working conditions mentioned above ( Figure 4 The current maximum grid height, Figure 5 Minimum height of the first cell Figure 6 Maximum height of the first grid Figure 7 The minimum height of the second frame is exactly the first height value in the height sequence under each working condition that exceeds the threshold. Therefore, this embodiment achieves accurate calculation of the height difference between the lower edge of the spreader and the upper edge of the box under four typical working conditions using a unified adaptive search method, without the need to pre-identify which working condition the current frame belongs to, nor the need to accurately separate the point cloud of the lower edge of the spreader and the point cloud of the top edge of the box.

[0095] S106: Determine whether the spreader and the container to be grabbed are in a docking state based on the width difference and height difference.

[0096] In step S106, a comprehensive determination is made based on the width and height differences to determine whether the spreader and the container to be grabbed are in a docked state. Docked state refers to the state where, during the descent of the spreader, the locking head or the lower edge of the spreader is stably and securely aligned with the top corner fittings of the container. Docked state determination is a critical trigger signal in the automated loading and unloading process. Only after confirming stable docking can the lifting and unloading equipment perform subsequent actions such as locking and lifting. If a docked state determination fails to detect (the container is actually docked but not detected), the equipment will be unable to perform the locking action in time, leading to an interruption of operations. If a false detection occurs (the container is not actually docked but is mistakenly judged as docked), the equipment may perform locking while the spreader is suspended in the air, potentially causing the container to fall or damage the trailer, resulting in safety accidents.

[0097] This embodiment combines the width difference and height difference, two geometric features with clear physical significance, for joint judgment. This is because when the spreader is engaged, the spreader's locking mechanism and base frame extend beyond the container's edge, causing the spreader's lateral width near the container's top height to exceed the container's baseline width, thus significantly increasing the width difference. Simultaneously, the vertical distance between the spreader's lower edge and the container's top edge decreases significantly, thus reducing the height difference. Conversely, when the spreader is not yet engaged and is still suspended above the container, its lateral width has not yet extended beyond the container, resulting in a smaller or zero width difference, and a larger gap exists between the spreader's lower edge and the container's top edge, leading to a larger height difference. Therefore, the width difference and height difference reflect the relative positional relationship between the spreader and the container from two independent dimensions. Using them together effectively improves the accuracy and robustness of the engagement judgment.

[0098] In one possible implementation, step S106 can be performed specifically as follows: S1061: When the width difference is greater than the preset width threshold and the height difference is less than the first preset height threshold, or when the height difference is less than the second preset height threshold, it indicates that the spreader and the container to be grabbed are in the container-attached state; wherein, the first preset height threshold is greater than the second preset height threshold.

[0099] In this step, a hard threshold condition is used to determine the box-mounted status. The preset width threshold, the first preset height threshold, and the second preset height threshold are all pre-set judgment thresholds, which can be adaptively adjusted according to factors such as the type of lifting equipment used, the measurement accuracy of the lidar, and the loading status of the trailer.

[0100] The above judgment conditions include two situations; if either situation is met, the box is judged to be in a landing state: The first scenario is a combined condition: when the width difference is greater than a preset width threshold and the height difference is less than a first preset height threshold, the container is considered to be in a landing state. This condition requires that the width and height dimensions simultaneously meet the landing conditions, meaning that the lateral width of the spreader has significantly exceeded the reference width of the container (the width difference is large enough), and the lower edge of the spreader is close to the upper edge of the container top (the height difference is small enough). This branch corresponds to a typical landing condition, where the spreader is stably attached to the top of the container, the locking head and base frame structure extend beyond the sides of the container, and both the width and height differences are significant.

[0101] The second scenario involves a single condition for forced container landing: when the height difference is less than the second preset height threshold, the container is directly determined to be in a landing state regardless of whether the width difference meets the condition. The second preset height threshold is less than the first preset height threshold, representing a more stringent height threshold. When the height difference is less than the second preset height threshold, it indicates that the distance between the lower edge of the spreader and the upper edge of the container top is extremely small, and the spreader is very tightly attached to the top of the container. In this case, even if the width difference is not significant due to point cloud obstruction or the special posture of the spreader, it can be confidently concluded that the spreader has landed on the container, and therefore the landing state is directly determined.

[0102] By setting the above two conditions, it can cover typical box-fitting conditions where both width and height characteristics are obvious, and can also make correct judgments in extreme fitting conditions where width characteristics are unreliable but height characteristics are extremely clear, thus taking into account both the comprehensiveness and reliability of the detection.

[0103] In another possible implementation, considering that the above-mentioned hard threshold determination method may cause frequent jumps in the binning state between consecutive frames due to random disturbances in point cloud noise near the threshold boundary (i.e., the previous frame just exceeds the threshold and is judged as binning, the next frame falls back below the threshold due to noise and is judged as non-binning), a confidence mapping method based on the Sigmoid function can be adopted to transform the discrete hard threshold determination into a continuous confidence score, and then merge it into a total confidence score for determination. Based on this, when executing step S106, it can be specifically executed according to the following steps S1062-S1065: S1062: Map the width difference and height difference to width confidence and height confidence using the following formula:

[0104]

[0105] in, For width confidence; High confidence level; For width difference; For width difference; The preset width threshold; The first preset height threshold; and This is the preset gain coefficient.

[0106] Width confidence The value range is (0, 1), and the closer the value is to 1, the greater the probability of the box being hit based on the width dimension. High confidence level. The value range is (0, 1), and the closer the value is to 1, the greater the probability of the box landing from the height dimension. Preset gain coefficient. and This is used to adjust the transition slope of the Sigmoid function near the threshold. The larger the value, the steeper the transition and the closer it is to the hard threshold judgment; the smaller the value, the smoother the transition and the stronger the noise resistance.

[0107] S1063: Calculate the one-item confidence level of strong-boxing using the following formula:

[0108] in, To strengthen the confidence level of a single item in the box; The preset gain coefficient; The second preset height threshold is greater than the first preset height threshold.

[0109] Strong box single-item confidence level Dependent on height difference only Second preset height threshold When the height difference is much smaller hour, A value close to 1 indicates that the lifting device is extremely close to the top of the container and can be independently determined as having landed on the container.

[0110] S1064: Calculate the total confidence level using the following formula. :

[0111] in, The total confidence level is defined as the value within the range (0, 1). The design logic of this formula is as follows: when the strong-boxed individual item confidence level... At lower levels (i.e., when the height difference has not reached its minimum), the total confidence score is mainly determined by the width confidence score. and high confidence The product of these two dimensions is required to support the binning decision in order to obtain a high overall confidence level; when the binning individual item confidence level is strongly supported... When the value approaches 1 (i.e., the height difference is minimal), regardless of and How many values ​​should be taken, and what is the overall confidence level? All values ​​approach 1, thus achieving a judgment effect equivalent to the single-condition strong box condition in the second case of step S1061. This formula, through a smooth fusion of multiplicative and additive properties, ensures that the total confidence level continuously and stably reflects the probability of the box under various working conditions.

[0112] S1065: When the total confidence level is greater than the preset probability threshold, the spreader and the container to be grabbed are determined to be in the container-attached state.

[0113] In this step, the preset probability threshold is a pre-defined decision boundary, for example, it can be set to 0.5. When the calculated total confidence level... If the value is greater than 0.5, the spreader and the container to be grabbed in the current frame of 3D point cloud data are determined to be in a container-attached state; otherwise, it is determined to be in a non-attached state.

[0114] Based on the same technical concept, embodiments of this application also provide a container spreader landing detection device, such as... Figure 8 As shown, it includes: The acquisition module 801 is used to acquire current frame 3D point cloud data covering the area below the spreader and above the trailer; the area above the trailer is used to place the container to be grabbed. The filtering module 802 is used to filter out the three-dimensional point cloud data of the current frame when it is determined that the preset preconditions for container grabbing detection are met, and select the three-dimensional point cloud data that is extended forward and backward by a first preset distance in the driving direction based on the predetermined front surface position of the container to be grabbed, and extended to the left and right sides by a second preset distance in the horizontal direction perpendicular to the driving direction based on the center line of the trailer, and whose height is within a preset height range, as the region of interest point cloud. The segmentation module 803 is used to divide the region of interest point cloud into multiple point cloud cells along the height direction at preset intervals; The first determining module 804 is used to determine the reference width of the container to be grabbed based on the point cloud in at least one point cloud cell with the lowest height position; and to determine the width difference based on the difference between the width of the point cloud in the point cloud cell where the top height of the container is located and the width of the point cloud in the adjacent upper point cloud cell in the horizontal direction and the reference width. The second determining module 805 is used to determine the height difference that characterizes the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height of the point cloud cell where the top height of the container is located and at least one adjacent point cloud cell above it. The judgment module 806 is used to determine whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference.

[0115] Optionally, the device further includes: The third determining module is used to determine the height of the spreader, the yaw angle of the trailer, and whether there is a container to be grabbed on the trailer based on the current frame three-dimensional point cloud data after the acquisition module 801 acquires the current frame three-dimensional point cloud data covering the area below the spreader and above the trailer. The fourth determining module is used to determine the top height of the container to be grabbed based on the current frame three-dimensional point cloud data when the container to be grabbed is placed on the trailer. The fifth determining module is used to indicate that the preset preconditions for container landing detection have been met when the height of the lifting device is less than the first preset height, the yaw angle is less than the preset angle, and the height of the container top is determined.

[0116] Optionally, when the first determining module 804 determines the reference width of the container to be grabbed based on the point cloud in at least one point cloud cell with the lowest height position, it is specifically used for: Select the two non-empty point cloud cells with the lowest height from a set of point cloud cells; Calculate the width of the point cloud in the horizontal direction within each non-empty point cloud cell, and use the average of the two widths as the baseline width.

[0117] Optionally, when the first determining module 804 determines the width difference based on the difference between the width of the point cloud in the horizontal direction of the point cloud cell where the box top height is located and the width of the point cloud cell adjacent to it above, and the reference width, it is specifically used for: For the point cloud cell containing the height of the box top and its adjacent upper point cloud cells, the point cloud cells that are more than a third preset distance from the center line in the horizontal direction are determined as outer edge points; Calculate the first width of the outer edge point in the horizontal direction of the point cloud cell containing the box top height, and the second width of the outer edge point in the horizontal direction of the adjacent upper point cloud cell; The maximum of the difference between the first width and the reference width, and the difference between the second width and the reference width, is taken as the width difference.

[0118] Optionally, when the second determining module 805 determines the height difference characterizing the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height of the point cloud cell containing the container top height and at least one adjacent point cloud cell above it, the module is specifically used for: For the point cloud cell at the height of the box top and its adjacent first and second point cloud cells above it, the point cloud cells that are less than a fourth preset distance from the center line in the horizontal direction are determined as inner cavity points; wherein, among the adjacent first and second point cloud cells, the second point cloud cell is located above the first point cloud cell; Obtain the maximum height of the inner cavity point in the point cloud cell containing the top height of the box; Obtain the maximum and minimum heights of the inner cavity points in the first point cloud cell; Obtain the minimum height of the inner cavity point in the second point cloud cell; Arrange the maximum height of the point cloud cell containing the top height of the box, the minimum height of the first point cloud cell, the maximum height of the second point cloud cell, and the minimum height of the second point cloud cell in order from low to high to construct a height sequence. Find the first maximum height value in the height sequence whose difference from the top height of the box is greater than a preset threshold, and use the difference between the maximum height value and the top height of the box as the height difference.

[0119] Optionally, when the judgment module 806 is used to determine whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference, it is specifically used for: When the width difference is greater than a preset width threshold and the height difference is less than a first preset height threshold, or when the height difference is less than a second preset height threshold, it indicates that the spreader and the container to be grabbed are in a docking state; wherein, the first preset height threshold is greater than the second preset height threshold.

[0120] Optionally, when the judgment module 806 is used to determine whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference, it is specifically used for: The width difference and the height difference are mapped to width confidence and height confidence using the following formulas:

[0121]

[0122] in, For width confidence; High confidence level; The width difference; The width difference; The preset width threshold; The first preset height threshold; and The preset gain coefficient; The confidence level of a strongly boxed item is calculated using the following formula:

[0123] in, To strengthen the confidence level of a single item in the box; The preset gain coefficient; The second preset height threshold is set, and the first preset height threshold is greater than the second preset height threshold. The total confidence level is calculated using the following formula. :

[0124] When the total confidence level is greater than a preset probability threshold, it is determined that the spreader and the container to be grabbed are in a docking state.

[0125] Figure 9 A schematic diagram of an electronic device provided in this application embodiment includes: a processor 901, a memory 902, and a bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the electronic device runs the above-described information processing method, the processor 901 and the memory 902 communicate through the bus 903. The processor 901 executes the machine-readable instructions to perform the steps described in the embodiment of the container spreader landing detection method.

[0126] This application also provides a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps described in the embodiment of the container spreader landing detection method.

[0127] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, electronic equipment, and computer-readable storage media described above can be referred to the corresponding processes in the aforementioned embodiments of the container spreader loading detection method, and will not be repeated here.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, electronic devices, and computer-readable storage media can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or modules may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0131] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.

Claims

1. A method for detecting container landing on a container spreader, characterized in that, include: Collect current frame 3D point cloud data covering the area below the spreader and above the trailer; the area above the trailer is used to place the container to be grabbed. When it is determined that the preset preconditions for container grabbing detection are met, the three-dimensional point cloud data of the current frame is selected as the region of interest point cloud, which is based on the predetermined position of the front surface of the container to be grabbed, extends forward and backward by a first preset distance in the driving direction, extends to the left and right sides by a second preset distance in the horizontal direction perpendicular to the driving direction, and has a height within a preset height range. Along the height direction, the point cloud of the region of interest is divided into multiple point cloud cells at preset intervals; The reference width of the container to be grabbed is determined based on the point cloud in at least one point cloud cell with the lowest height position. And determine the width difference based on the difference between the width of the point cloud in the point cloud cell where the box top height is located and the width of the point cloud in the adjacent upper point cloud cell in the horizontal direction and the reference width; Based on the maximum height of the point cloud cell containing the container top height and at least one adjacent point cloud cell above it, determine the height difference used to characterize the distance between the lower edge of the spreader and the upper edge of the container top to be grabbed. Based on the width difference and the height difference, it is determined whether the spreader and the container to be grabbed are in a docking state.

2. The method according to claim 1, characterized in that, After acquiring the current frame's 3D point cloud data covering the area below the spreader and above the trailer, the method further includes: Based on the current frame's 3D point cloud data, determine the height of the spreader, the yaw angle of the trailer, and whether there is a container to be grabbed currently on the trailer; When a container to be grabbed is placed on the trailer, the top height of the container to be grabbed is determined based on the current frame's 3D point cloud data; When the height of the lifting device is less than the first preset height, the yaw angle is less than the preset angle, and the height of the top of the container is determined, it indicates that the preset preconditions for container placement detection have been met.

3. The method according to claim 1, characterized in that, Determining the reference width of the container to be grabbed based on the point cloud within at least one point cloud cell with the lowest height position includes: Select the two non-empty point cloud cells with the lowest height from a set of point cloud cells; Calculate the width of the point cloud in the horizontal direction within each non-empty point cloud cell, and use the average of the two widths as the baseline width.

4. The method according to claim 1, characterized in that, The step of determining the width difference based on the difference between the width of the point cloud in the point cloud cell at the height of the box top and the width of the point cloud cell above it in the horizontal direction and the reference width includes: For the point cloud cell containing the height of the box top and its adjacent upper point cloud cells, the point cloud cells that are more than a third preset distance from the center line in the horizontal direction are determined as outer edge points; Calculate the first width of the outer edge point in the horizontal direction of the point cloud cell containing the box top height, and the second width of the outer edge point in the horizontal direction of the adjacent upper point cloud cell; The maximum of the difference between the first width and the reference width, and the difference between the second width and the reference width, is taken as the width difference.

5. The method according to claim 1, characterized in that, The step of determining the height difference used to characterize the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height of the point cloud cell containing the container top height and at least one adjacent point cloud cell above it, includes: For the point cloud cell at the height of the box top and its adjacent first and second point cloud cells above it, the point cloud cells that are less than a fourth preset distance from the center line in the horizontal direction are determined as inner cavity points; wherein, among the adjacent first and second point cloud cells, the second point cloud cell is located above the first point cloud cell; Obtain the maximum height of the inner cavity point in the point cloud cell containing the top height of the box; Obtain the maximum and minimum heights of the inner cavity points in the first point cloud cell; Obtain the minimum height of the inner cavity point in the second point cloud cell; Arrange the maximum height of the point cloud cell containing the top height of the box, the minimum height of the first point cloud cell, the maximum height of the second point cloud cell, and the minimum height of the second point cloud cell in order from low to high to construct a height sequence. Find the first maximum height value in the height sequence whose difference from the top height of the box is greater than a preset threshold, and use the difference between the maximum height value and the top height of the box as the height difference.

6. The method according to claim 1, characterized in that, The step of determining whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference includes: When the width difference is greater than a preset width threshold and the height difference is less than a first preset height threshold, or when the height difference is less than a second preset height threshold, it indicates that the spreader and the container to be grabbed are in a docking state; wherein, the first preset height threshold is greater than the second preset height threshold.

7. The method according to claim 1, characterized in that, The step of determining whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference includes: The width difference and the height difference are mapped to width confidence and height confidence using the following formulas: in, For width confidence; High confidence level; The width difference; The width difference; The preset width threshold; The first preset height threshold; and The preset gain coefficient; The confidence level of a strongly boxed item is calculated using the following formula: in, To strengthen the confidence level of a single item in the box; The preset gain coefficient; The second preset height threshold is set, and the first preset height threshold is greater than the second preset height threshold. The total confidence level is calculated using the following formula. : When the total confidence level is greater than a preset probability threshold, it is determined that the spreader and the container to be grabbed are in a docking state.

8. A container spreader landing detection device, characterized in that, include: The acquisition module is used to acquire current frame 3D point cloud data covering the area below the spreader and above the trailer; the area above the trailer is used to place the container to be grabbed. The filtering module is used to filter out the three-dimensional point cloud data of the current frame when it is determined that the preset preconditions for container grabbing detection are met. The three-dimensional point cloud data is selected as the region of interest point cloud based on the preset position of the front surface of the container to be grabbed as a reference, which is extended forward and backward by a first preset distance in the driving direction, and extended to the left and right sides by a second preset distance in the horizontal direction perpendicular to the driving direction based on the center line of the trailer, and the height is within a preset height range. The segmentation module is used to divide the region of interest point cloud into multiple point cloud cells along the height direction at preset intervals; The first determining module is used to determine the reference width of the container to be grabbed based on the point cloud in at least one point cloud cell with the lowest height position. And determine the width difference based on the difference between the width of the point cloud in the point cloud cell where the box top height is located and the width of the point cloud in the adjacent upper point cloud cell in the horizontal direction and the reference width; The second determining module is used to determine the height difference that characterizes the distance between the lower edge of the spreader and the upper edge of the top of the container to be grabbed, based on the maximum height of the point cloud cell where the top height of the container is located and at least one adjacent point cloud cell above it. The judgment module is used to determine whether the spreader and the container to be grabbed are in a docking state based on the width difference and the height difference.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the memory via the bus, and the machine-readable instructions, when executed by the processor, perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 7.

Images