A container truck automatic loading and unloading box control method, system, device and medium

Abstract

The application provides a container truck automatic loading and unloading box control method, system, equipment and medium, wherein the method comprises the following steps: synchronously collecting three-dimensional point cloud and image data of the container truck through fixedly installed sensors and cameras; fusing the three-dimensional point cloud and the image data to perform target identification and lock head positioning; converting the identification and positioning results into a world coordinate system and generating a spreader motion control instruction to send to a crane; and the PLC of the crane controls the spreader to move to a target position to execute an end alignment program, so as to solve the problems of low efficiency and safety risks of the container truck automatic loading and unloading box operation.

Description

Technical Field

[0001] This invention relates to the fields of automation control and robotics, and in particular to a method, system, equipment and medium for automatic loading and unloading control of container trucks. Background Technology

[0002] Container shipping plays a crucial role in global logistics and trade. Traditional container loading and unloading operations primarily rely on operators of large lifting equipment (such as quay cranes and yard cranes) who visually observe and manually guide the spreader to align with the lock holes on the container or the locks on the truck. This method has the following significant drawbacks: Low operational efficiency: The manual alignment process is time-consuming and relies entirely on the driver's experience and skill. Alignment is even more difficult under adverse weather conditions such as night, rain, snow, and fog, which seriously affects the overall loading and unloading efficiency.

[0003] High safety risks: Manual operation carries the risk of blind spots and misjudgment, which may lead to collisions between the spreader and the vehicle, container or personnel on the ground, causing equipment damage and personal injury accidents.

[0004] High labor costs: The crane requires professional and experienced crane operators, resulting in high labor costs. In addition, the crane faces the problems of an aging workforce and difficulties in recruitment.

[0005] Low level of automation: It is difficult to adapt to the construction requirements of modern and automated terminals, becoming a bottleneck restricting the improvement of the overall automation level of ports.

[0006] To address these issues, the industry has proposed several automation solutions. For example, sensors (cameras, LiDAR) can be mounted on a lifting device and moved with it to identify targets. However, this "follow-up" approach has inherent drawbacks: the sensors operate in harsh environments and are susceptible to shaking and vibration from the lifting device, leading to unstable data and decreased accuracy; furthermore, the sensor wiring is complex and maintenance is difficult.

[0007] Therefore, there is an urgent need to propose a control method, system, equipment, and medium for automated container truck loading and unloading to solve the technical problems of low efficiency and safety risks in automated container truck loading and unloading operations. Summary of the Invention

[0008] To overcome the problems existing in the related technologies, this disclosure provides a method, system, equipment and medium for controlling the automatic loading and unloading of containers by trucks, so as to solve the technical problems of low efficiency and safety risks in the automatic loading and unloading of containers by trucks in the related technologies.

[0009] This specification provides one or more embodiments of an automated container truck loading and unloading control method, including the following steps: The three-dimensional point cloud and image data of the container truck are collected synchronously by fixedly installed sensors and cameras; The target recognition and lock-on localization are performed by fusing the 3D point cloud and the image data; The identification and positioning results are converted into a world coordinate system and the resulting spreader motion control commands are sent to the crane. The crane's PLC controls the lifting device to move to the target position and executes the end-effector alignment program.

[0010] Preferably, the method further includes the following steps: Confirm changes in the number or height of containers using LiDAR scanning; Verify the job completion status by combining the feedback signals from the PLC.

[0011] Preferably, the process of fusing the 3D point cloud and the image data for target recognition and lock-on localization specifically includes the following steps: Using calibration parameters, the three-dimensional point cloud is projected onto a two-dimensional image plane to achieve point cloud coloring; A deep learning object detection algorithm was used to identify key features of containers and locks in images. The target and lock head were segmented and located based on point cloud data and image recognition results.

[0012] Preferably, the step of segmenting and locating the lock head based on point cloud data and image recognition results specifically includes the following steps: Create a 3D point cloud template for a standard lock head; The matching target point cloud clusters are obtained by using coarse localization of two-dimensional images and filtering of three-dimensional view frustums. The lock head is located by matching the three-dimensional point cloud template of the standard lock head with the target point cloud cluster using an iterative nearest point algorithm.

[0013] Preferably, the PLC control of the crane moves the lifting device to the target position to execute the end-effector alignment program, specifically including the following steps: The crane's PLC queries the current world coordinates of the lifting device; Obtain the deviation between the current world coordinates and the target position coordinates; The PLC controls the lifting device to move towards the target position coordinates until the norm of the deviation is less than a preset accuracy threshold.

[0014] This specification provides one or more embodiments of an automated container truck loading and unloading control system, including a data acquisition module, a data fusion module, a control command generation module, and a control module; The acquisition module is used to synchronously acquire three-dimensional point cloud and image data of container trucks through fixedly installed sensors and cameras; The data fusion module is used to fuse the 3D point cloud and the image data for target recognition and lock-on localization; The control command generation module is used to convert the identification and positioning results into a world coordinate system and generate a spreader motion control command to send to the crane; The control module is used by the PLC control system of the crane to move the lifting device to the target position and execute the end-effector alignment program.

[0015] Preferably, it also includes a verification module, configured as follows: Confirm changes in the number or height of containers using LiDAR scanning; Verify the job completion status by combining the feedback signals from the PLC.

[0016] Preferably, the data fusion module includes a coloring unit, a recognition unit, and a segmentation unit; The coloring unit is used to project the three-dimensional point cloud onto a two-dimensional image plane using calibration parameters to achieve point cloud coloring. The recognition unit is used to identify key features of containers and locks in images using a deep learning object detection algorithm; The segmentation unit is used to segment the target and lock head location based on point cloud data and image recognition results.

[0017] This specification provides one or more embodiments of a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the automatic container loading and unloading control method for container trucks as described above.

[0018] This specification provides one or more embodiments of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described automatic container truck loading and unloading control method.

[0019] The present disclosure provides an automatic container truck loading and unloading control method, system, equipment, and medium. Its advantages lie in the simultaneous acquisition of 3D point cloud and image data of the container truck through fixedly installed sensors and cameras. The 3D point cloud acquired by the sensors accurately obtains the spatial geometric information of the truck and container, while the image data acquired by the cameras provides rich texture, color, and other detailed information. The simultaneous acquisition of both ensures data consistency in the time dimension, providing complete and spatiotemporally matched raw data support for subsequent target recognition and positioning. This effectively avoids information deviations caused by data asynchrony and improves the quality of basic data for subsequent processing. Furthermore, the fusion of the 3D point cloud and image data for target recognition and lock positioning allows for precise determination of the container and lock's position and orientation based on the spatial geometric information of the 3D point cloud. The texture details of the image data help distinguish the target from the background and identify the lock's characteristic details, reducing recognition errors caused by factors such as lighting changes and occlusion. Through fusion processing, accurate identification of container targets and sub-millimeter-level precise positioning of lock heads can be achieved, providing reliable target coordinate information for subsequent spreader alignment. The identification and positioning results are converted into a world coordinate system and used to generate spreader motion control commands, which are then sent to the crane. This conversion ensures a unified spatial reference for data from different devices, guaranteeing the spatial reference consistency of the spreader motion control commands and avoiding motion deviations caused by inconsistencies in coordinate systems. The spreader motion control commands generated based on the unified coordinate system can precisely plan the spreader's motion path, including translation and rotation parameters. The accuracy and real-time performance of these commands ensure that the spreader quickly approaches the target position along the optimal path, reducing movement time and energy consumption, while also lowering the risk of collisions. The crane's PLC controls the spreader to move to the target position and executes the end-effector alignment program. As the core of industrial control, the PLC possesses high reliability and rapid response capabilities, accurately executing the spreader's movement control to ensure that the spreader stably reaches the target position along the preset path. The end-of-line alignment program achieves high-precision docking between the spreader and the container lock by precisely adjusting the attitude and position of the spreader. The docking error can be controlled within a very small range, effectively avoiding mechanical impact and damage during the docking process, improving the efficiency and safety of loading and unloading operations, and meeting the stringent requirements of automated container loading and unloading. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1A flowchart illustrating an automated container loading and unloading control method for container trucks provided for one or more embodiments of this specification; Figure 2 A schematic diagram of an automated container truck loading and unloading control system provided for one or more embodiments of this specification; Figure 3 This is a schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this invention.

[0023] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0024] Method Implementation Examples According to embodiments of the present invention, an automatic container loading and unloading control method for container trucks is provided, such as... Figure 1 The diagram shown is a flowchart illustrating the automated container truck loading and unloading control method provided in this embodiment. The automated container truck loading and unloading control method according to this embodiment includes the following steps: S110. The system synchronously acquires 3D point cloud and image data of container trucks through fixedly installed sensors and cameras. The sensors can be 3D LiDAR, which are fixedly installed on brackets under the crane trolley or on one or both sides of the truck lane. They are used to scan container trucks entering the work area and acquire their complete 3D point cloud data. The cameras are fixedly installed in conjunction with the 3D LiDAR to capture high-resolution color or grayscale images of the work area. When a container truck enters the scanning area of ​​the fixed sensors, the system automatically triggers the LiDAR and cameras to perform synchronous data acquisition.

[0025] The collected data is preprocessed, starting with point cloud denoising using the Statistical Outlier Removal method. For each point in the point cloud, the average distance to its k nearest neighbors is calculated. If this average distance is greater than a threshold determined by the global average distance and standard deviation, the point is considered noise and removed. The average distance of the k nearest neighbors is The global average distance is The standard deviation is The removal condition is: in This is an adjustable parameter.

[0026] Image distortion correction is performed on the data: To eliminate radial and tangential distortion caused by the camera lens, the following distortion model is used to adjust the original image coordinates. Correction is performed to obtain distortion-free coordinates. : ; in, , The radial distortion coefficient is... These are the tangential distortion coefficients, which are obtained through camera calibration.

[0027] S120: Select a high-performance embedded computer or industrial control computer as the "brain" of the system, responsible for running the core algorithm, fusing 3D point cloud and image data for target recognition and lock head localization. The core controller completes data fusion and target recognition within 0.5 seconds, and calculates the precise 3D coordinates of the four lock heads on the truck in the crane's world coordinate system. , ... S130. The identification and positioning results are converted into the world coordinate system and the motion control commands for the spreader are generated and packaged and sent to the crane through the communication interface.

[0028] S140. After receiving the target coordinates, the crane's PLC (Programmable Logic Controller) immediately queries the current world coordinates of the spreader, starts the trolley, hoisting mechanism, controls the spreader to move to the target position to execute the end alignment program, completes the precise alignment with the lock head, and lowers the container.

[0029] The method provided in this embodiment synchronously acquires 3D point cloud and image data of container trucks using fixedly installed sensors and cameras. The 3D point cloud acquired by the sensors can accurately obtain the spatial geometric information of the truck and container, while the image data acquired by the camera can provide rich texture, color and other detailed information. The synchronous acquisition of the two can ensure the consistency of data in the time dimension, providing complete and spatiotemporally matched raw data support for subsequent target recognition and positioning, effectively avoiding information deviation caused by data asynchrony, and improving the quality of basic data for subsequent processing. The method integrates 3D point cloud and image data for target recognition and lock positioning. The spatial geometric information of the 3D point cloud can accurately determine the position and orientation of the container and lock, while the texture details of the image data can help distinguish the target from the background and identify the feature details of the lock, reducing recognition errors caused by factors such as changes in lighting and occlusion. Through fusion processing, accurate identification of container targets and sub-millimeter-level precise positioning of lock heads can be achieved, providing reliable target coordinate information for subsequent spreader alignment. The identification and positioning results are converted into a world coordinate system and used to generate spreader motion control commands, which are then sent to the crane. This conversion ensures a unified spatial reference for data from different devices, guaranteeing the spatial reference consistency of the spreader motion control commands and avoiding motion deviations caused by inconsistencies in coordinate systems. The spreader motion control commands generated based on the unified coordinate system can precisely plan the spreader's motion path, including translation and rotation parameters. The accuracy and real-time performance of these commands ensure that the spreader quickly approaches the target position along the optimal path, reducing movement time and energy consumption, while also lowering the risk of collisions. The crane's PLC controls the spreader to move to the target position and executes the end-effector alignment program. As the core of industrial control, the PLC possesses high reliability and rapid response capabilities, accurately executing the spreader's movement control to ensure that the spreader stably reaches the target position along the preset path. The end-of-line alignment program achieves high-precision docking between the spreader and the container lock by precisely adjusting the attitude and position of the spreader. The docking error can be controlled within a very small range, effectively avoiding mechanical impact and damage during the docking process, improving the efficiency and safety of loading and unloading operations, and meeting the stringent requirements of automated container loading and unloading.

[0030] To implement the method proposed in this embodiment, four key coordinate systems are defined: the world coordinate system... 3D LiDAR coordinate system Camera coordinate system and image pixel coordinate system All algorithms revolve around the precise transformation of these coordinate systems.

[0031] Step 1: System Calibration and Coordinate System 1 (Completed Offline) Sensor joint calibration: accurately calibrating the relative positional relationship (external parameters) between a fixedly installed 3D LiDAR and a camera.

[0032] World coordinate system calibration: The crane's mechanical coordinate system is used as the global world coordinate system. Through a calibration program, the transformation matrix of the fixedly installed sensors relative to the world coordinate system is accurately calculated. This is a crucial step in this scheme, ensuring that the local coordinates of any point acquired by the sensors can be converted into world coordinates.

[0033] This conversion is achieved through a rigid body transformation matrix This matrix, obtained through offline calibration, is a core parameter for global localization. A point in the LiDAR coordinate system... Points transformed to world coordinate system The process is described by the following formula: ; ; in, For rotation matrix, It is a translation vector.

[0034] In one embodiment, the following steps are also included: The system uses LiDAR scanning to confirm changes in the number or height of containers (e.g., to determine the number or height of containers on a truck), and combines this with feedback signals from the PLC to verify the completion status of the operation. Once the operation is confirmed successful, the system clears the current target and prepares to receive the next truck.

[0035] The method provided in this embodiment confirms the number or height changes of containers through LiDAR scanning. With its high-resolution detection capability, it can accurately count the number of containers and monitor height changes in real time, providing reliable data support for controlling the progress of operations. Combined with the feedback signal from the PLC to verify the completion status of the operation, the real-time and accurate feedback signal can be used to objectively judge whether the operation meets the standards, and promptly identify and correct problems. Together, the two improve the level of automated supervision of operations and ensure the closed-loop reliability of the loading and unloading process.

[0036] In one embodiment, fusing 3D point cloud and image data for target recognition and lock localization specifically includes the following steps: Using calibration parameters, a 3D point cloud is projected onto a 2D image plane to achieve point cloud coloring. Specifically, firstly, LiDAR points are... Points transformed to camera coordinate system ( Then, using the camera intrinsic parameter matrix Project it onto pixel coordinates : ; Left vector u: The column coordinates (horizontal pixel position) of the spatial point in the image pixel coordinate system. v: The row coordinates (vertical pixel position) of the spatial point in the image pixel coordinate system. The depth value of a spatial point in the camera coordinate system (distance along the camera's optical axis). The camera intrinsic parameter matrix K is a 3×3 upper triangular matrix containing the camera's intrinsic parameters: The camera's focal length in the horizontal direction (unit: pixels), calculated by dividing the physical focal length f (unit: mm) by the horizontal pixel size. (Unit: mm / pixel) This is obtained, i.e.

[0037] The camera's focal length in the vertical direction (unit: pixels), similarly... , (Vertical size of pixels) The column coordinate (unit: pixels) of the origin (principal point) of the image coordinate system in the pixel coordinate system, that is, the horizontal pixel position of the intersection of the optical axis and the image plane. The row coordinates (unit: pixels) of the origin (principal point) of the image coordinate system in the pixel coordinate system, that is, the perpendicular pixel position of the intersection of the optical axis and the image plane. Spatial points in camera coordinate system : The X-axis coordinate of the spatial point in the camera coordinate system (horizontally to the right) : The Y-axis coordinate (vertically downward) of the spatial point in the camera coordinate system. : The Z-axis coordinate of the spatial point in the camera coordinate system (forward along the optical axis, consistent with the depth value). In the images, deep learning object detection algorithms (such as YOLO) are used to identify key features of containers and locks, such as truck and container outlines, and key features of container corner fittings or truck locks. In the 3D point cloud, a density-based spatial clustering algorithm (DBSCAN) is used to segment the point cloud clusters of trucks and containers.

[0038] DBSCAN defines a cluster using two parameters: neighborhood radius. and minimum points A point Defined as the core point, if its - The number of points in the neighborhood is greater than or equal to : By starting from the core point and continuously expanding its achievable neighborhood, a point cloud cluster with arbitrary shapes can eventually be formed.

[0039] Based on point cloud data and image recognition results, the locations of targets such as truck decks and container surfaces, as well as key parts such as locks / keyholes, are segmented.

[0040] The lock head is segmented and located based on point cloud data and image recognition results. A method based on 3D model matching is used, which includes the following steps: Create a 3D point cloud template for a standard lock head .

[0041] To obtain the target point cloud cluster for matching The matching target point cloud clusters are obtained by using coarse localization of two-dimensional images and filtering of three-dimensional view frustums. The specific steps are as follows: a) Coarse localization of two-dimensional images: First, the system uses a deep learning object detection model (such as YOLO) to analyze the high-definition image, quickly identify the position of each lock in the image, and generate a two-dimensional bounding box.

[0042] b) 3D Point Cloud Filtering: Subsequently, the system uses the calibrated extrinsic parameters between the camera and LiDAR to back-project the bounding box on the 2D image, constructing a frustum in 3D space. The system retains only the LiDAR point cloud that falls within this frustum, thus filtering the massive amount of raw point cloud data into a small-scale point cloud that is only related to a single lock.

[0043] c) Precise point cloud segmentation: Finally, the system runs a clustering algorithm (such as DBSCAN) on the filtered small-scale point cloud.

[0044] Because the lock head point cloud and the background point cloud of the truck bed are physically separated in space, this algorithm can effectively separate the "target point cloud cluster" representing the lock head itself. "Accurately segmented. Obtaining the target point cloud cluster." Then, the Iterative Closest Point (ICP) algorithm is used to accurately calculate the 3D pose of the lock. The goal of the ICP algorithm is to find an optimal rotation matrix. Translation vector The objective function is to minimize the distance error between the transformed template point cloud and the target point cloud.

[0045] in, In the target point cloud In and the transformed template points The nearest point. The algorithm continuously optimizes through iterative calculations. and Continue until the error converges or the maximum number of iterations is reached. When the optimal transformation matrix is ​​obtained... and Then, the precise pose of the lock head in the sensor coordinate system can be obtained. If the center point coordinates of the standard template are... The coordinates of the center point of the lock head are then identified. for: ; Most importantly, using the calibrated transformation matrix The calculated coordinates of the lock's center point By transforming the coordinates to the world coordinate system of the crane, its precise global three-dimensional coordinates can be obtained.

[0046] Based on the target point cloud cluster, an iterative nearest-point algorithm is used to match the 3D point cloud template of the standard lock head, thus obtaining the lock head localization. The target is no longer a relative pose, but an absolute world coordinate. For example, the target of a packing operation is the precise 3D coordinates of the four lock heads on the truck in the world coordinate system. .

[0047] The method provided in this embodiment utilizes calibration parameters to project a three-dimensional point cloud onto a two-dimensional image plane to achieve point cloud coloring, enabling the point cloud data to acquire the texture and color information of the image, thus enhancing the data's recognizability. A deep learning target detection algorithm is employed to identify key features of containers and locks in the image. Leveraging the algorithm's strong learning and generalization capabilities, it accurately captures target features, improving recognition accuracy. Furthermore, based on the point cloud data and image recognition results, the target and lock are segmented and located. By combining the spatial geometric information of the point cloud with the feature details of the image, high-precision positioning of the target and lock is achieved, providing accurate coordinates for subsequent spreader alignment and significantly improving the reliability and accuracy of target recognition and lock positioning.

[0048] In one embodiment, the crane's PLC controls the spreader to move to the target position to execute an end-effector alignment procedure, specifically including the following steps: The crane's PLC queries the current world coordinates of the lifting device. .

[0049] The core controller directly sends the world coordinates of the target to the crane PLC, and the crane PLC obtains the deviation vector between the current world coordinates and the target position coordinates. : The deviation vector As input to a crane's three-dimensional motion control system (such as a PID controller). In a digital control system, the output of the PID control law... The k-th sampling time can be represented as: in, The deviation vector at time k , These are the proportional, integral, and differential gain matrices, respectively.

[0050] PLC controller output Control the spreader to move towards the target position until the norm of the deviation vector is reached. Less than the preset accuracy threshold.

[0051] The method provided in this embodiment uses the crane's PLC to query the current world coordinates of the spreader, providing a precise initial position reference for subsequent actions; by obtaining the deviation between the current world coordinates and the target position coordinates, the direction and distance that the spreader needs to move can be determined; the PLC controls the spreader to move towards the target position coordinates until the norm of the deviation is less than a preset accuracy threshold, and through continuous deviation correction, it can ensure that the spreader finally reaches the target position accurately, meeting the high-precision requirements of end-point alignment and effectively improving the accuracy and reliability of loading and unloading operations.

[0052] System Implementation Examples According to embodiments of the present invention, an automatic container loading and unloading control system for container trucks is provided, such as... Figure 2 The diagram shown is a structural schematic of the automatic container truck loading and unloading control system provided in this embodiment. The automatic container truck loading and unloading control system according to this embodiment includes a data acquisition module 21, a data fusion module 22, a control command generation module 23, and a control module 24.

[0053] The acquisition module 21 is used to synchronously acquire 3D point cloud and image data of the container truck through fixedly installed sensors and cameras.

[0054] The data fusion module 22 is used to fuse 3D point cloud and image data for target recognition and lock-on localization.

[0055] The control command generation module 23 is used to convert the identification and positioning results into a world coordinate system and generate motion control commands for the spreader to be sent to the crane.

[0056] Control module 24 is used for the PLC control of the crane to move the lifting device to the target position and execute the end alignment program.

[0057] The system provided in this embodiment includes an acquisition module 21 for synchronously acquiring 3D point cloud and image data of a container truck using fixedly installed sensors and cameras. The 3D point cloud acquired by the sensors can accurately obtain the spatial geometric information of the truck and container, while the image data acquired by the camera can provide rich texture, color and other detailed information. The synchronous acquisition of both can ensure the consistency of data in the time dimension, providing complete and spatiotemporally matched raw data support for subsequent target recognition and positioning, effectively avoiding information deviation caused by data asynchrony, and improving the quality of basic data for subsequent processing. The data fusion module 22 is used to fuse 3D point cloud and image data for target recognition and lock positioning. The spatial geometric information of the 3D point cloud can accurately determine the position and orientation of the container and lock, while the texture details of the image data can help distinguish the target from the background and identify the feature details of the lock, reducing recognition errors caused by factors such as changes in lighting and occlusion. Through fusion processing, accurate identification of container targets and sub-millimeter-level precise positioning of lock heads can be achieved, providing reliable target coordinate information for subsequent spreader alignment. The control command generation module 23 is used to convert the identification and positioning results into a world coordinate system and generate spreader motion control commands to be sent to the crane. Converting the identification and positioning results into a world coordinate system can achieve a unified spatial reference for data from different devices, ensuring the spatial reference consistency of spreader motion control commands and avoiding motion deviations caused by inconsistencies in coordinate systems. The spreader motion control commands generated based on a unified coordinate system can accurately plan the spreader's motion path, including translation, rotation, and other motion parameters. The accuracy and real-time performance of the commands ensure that the spreader quickly approaches the target position along the optimal path, reducing motion time and energy consumption, while also reducing the risk of collisions. The control module 24 is used by the crane's PLC to control the spreader to move to the target position and execute the end-point alignment program. As the core of industrial control, the PLC has high reliability and fast response capabilities, and can accurately execute the spreader's movement control, ensuring that the spreader stably reaches the target position along the preset path. The end-of-line alignment program achieves high-precision docking between the spreader and the container lock by precisely adjusting the attitude and position of the spreader. The docking error can be controlled within a very small range, effectively avoiding mechanical impact and damage during the docking process, improving the efficiency and safety of loading and unloading operations, and meeting the stringent requirements of automated container loading and unloading.

[0058] In one embodiment, a verification module is also included, configured to: confirm changes in the number or height of containers through LiDAR scanning, and verify the completion status of the operation in conjunction with feedback signals from the PLC.

[0059] The system provided in this embodiment confirms the number or height changes of containers through LiDAR scanning. With its high-resolution detection capabilities, it can accurately count the number of containers and monitor height changes in real time, providing reliable data support for controlling the progress of operations. Combined with the feedback signals from the PLC to verify the completion status of operations, the real-time and accurate feedback signals can be used to objectively judge whether the operations meet the standards, and promptly identify and correct problems. Together, they improve the level of automated supervision of operations and ensure the closed-loop reliability of the loading and unloading process.

[0060] In one embodiment, the data fusion module 22 includes a coloring unit, a recognition unit, and a segmentation unit; The shading unit is used to project a 3D point cloud onto a 2D image plane using calibration parameters to achieve point cloud shading.

[0061] The recognition unit is used to identify key features of containers and locks in images using a deep learning object detection algorithm.

[0062] The segmentation unit is used to segment the target and lock head location based on point cloud data and image recognition results.

[0063] The system provided in this embodiment uses calibration parameters to project a 3D point cloud onto a 2D image plane to achieve point cloud coloring, allowing the point cloud data to obtain the texture and color information of the image, thus enhancing the data's recognizability. It employs a deep learning target detection algorithm to identify key features of containers and locks in the image. Leveraging the algorithm's strong learning and generalization capabilities, it accurately captures target features, improving recognition accuracy. Furthermore, based on the point cloud data and image recognition results, the system segments and locates the target and locks. By combining the spatial geometric information of the point cloud with the feature details of the image, it achieves high-precision positioning of the target and locks, providing accurate coordinates for subsequent spreader alignment and significantly improving the reliability and accuracy of target recognition and lock positioning.

[0064] The embodiments of the present invention are system embodiments corresponding to the above method embodiments. The specific operations of each module processing step can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0065] like Figure 3 As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the automatic container truck loading and unloading control method in the above embodiments, or when the computer program is executed by a processor, it implements the automatic container truck loading and unloading control method in the above embodiments.

[0066] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0067] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. Units described as separate components may or may not be physically separate. Components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention are known to those skilled in the art.

Claims

1. A method for controlling the automatic loading and unloading of containers on a container truck, characterized in that, Includes the following steps: The three-dimensional point cloud and image data of the container truck are collected synchronously by fixedly installed sensors and cameras, among which the sensors are three-dimensional LiDAR. The target recognition and lock-on localization are performed by fusing the 3D point cloud and the image data; The identification and positioning results are converted into a world coordinate system and the resulting spreader motion control commands are sent to the crane. The crane's PLC controls the spreader to move to the target position and executes the end-effector alignment program; It also includes the following steps: The number or height changes of containers can be confirmed by 3D LiDAR scanning; Verify the job completion status by combining feedback signals from the PLC; The process of fusing the 3D point cloud and the image data for target recognition and lock-on localization specifically includes the following steps: Using calibration parameters, the three-dimensional point cloud is projected onto a two-dimensional image plane to achieve point cloud coloring; A deep learning object detection algorithm was used to identify key features of containers and locks in images. The target and lock head were segmented and located based on point cloud data and image recognition results; The process of segmenting and locating the lock head based on point cloud data and image recognition results specifically includes the following steps: Create a 3D point cloud template for a standard lock head; The matching target point cloud clusters are obtained by using coarse localization of two-dimensional images and three-dimensional view frustum filtering. The specific steps are as follows: a) Coarse localization of two-dimensional images: Analyze high-resolution images using a deep learning object detection model to quickly identify the position of each lock in the image and generate a two-dimensional bounding box; b) 3D point cloud filtering: Using the calibrated extrinsic parameters between the camera and the 3D LiDAR, the 2D bounding box on the 2D image is back-projected to construct a view frustum in 3D space; c) Precise point cloud segmentation: running clustering algorithms on the filtered small-area point cloud; The lock head is located by matching the three-dimensional point cloud template of the standard lock head with the target point cloud cluster using an iterative nearest point algorithm.

2. The automatic loading and unloading control method for container trucks as described in claim 1, characterized in that, The PLC control of the crane moves the lifting device to the target position to execute the end-effector alignment program, specifically including the following steps: The crane's PLC queries the current world coordinates of the lifting device; Obtain the deviation between the current world coordinates and the target position coordinates; The PLC controls the lifting device to move towards the target position coordinates until the norm of the deviation is less than a preset accuracy threshold.

3. A container truck automatic loading and unloading control system applying the method as described in any one of claims 1-2, characterized in that, It includes an acquisition module, a data fusion module, a control command generation module, and a control module; The acquisition module is used to synchronously acquire three-dimensional point cloud and image data of container trucks through fixedly installed sensors and cameras; The data fusion module is used to fuse the 3D point cloud and the image data for target recognition and lock-on localization; The control command generation module is used to convert the identification and positioning results into a world coordinate system and generate a spreader motion control command to send to the crane; The control module is used for the crane's PLC control to move the spreader to the target position and execute the end-effector alignment program; It also includes a verification module, configured as follows: The number or height changes of containers can be confirmed by 3D LiDAR scanning; Verify the job completion status by combining the feedback signals from the PLC.

4. The container truck automatic loading and unloading control system as described in claim 3, characterized in that, The data fusion module includes a coloring unit, a recognition unit, and a segmentation unit; The coloring unit is used to project the three-dimensional point cloud onto a two-dimensional image plane using calibration parameters to achieve point cloud coloring. The recognition unit is used to identify key features of containers and locks in images using a deep learning object detection algorithm; The segmentation unit is used to segment the target and lock head location based on point cloud data and image recognition results.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the automatic container loading and unloading control method for container trucks as described in any one of claims 1 to 2.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the container truck automatic loading and unloading control method as described in any one of claims 1 to 2.

Images