Intelligent container truck guiding method and system based on three-dimensional laser radar technology

Abstract

The embodiment of the invention provides an intelligent container truck guiding method and system based on a three-dimensional laser radar technology, and the method comprises the steps: scanning an operation lane in real time through a three-dimensional laser radar sensor, obtaining the three-dimensional contour data of a truck and a container of the truck, and calculating the deviation value of the truck and a preset alignment point. According to the deviation value of the truck and a preset alignment point, calculating an optimal adjustment path from the truck to the alignment point, generating a guide instruction and sending the guide instruction to the guide indication equipment; the guide indication equipment guides a driver to adjust the position of the truck through a visual and / or sound signal based on the guide instruction until a preset alignment point is reached; and after confirming that a preset alignment point is reached, notifying a wharf central management system through a communication interface to carry out the next operation.

Description

Technical Field

[0001] This document relates to the field of logistics automation technology, and particularly to an intelligent container truck guiding method and system based on three-dimensional lidar technology. Background Art

[0002] Traditional container terminal operations rely on manual visual judgment and manual operations, which are not only inefficient but also prone to positioning deviations, affecting the overall operation efficiency and safety. In recent years, although there have been various truck guiding systems applying technologies such as computer vision and laser scanning, most systems have problems such as being sensitive to environmental light, limited recognition accuracy, or poor adaptability. In terms of three-dimensional truck positioning, currently, at home and abroad, the main method is to use a 2D laser scanner and a rotating cloud platform. This method requires waiting for the swinging cloud platform to complete the swing, then performing three-dimensional truck scanning and then completing the recognition algorithm, and its real-time performance cannot meet the usage requirements of automated ports. Therefore, it is particularly important to develop a container truck guiding system that can detect all-weather, with high precision and in real time, and to develop a container truck guiding system that can work all-weather and with high precision. Summary of the Invention

[0003] One or more embodiments of this specification provide an intelligent container truck guiding method based on three-dimensional lidar technology, including:

[0004] Real-time scanning of the operation lane by a three-dimensional lidar sensor to obtain three-dimensional contour data of the truck and its container, and calculating the deviation amount between the truck and a preset alignment point;

[0005] According to the deviation amount between the truck and the preset alignment point, calculating the optimal adjustment path of the truck to the alignment point, and generating a guiding instruction to send to the guiding indication device;

[0006] The guiding indication device guides the driver to adjust the position of the truck through visual and / or sound signals based on the guiding instruction until the preset alignment point is reached;

[0007] After confirming that the preset alignment point is reached, notify the terminal central management system through the communication interface for the next operation.

[0008] Further, the three-dimensional lidar sensor is installed in the key operation area of the terminal;

[0009] The specific method for real-time scanning of the operation lane by a three-dimensional lidar sensor to obtain three-dimensional contour data of the truck and its container is:

[0010] The operation lane is scanned in real time by a 3D lidar sensor to obtain 3D point cloud data of the truck and its container under the crane. According to the pre-set calibration parameters, the coordinate system of the 3D point cloud data is adjusted: the ground is taken as the xy plane, the direction perpendicular to the ground towards the sky is taken as the z axis, and the origin of the coordinate system is set at the mid-position of the container truck lane.

[0011] According to the range of the operation lane, ROI filtering is performed on the 3D point cloud data to retain the 3D point cloud data within the operation lane.

[0012] Voxel filtering is used to downsample the retained 3D point cloud data.

[0013] After the downsampling process is completed, the ground point cloud data within the range of z = 0 ± 5 cm is eliminated to obtain the 3D contour data of the truck and the container it loads.

[0014] Further, the specific method of using voxel filtering to downsample the retained 3D point cloud data is as follows:

[0015] The point cloud space is divided into multiple small 3D volume units (voxels), and then the points within each voxel are merged or a representative point is selected:

[0016] First, determine the side length of each voxel, denoted as v x , v y , v z , and distinguish the corresponding x, y, and z axis directions of the 3D space.

[0017] For each point P(xi, yi, zi) in the point cloud, calculate the voxel index it belongs to, specifically obtained by dividing the coordinates of the point by the voxel size and rounding down, as follows:

[0018]

[0019] Use the calculated index I x , I y , I z to identify the voxel to which each point belongs, and use a hash table data structure to store the set of points within each voxel.

[0020] For each non-empty voxel, perform the operation of point aggregation or selection of a representative point;

[0021] Combine the representative points of each voxel or the points after aggregation processing to form the downsampled 3D point cloud data.

[0022] Further, for each non-empty voxel, the selection of a representative point specifically includes:

[0023] Take the geometric center of all points within the voxel as the representative point of the voxel;

[0024] Calculate the average coordinate value of all points within the voxel as the representative point;

[0025] Select the point closest to the voxel center as the representative point.

[0026] Furthermore, the cycle time is 100 milliseconds.

[0027] Furthermore, perform principal component analysis dimensionality reduction on the points after the aggregation process. Specifically:

[0028] First, perform centering processing on all point cloud data. Subtract the mean vector of the data set from the position of each point cloud data point, so that the centroid of the point cloud data set is located at the origin, eliminating the translation effect:

[0029] p’ i =p i -p′;

[0030] where p i is the original data point, p i =(x i ,y i ,z i ), p′ is the average position vector of all points, and p’ i is the data point after centering, p i ′=(x i ′,y i ′,z i ′);

[0031] Construct a covariance matrix to describe the variance and covariance of the centered points in three dimensions. Its calculation formula is:

[0032]

[0033] where C jk represents the element in the j-th row and k-th column of the covariance matrix, p′ ij and p′ ik are the coordinate values of the centered point i in the j-th and k-th dimensions respectively and are the average values of all points in the j-th and k-th dimensions.

[0034] Perform eigenvalue decomposition on the covariance matrix to obtain a set of eigenvectors v1, v2, …, v n and the corresponding eigenvalues λ1, λ2, …, λ n , and the eigenvalues are arranged in descending order. The largest eigenvalue corresponds to the first principal component direction, the second largest corresponds to the second principal component, and so on, to obtain

[0035] Project the point cloud data onto the first few principal components to achieve data dimensionality reduction and obtain a two-dimensional point dataset; the points projected onto the first k principal components The specific calculation method is as follows:

[0036]

[0037] Furthermore, in the two-dimensional point cloud dataset, judge the shape of each aggregated point cloud, and obtain the truck head point cloud and the container point cloud through shape comparison.

[0038] One or more embodiments of this specification provide an intelligent container truck guiding system based on three-dimensional lidar technology, including: a three-dimensional lidar sensor, a data processing unit, a guiding indication device, and a communication interface;

[0039] The three-dimensional lidar sensor is used to scan the operation lane in real time, obtain the three-dimensional contour data of the truck and its container, and calculate the deviation amount between the truck and the preset alignment point;

[0040] The data processing unit is used to calculate the optimal adjustment path of the truck to the alignment point according to the deviation amount between the truck and the preset alignment point, and generate a guiding instruction to send to the guiding indication device;

[0041] The guiding indication device is used to guide the driver to adjust the position of the truck based on the guiding instruction through visual and / or sound signals until the preset alignment point is reached;

[0042] The communication interface is connected to the terminal management system, and is used to notify the terminal central management system to perform the next operation after confirming that the preset alignment point has been reached.

[0043] One or more embodiments of this specification provide an electronic device, including a processor; and,

[0044] A memory arranged to store computer-executable instructions, the computer-executable instructions, when executed, cause the processor to implement the steps of the above-mentioned intelligent container truck guiding method based on three-dimensional lidar technology.

[0045] One or more embodiments of this specification provide a storage medium for storing computer-executable instructions, the computer-executable instructions, when executed, implement the steps of the above-mentioned intelligent container truck guiding method based on three-dimensional lidar technology.

[0046] Adopting the embodiments of the present invention improves the automation level and operation efficiency of container loading and unloading operations, reduces the operation error rate, and provides key technical support for the realization of an intelligent terminal.

[0047] The above description is only an overview of the technical solution of the present invention. In order to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other objects, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically exemplified below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0049] Figure 1 A flowchart of an intelligent container truck guiding method based on three-dimensional lidar technology provided for one or more embodiments of this specification;

[0050] Figure 2 A schematic diagram of the composition of an intelligent container truck guiding system based on three-dimensional lidar technology provided for one or more embodiments of this specification;

[0051] Figure 3 A schematic diagram of the structure of an electronic device provided for one or more embodiments of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0052] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0053] Method Embodiment

[0054] According to an embodiment of the present invention, an intelligent container truck guiding method based on three-dimensional lidar technology is provided. Figure 1 A flowchart of an intelligent container truck guiding method based on three-dimensional lidar technology provided for one or more embodiments of this specification, as Figure 1 shown, the intelligent container truck guiding method based on three-dimensional lidar technology according to the embodiment of the present invention specifically includes:

[0055] S1. Real - time scan the operation lane through a 3D lidar sensor, obtain the 3D contour data of the truck and its container, and calculate the deviation amount between the truck and the preset alignment point.

[0056] The 3D lidar sensor is installed in the key operation area of the wharf to perform full - range and non - contact 3D scanning on the incoming container trucks;

[0057] The specific method for obtaining the 3D contour data of the truck and its container by real - time scanning the operation lane through a 3D lidar sensor is as follows:

[0058] The 3D lidar sensor scans the operation lane at regular intervals. Obtain the 3D point cloud data of the truck and its container under the crane. According to the pre - set calibration parameters, adjust the coordinate system of the 3D point cloud data: take the ground as the xy plane, the direction perpendicular to the ground towards the sky as the z - axis, and set the origin of the coordinate system as the mid - position of the container truck lane. In this embodiment, the operation lane is scanned at a frequency of 10 hz, that is, the cycle time is 100 milliseconds.

[0059] According to the range of the operation lane, perform ROI filtering on the 3D point cloud data to retain the 3D point cloud data within the operation lane range;

[0060] Use voxel filtering to perform downsampling on the retained 3D point cloud data;

[0061] After the downsampling process, eliminate the ground point cloud data within the range of z = 0 ± 5 cm to obtain the 3D contour data of the truck and its loaded container.

[0062] The specific method for using voxel filtering to perform downsampling on the retained 3D point cloud data is as follows:

[0063] Divide the point cloud space into multiple small 3D volume units (voxels), and then merge or select representative points for the points within each voxel:

[0064] First, determine the side length of each voxel, denoted as v x ,v y ,v z , and distinguish the corresponding x, y, z - axis directions in 3D space;

[0065] For each point P(xi, yi, zi) in the point cloud, calculate the voxel index it belongs to, specifically obtained by dividing the coordinates of the point by the voxel size and taking the floor value, as follows:

[0066]

[0067] Use the calculated index I x ,I y ,I zIdentify the voxel to which each point belongs, and use a hash table data structure to store the set of points within each voxel.

[0068] For each non-empty voxel, perform point aggregation or select a representative point; specifically, selecting a representative point includes:

[0069] Take the geometric center of all points within the voxel as the representative point of the voxel;

[0070] Calculate the average coordinate value of all points within the voxel as the representative point;

[0071] Select the point closest to the voxel center as the representative point.

[0072] Combine the representative points of each voxel or the points after aggregation processing to form the downsampled three-dimensional point cloud data.

[0073] Perform principal component analysis dimensionality reduction on the points after the aggregation processing. Specifically:

[0074] First, perform centering processing on all point cloud data. Subtract the mean vector of the dataset from the position of each point cloud data point so that the centroid of the point cloud dataset is at the origin, eliminating the translation effect:

[0075] p’ i = p i - p′;

[0076] where p i is the original data point, p i = (x i , y i , z i ), p′ is the average position vector of all points, and p’ i is the data point after centering, p i ′ = (x i ′, y i ′, z i ′);

[0077] Construct a covariance matrix to describe the variance and covariance of the centered points in three dimensions. Its calculation formula is:

[0078]

[0079] where C jk represents the element in the j-th row and k-th column of the covariance matrix, and p′ i ′ and p′ ik are the coordinate values of the centered point i in the j-th and k-th dimensions respectively and are the average values of all points in the j-th and k-th dimensions.

[0080] Perform eigenvalue decomposition on the covariance matrix to obtain a set of eigenvectors v1, v2, …, v n and the corresponding eigenvalues λ1, λ2, …, λ n , where the eigenvalues are arranged in descending order. The largest eigenvalue corresponds to the direction of the first principal component, the second largest corresponds to the second principal component, and so on, to obtain

[0081] Project the point cloud data onto the first few principal components to achieve data dimensionality reduction and obtain a two-dimensional point dataset; the points projected onto the first k principal components The specific calculation method is as follows:

[0082]

[0083] In the two-dimensional point cloud dataset, judge the shape of each aggregated point cloud, and obtain the truck head point cloud and the container point cloud through shape comparison.

[0084] S2. According to the deviation amount between the truck and the preset alignment point, calculate the optimal adjustment path of the truck to the alignment point, and generate a guiding instruction to send to the guiding indication device.

[0085] S3. The guiding indication device guides the driver to adjust the position of the truck based on the guiding instruction through visual and / or sound signals until the preset alignment point is reached.

[0086] Specifically, the guiding indication device includes an LED display screen, which displays the guiding instruction or guiding route through the LED display screen, provides a sound prompt for the truck driving route, or provides intuitive guiding information to the driver through an in-vehicle terminal, etc.

[0087] S4. After confirming that the preset alignment point is reached, notify the terminal central management system through the communication interface to perform the next operation.

[0088] After confirming that the truck position alignment is successful, perform real-time communication with the terminal central management system through the communication interface, synchronize information such as the operation progress and vehicle status, and realize the intelligent management of operation scheduling.

[0089] The beneficial effects of the present invention are as follows:

[0090] By adopting the embodiment of the present invention, the automation level and operation efficiency of container loading and unloading operations are improved, the operation error rate is reduced, and key technical support is provided for realizing an intelligent terminal.

[0091] System embodiment

[0092] According to an embodiment of the present invention, an intelligent container truck guiding system based on three-dimensional lidar technology is provided, Figure 2A schematic diagram of the composition of an intelligent container truck guidance system provided for one or more embodiments of this specification, as Figure 2 shown. The intelligent container truck guidance system based on three-dimensional lidar technology according to an embodiment of the present invention specifically includes: a three-dimensional lidar sensor 20, a data processing unit 22, a guidance indication device 24, and a communication interface 26;

[0093] The three-dimensional lidar sensor 20 is used to scan the operation lane in real time, obtain the three-dimensional contour data of the truck and its container, and calculate the deviation amount between the truck and the preset alignment point;

[0094] The data processing unit 22 is used to calculate the optimal adjustment path of the truck to the alignment point according to the deviation amount between the truck and the preset alignment point, and generate a guidance instruction to send to the guidance indication device;

[0095] The guidance indication device 24 is used to guide the driver to adjust the position of the truck based on the guidance instruction through visual and / or sound signals until the preset alignment point is reached;

[0096] The communication interface 26 is connected to the terminal management system, and is used to notify the terminal central management system to perform the next operation after confirming that the preset alignment point is reached.

[0097] The embodiment of the present invention is a system embodiment corresponding to the above method embodiment. The specific operations of each module can be understood with reference to the description of the method embodiment, and will not be elaborated here.

[0098] Device Embodiment 1

[0099] An embodiment of the present invention provides an electronic device, as Figure 3 shown, including: a memory 30, a processor 32, and a computer program stored on the memory 30 and executable on the processor 32. When the computer program is executed by the processor 32, the following method steps are implemented:

[0100] S1. Scan the operation lane in real time through a three-dimensional lidar sensor, obtain the three-dimensional contour data of the truck and its container, and calculate the deviation amount between the truck and the preset alignment point.

[0101] S2. Calculate the optimal adjustment path of the truck to the alignment point according to the deviation amount between the truck and the preset alignment point, and generate a guidance instruction to send to the guidance indication device;

[0102] S3. The guidance indication device guides the driver to adjust the position of the truck based on the guidance instruction through visual and / or sound signals until the preset alignment point is reached;

[0103] S4. After confirming reaching the preset alignment point, notify the terminal central management system through the communication interface to perform the next operation.

[0104] Second Embodiment of the Device

[0105] An embodiment of the present invention provides a computer-readable storage medium, on which an implementation program for information transmission is stored. When the program is executed by the processor 32, the following method steps are implemented:

[0106] S1. Real-time scan the operation lane through a 3D lidar sensor to obtain the 3D contour data of the truck and its container, and calculate the deviation amount between the truck and the preset alignment point.

[0107] S2. According to the deviation amount between the truck and the preset alignment point, calculate the optimal adjustment path from the truck to the alignment point, and generate a guiding instruction to send to the guiding indication device;

[0108] S3. The guiding indication device guides the driver to adjust the position of the truck based on the guiding instruction through visual and / or sound signals until reaching the preset alignment point;

[0109] S4. After confirming reaching the preset alignment point, notify the terminal central management system through the communication interface to perform the next operation.

[0110] The computer-readable storage medium described in this embodiment includes but is not limited to: ROM, RAM, magnetic disk or optical disc, etc.

[0111] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent container truck guiding method based on three-dimensional lidar technology, characterized in that, Including: Real-time scanning of the operation lane through a 3D lidar sensor to obtain the 3D contour data of the truck and its container, and calculating the deviation amount between the truck and the preset alignment point; Calculating the optimal adjustment path from the truck to the alignment point according to the deviation amount between the truck and the preset alignment point, and generating a guiding instruction to send to the guiding indication device; Based on the guiding instruction, the guiding indication device guides the driver to adjust the position of the truck through visual and / or sound signals until the preset alignment point is reached; After confirming reaching the preset alignment point, notify the terminal central management system through the communication interface to perform the next operation.

2. The method according to claim 1, characterized in that, The 3D lidar sensor is installed in the key operation area of the terminal; The specific method for obtaining the 3D contour data of the truck and its container by real-time scanning of the operation lane through the 3D lidar sensor is as follows: Real-time scanning of the operation lane through the 3D lidar sensor to obtain the 3D point cloud data of the truck and its container under the crane. According to the pre-set calibration parameters, adjust the coordinate system of the 3D point cloud data: taking the ground as the xy plane, the direction perpendicular to the ground towards the sky as the z axis, and setting the origin of the coordinate system at the middle position of the container truck lane; According to the range of the operation lane, perform ROI filtering on the 3D point cloud data to retain the 3D point cloud data within the operation lane; Performing downsampling processing on the retained 3D point cloud data using voxel filtering; After the downsampling processing is completed, eliminate the ground point cloud data within the range of z axis of 0 ± 5 cm to obtain the 3D contour data of the truck and its loaded container.

3. The method according to claim 2, wherein The specific method for performing downsampling processing on the retained 3D point cloud data using voxel filtering is as follows: Dividing the point cloud space into multiple small 3D volume units (voxels), and then merging or selecting representative points for the points within each voxel: First, determine the side length of each voxel, denoted as v x , v y , v z , and distinguish the corresponding directions of the x, y, and z axes in three-dimensional space; For each point P(xi, yi, zi) in the point cloud, calculate the voxel index it belongs to, specifically obtained by dividing the coordinates of the point by the voxel size and rounding down, as follows: Using the calculated index I x , I y , I z Identify the voxel to which each point belongs, and use a hash table data structure to store the set of points within each voxel. For each non-empty voxel, perform the operation of point aggregation or selecting representative points; Combining the representative points of each voxel or the points after aggregation processing to form the downsampled 3D point cloud data.

4. The method according to claim 3, wherein For each non-empty voxel, the specific method of selecting representative points includes: Taking the geometric center of all points within the voxel as the representative point of the voxel; Calculating the coordinate average value of all points within the voxel as the representative point; Selecting the point closest to the voxel center as the representative point.

5. The method according to claim 2, wherein The cycle time is 100 milliseconds.

6. The method according to claim 3, characterized in that Performing principal component analysis dimensionality reduction on the points after the aggregation processing, specifically: First, perform centering processing on all point cloud data, subtracting the mean vector of the dataset from the position of each point cloud data point, so that the centroid of the point cloud dataset is located at the origin, eliminating the translation influence: p’ i = p i - p′; Among them, p i is the original data point, p i =(x i , y i , z i ), p' is the average position vector of all points, p' i is the data point after centering, p i '=(x i ', y i ', z i '); Constructing a covariance matrix to describe the variance and covariance of the centered points in three dimensions, and its calculation formula is: Among them, C jk represents the element in the j-th row and k-th column of the covariance matrix, p′ ij and p′ ik are the coordinate values of the centered point i in the j-th and k-th dimensions respectively and are the average values of all points in the j-th and k-th dimensions. Perform eigenvalue decomposition on the covariance matrix to obtain a set of eigenvectors v1, v2, …, v n and the corresponding eigenvalues λ1, λ2, …, λ n , where the eigenvalues are arranged in descending order. The largest eigenvalue corresponds to the direction of the first principal component, the second largest corresponds to the second principal component, and so on, resulting in C vi = λ i v i ; Project the point cloud data onto the first few principal components to achieve data dimensionality reduction and obtain a two-dimensional point dataset; the points projected onto the first k principal components The specific calculation method is as follows:

7. The method according to claim 6, characterized in that In the two-dimensional point cloud dataset, judge the shape of each aggregated point cloud, and obtain the head point cloud and the container point cloud through shape comparison.

8. An intelligent container truck guidance system based on three-dimensional lidar technology, characterized in that, Including: 3D lidar sensor, data processing unit, guiding indication device, and communication interface; The three-dimensional lidar sensor is used to scan the operating lane in real time, obtain the three-dimensional contour data of the truck and its container, and calculate the deviation amount between the truck and the preset alignment point; The data processing unit is used to calculate the optimal adjustment path of the truck to the alignment point according to the deviation amount between the truck and the preset alignment point, and generate a guiding instruction to send to the guiding indication device; The guiding indication device is used to guide the driver to adjust the position of the truck based on the guiding instruction through visual and / or sound signals until the preset alignment point is reached; The communication interface is connected to the terminal management system and is used to notify the terminal central management system to perform the next operation after confirming that the preset alignment point has been reached.

9. An electronic device, characterized in that, It includes: A processor; And, A memory arranged to store computer-executable instructions, and the computer-executable instructions, when executed, cause the processor to implement the steps of the intelligent container truck guiding method based on three-dimensional lidar technology according to any one of claims 1 to 7.

10. A storage medium, characterized in that, For storing computer-executable instructions, and the computer-executable instructions, when executed, implement the steps of the intelligent container truck guiding method based on three-dimensional lidar technology according to any one of claims 1 to 7.

Images