Container positioning method, system, device and electronic equipment

Abstract

The present application is dedicated to providing a container positioning method, system, device and electronic device, which is used to provide a container positioning method that can completely determine the position of the container and the adjacent container. First, the container slot gap is obtained, and the point cloud data is segmented before processing. Each group of point cloud data obtained by segmentation roughly corresponds to a container edge. In the subsequent processing process, it is no longer necessary to cluster and fit the panoramic point cloud data with a large degree of discreteness. It is only necessary to group and process the groups of point cloud data with a relatively concentrated distribution in the segmented point cloud number, which improves the processing efficiency and enables the computer to complete the analysis of multiple container point cloud data within a specified time, so as to realize the positioning of more containers in the loading and unloading operation scene.

Description

Technical Field

[0001] The present application relates to the field of crane loading and unloading, and in particular to a container positioning method, system, device and electronic equipment. Background Art

[0002] One of the important tasks in port loading and unloading is to load and unload containers. Specifically, containers are transferred between container trucks or container storage areas through spreaders. In container loading and unloading, it is necessary to prevent "bowling" accidents. "Bowling" refers to an accident in which a container knocks down other containers due to a series of collisions or a single container rolling down. Because it is similar to the sport of bowling, such accidents are called "bowling" in port loading and unloading operations.

[0003] To prevent such accidents, the location of containers needs to be obtained in time during loading and unloading operations for accident warning. At present, the solution of manually determining the location of containers by visual means is limited by blind spots and language communication speed; and the solution through vision or radar limits the speed of generating container location diagrams due to limited system resources. In order to ensure that the generated diagram can keep up with the speed of loading and unloading operations, generally only the container undergoing loading and unloading operations is located and a diagram is provided, which makes it difficult to take into account other neighboring containers, so there is still a risk of accidents.

[0004] Therefore, in order to improve the accident warning mechanism, it is necessary to provide a container positioning method that can completely determine the positions of the own and neighboring containers. Summary of the invention

[0005] In view of this, the present application is dedicated to providing a container positioning method, system, device and electronic device, so as to provide a container positioning method capable of completely determining the positions of the own container and the adjacent containers.

[0006] In a first aspect, an embodiment of the present application provides a container positioning method, including:

[0007] Obtaining panoramic point cloud data, wherein the panoramic point cloud data is data collected by a laser radar during loading and unloading operations of the container;

[0008] Obtaining container slot gaps, segmenting and clustering the panoramic point cloud data based on the container slot gaps, and obtaining segmented point cloud data;

[0009] Plane feature fitting and coordinate calculation are performed on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene.

[0010] Optionally, after obtaining the panoramic point cloud data, obtaining the container slot clearance includes:

[0011] Acquire point cloud data within a preset range in the panoramic point cloud data as first point cloud data, perform clustering and edge feature fitting on the first point cloud data, and acquire edge point cloud data;

[0012] Performing line feature fitting on the edge point cloud data to obtain the edge line of the container and the edge line of the adjacent container;

[0013] The container slot gap is calculated according to the own container edge line and the adjacent container edge line.

[0014] Optionally, performing line feature fitting on the edge point cloud data to obtain the edge line of the container and the edge line of the adjacent container includes:

[0015] Selecting point cloud data from the edge point cloud data to form a point cloud variance matrix;

[0016] Calculating a first eigenvalue along a first coordinate axis, a second eigenvalue along a second coordinate axis, and a third eigenvalue along a third coordinate axis of the point cloud variance matrix;

[0017] When the first eigenvalue is greater than both the second eigenvalue and the third eigenvalue, the point cloud data corresponding to the point cloud variance matrix is ​​fitted as the Benbei container edge line and the adjacent container edge line.

[0018] Optionally, obtaining panoramic point cloud data includes:

[0019] Acquire multiple groups of discrete point cloud data collected by the laser radar;

[0020] The discrete point cloud data are converted to the same coordinate system and spliced ​​and merged through an external parameter conversion matrix to obtain the panoramic point cloud data. The external parameter conversion matrix is ​​used to perform coordinate conversion on the data collected by the laser radar.

[0021] Optionally, after obtaining the panoramic point cloud data, the method further includes:

[0022] Obtaining the acquisition time of the laser radar and the speed of the spreader during the acquisition time;

[0023] According to the spreader speed during the acquisition time, assigning a corresponding spreader speed to each point cloud data in the panoramic point cloud data;

[0024] Integrate the lifting device speed corresponding to each point cloud data to obtain the position and posture corresponding to each point cloud data;

[0025] Back-propagation is performed on the posture corresponding to each point cloud data, each point cloud data is converted to the same coordinate system, and the panoramic point cloud data after motion distortion correction is obtained.

[0026] Optionally, performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene includes:

[0027] Selecting point cloud data from the segmented point cloud data to form a plane point cloud variance matrix;

[0028] Calculate a first eigenvalue of a plane corresponding to a plane perpendicular to the first coordinate axis, a second eigenvalue of a plane corresponding to a plane perpendicular to the second coordinate axis, and a third eigenvalue of a plane corresponding to a plane perpendicular to the third coordinate axis of the planar point cloud variance matrix;

[0029] When the first eigenvalue of the plane is smaller than both the second eigenvalue of the plane and the third eigenvalue of the plane, fitting the point cloud data corresponding to the plane point cloud variance matrix into a container plane;

[0030] The coordinates of the point cloud data on the container plane are calculated to obtain the position of the container in the loading and unloading operation scene.

[0031] Optionally, after obtaining the segmented point cloud data, the method further includes:

[0032] The number of each group of point cloud data in the segmented point cloud data is counted and compared with a preset value range to verify the accuracy of the segmented point cloud data.

[0033] In a second aspect, an embodiment of the present application provides a container positioning system, including:

[0034] Cranes, used for loading and unloading containers in Benbe;

[0035] at least four laser radars for collecting data during loading and unloading operations of the container;

[0036] A positioning device is used to obtain panoramic point cloud data, which is data collected by a laser radar during loading and unloading operations on the container; obtain container slot gaps, segment and cluster the panoramic point cloud data based on the container slot gaps to obtain segmented point cloud data; perform plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene;

[0037] The laser radar is installed under the trolley platform of the crane. Every two laser radars whose main collection directions are perpendicular to each other form a group, and at least two groups of laser radars are located on the diagonal lines of the trolley platform of the crane.

[0038] In a third aspect, an embodiment of the present application provides a container positioning device, including:

[0039] A first acquisition unit is used to acquire panoramic point cloud data, where the panoramic point cloud data is data collected by a laser radar during loading and unloading operations of the container;

[0040] A second acquisition unit is used to acquire container slot gaps, and segment and cluster the panoramic point cloud data based on the container slot gaps to acquire segmented point cloud data;

[0041] The calculation unit is used to perform plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene.

[0042] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor;

[0043] The memory is connected to the processor and is used to store programs;

[0044] The processor is used to implement the container positioning method as described in the first aspect by running the program in the memory.

[0045] The present application provides a container positioning method. First, the panoramic point cloud data and the container slot gap are obtained, and the panoramic point cloud data is segmented into multiple groups of point cloud data by the container slot gap before subsequent processing, collectively referred to as segmented point cloud data. Each group of point cloud data in the segmented point cloud data roughly corresponds to a container edge. In the subsequent processing process, it is no longer necessary to cluster and fit the panoramic point cloud data with a large degree of discreteness. It is only necessary to group the groups of point cloud data with a relatively concentrated distribution in the segmented point cloud data to perform plane feature fitting and coordinate calculation to obtain the container position, thereby improving the processing efficiency. This enables the computer to complete the analysis of multiple container point cloud data within a specified time, and realize the positioning of more containers in the loading and unloading operation scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0047] Figure 1 A flowchart of a container positioning method provided in an embodiment of the present application.

[0048] Figure 2 A schematic diagram of a laser radar installation method provided in an embodiment of the present application.

[0049] Figure 3 This is a structural schematic diagram of a container positioning device in an embodiment of the present application.

[0050] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The first embodiment of the present application provides a container positioning method, such as Figure 1 As shown, the method may include the following steps:

[0053] Step 101, obtain the panoramic point cloud data, which is the data collected by the laser radar during the loading and unloading operations of the container.

[0054] The panoramic point cloud data includes the point cloud data obtained by detecting all the containers and neighboring containers that need to be positioned; in other words, it is the point cloud data obtained by detecting the containers within the specified range in the loading and unloading operation scene. The specified range is set according to actual needs. For example, if the collection range covered by the laser radar is set to cover the entire loading and unloading operation scene, then the panoramic point cloud data includes the point cloud data of all containers in the loading and unloading operation scene.

[0055] Since the collection range of a single laser radar is limited, for example, if the maximum effective collection range of a common laser radar is projected onto the ground to form an approximate rectangle, the scattered data collected outside the rectangle is distorted and not used. When the collection range is insufficient, multiple laser radars can be set up to expand the collection range by merging the point cloud data of multiple laser radars to obtain complete panoramic point cloud data.

[0056] The method described in this embodiment can be executed simultaneously with the container loading and unloading operations and the container position can be generated in real time. Therefore, the panoramic point cloud data is the point cloud data acquired by the laser radar in real time during the loading and unloading operations of the container. The method described in this embodiment does not need to stop the loading and unloading work specifically to collect the point cloud data, and will not affect the normal container loading and unloading work.

[0057] Step 102: Obtain container slot gaps, segment and cluster the panoramic point cloud data based on the container slot gaps, and obtain segmented point cloud data.

[0058] The container slot gap data can be used to segment the point cloud. Specifically, the panoramic point cloud data includes the point cloud data of multiple containers. The container slot gap can be used to separate the data of each container. Each segmentation is performed every time a container slot gap is used. The panoramic point cloud data can be segmented into multiple groups of point cloud data. Each group of point cloud data corresponds to a container. After segmentation, each group of point cloud data mainly includes the point cloud data of the corresponding container and a small amount of discrete error data.

[0059] Each group of segmented point cloud data is still discrete and includes some noise data, so each group of segmented point cloud data is clustered to obtain segmented point cloud data. The segmented point cloud data is multiple groups of point cloud data corresponding to multiple containers after removing noise data.

[0060] The method for obtaining the container slot gap can be set according to actual needs. For example, each time the method is executed, the point cloud data is analyzed, and after the container edge is determined, the distance between the two container edges is calculated as the container slot gap; or, in order to reduce the system resource consumption of obtaining the container slot gap and improve efficiency, the above method of analyzing the point cloud data to obtain the container slot gap can be executed once at regular intervals to record the latest container slot gap, and the latest container slot gap can be directly obtained when the container slot gap is needed; or, when it is determined that the containers are stacked neatly, the manually input container slot gap data is accepted.

[0061] Step 103: perform plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene.

[0062] In step 102, the segmentation of the point cloud data has been completed and the segmented point cloud data has been obtained, but the segmented point cloud data is still a plurality of discrete point data. Therefore, it is necessary to perform plane feature fitting on each set of segmented point cloud data, and fit the plurality of point data into a figure on the plane. The point cloud data of the container is generally fitted into a parallelogram on a plane.

[0063] Then, the coordinates of the fitted graphics in the specified coordinate system are calculated to quantify the container position through data, such as calculating the coordinates of each corner point of the parallelogram fitted by the container, and a schematic diagram can be generated using the calculated coordinates for external display.

[0064] Optionally, the principal components analysis (PCA) method may be combined to perform plane feature fitting to further reduce the amount of calculation for plane feature fitting and further improve the efficiency of container positioning.

[0065] The second embodiment of the present application further defines the container positioning method in the first embodiment in more detail and specificity. Some or all of the technical features in the second embodiment can be combined with or replaced with the first embodiment alone or in combination, so as to obtain more feasible container positioning methods.

[0066] The container positioning method in the second embodiment of the present application is described in detail below:

[0067] Optionally, after obtaining the panoramic point cloud data, obtaining the container slot clearance includes: obtaining point cloud data within a preset range in the panoramic point cloud data as first point cloud data, clustering and edge feature fitting the first point cloud data, and obtaining edge point cloud data; performing line feature fitting on the edge point cloud data to obtain the edge line of the current container and the edge line of the adjacent container; and calculating the container slot clearance based on the edge line of the current container and the edge line of the adjacent container.

[0068] This embodiment provides a method for obtaining container slot clearance.

[0069] Specifically, the point cloud data within a preset range in the panoramic point cloud data is obtained as the first point cloud data. In practical applications, the point cloud data within the preset range after excluding all abnormal situations is used as the first point cloud data. The abnormal situation refers to the point cloud data that obviously does not belong to the container. For example, when the laser radar is collecting data, a bird flies over the preset range, resulting in a cluster of bird-shaped point clouds in the point cloud data. These abnormal point cloud data need to be removed.

[0070] The preset range can be set as needed. It is recommended that the preset range include the container and its neighboring containers that are undergoing loading and unloading operations, and the container slot gap is calculated using the point cloud data of the container and its neighboring containers to improve the accuracy of the container slot gap data. Clustering is performed on the first point cloud data. Optionally, Euclidean clustering can be performed to further remove noise data.

[0071] The clustering result will include multiple groups of point cloud data. Edge feature fitting is performed on each group of clustered point cloud to obtain edge point cloud data. The edge point cloud data includes multiple groups of point cloud data, and each group of point cloud data corresponds to an edge of the actual container. Line feature fitting is performed on the discrete edge point cloud data to convert the point data into lines.

[0072] The result of line feature fitting may include multiple edge lines of the own container and adjacent container, and the preset range setting is required to determine the calculation of the container slot gap. For example, if the preset range includes a complete own container and a complete adjacent container, the result of line feature fitting should be fitted into four line segments in the long side direction of the container. According to the coordinates of the point cloud data on the middle two line segments, the distance between the middle two line segments is calculated as the container slot gap.

[0073] Optionally, line feature fitting is performed on the edge point cloud data to obtain the edge line of the original container and the adjacent container edge line, including: selecting point cloud data in the edge point cloud data to form a point cloud variance matrix; calculating the first eigenvalue along the first coordinate axis, the second eigenvalue along the second coordinate axis, and the third eigenvalue along the third coordinate axis of the point cloud variance matrix; when the first eigenvalue is greater than both the second eigenvalue and the third eigenvalue, fitting the point cloud data corresponding to the point cloud variance matrix as the original container edge line and the adjacent container edge line.

[0074] Since the three-dimensional space coordinate system has three directions, X, Y, and Z, the point cloud variance matrix established is a rank three matrix with three eigenvalues. Each eigenvalue represents the variance of the position distribution of all point cloud data in the matrix in a certain direction. For example, when the eigenvalue corresponding to the X-axis direction is large, it means that the point cloud data constituting the matrix is ​​unevenly distributed in the X-axis direction, resulting in a large variance in the X coordinate.

[0075] It is particularly important to note that the naming of the X-axis and the Y-th eigenvalue is only for the convenience of distinguishing the three eigenvalues, and does not limit their directions. For different matrices, the directions of the first coordinate axes corresponding to their first eigenvalues ​​are not necessarily the same. For example, for the point cloud variance matrix A, the matrix corresponds to a container edge oriented in the direction of the X-axis of the three-dimensional space coordinate system, and the first eigenvalue of matrix A is set to correspond to the X-axis direction of the coordinate axis; and for the point cloud variance matrix B, the matrix corresponds to a container edge oriented in the direction of the Y-axis of the three-dimensional space coordinate system, and the first eigenvalue of matrix B can be set to correspond to the Y-axis direction of the coordinate axis.

[0076] If the first eigenvalue is greater than both the second eigenvalue and the third eigenvalue (it is generally believed that the first eigenvalue should be much greater than the second and third eigenvalues), it means that the point cloud data constituting the point cloud variance matrix is ​​unevenly distributed in the direction of the first eigenvalue, but evenly distributed in the directions of the second and third eigenvalues. Combined with the previous segmentation, it can be analyzed that the point cloud data constituting the point cloud variance matrix is ​​a line segment facing the direction of the first coordinate axis, and the eigenvector corresponding to the first eigenvalue is the normal vector of the line where the line segment is located. Then, the point cloud data corresponding to the matrix can be fitted to obtain the container edge line, specifically including the original container edge line and the adjacent container edge line.

[0077] It is particularly important to note that even after multiple filtering, it is still impossible to ensure that each set of data in the edge point cloud data only includes the point cloud of the container edge corresponding to the set of data. Therefore, for a set of data in the edge point cloud data, generally all point clouds in the set of data are selected to form a point cloud variance matrix. When the eigenvalue of the point cloud variance matrix does not meet the requirements (the first eigenvalue is greater than both the second eigenvalue and the third eigenvalue), it is considered that the line feature fitting fails. The discrete data in the set of data can be removed again and the data with a higher degree of aggregation can be selected to form a new point cloud variance matrix for calculation until a point cloud variance matrix that meets the requirements is obtained.

[0078] For example, as shown below, the fitting method is to calculate the point cloud variance matrix The three eigenvalues ​​of and the corresponding eigenvector, when the eigenvalue Greater than the other two eigenvalues When , the fitting is successful, and its corresponding eigenvector is the normal vector. is a 3×1 point cloud coordinate matrix, is the coordinate of the point cloud centroid, is the number of point clouds:

[0079]

[0080] Optionally, obtaining panoramic point cloud data includes: obtaining multiple groups of discrete point cloud data collected by a laser radar; converting the discrete point cloud data to the same coordinate system through an external parameter transformation matrix and splicing and merging them to obtain the panoramic point cloud data, and the external parameter transformation matrix is ​​used to perform coordinate transformation on the data collected by the laser radar.

[0081] In the case where multiple laser radars are set up due to insufficient collection range of a single laser radar, the data collected by each laser radar is collected by establishing a spatial coordinate system with the radar itself as the origin. The reference systems of these data are different and cannot be directly merged and calculated.

[0082] Therefore, this embodiment proposes that for multiple sets of discrete point cloud data collected by multiple laser radars, the external parameter conversion matrix from the radar to the center of the spreader obtained by pre-calibration is , the data collected by each radar is converted from the radar coordinate system with itself as the origin to a coordinate system with a unified origin, such as the center of the sling as the origin, and the converted point clouds are spliced ​​to obtain the panoramic point cloud data.

[0083] For example, a specific conversion method is as follows: is the point cloud in the radar coordinate system before conversion, is the converted point cloud, For the spliced ​​panoramic point cloud, is the three-dimensional rotation matrix, is a 3×1 translation vector:

[0084]

[0085] Optionally, after obtaining the panoramic point cloud data, it also includes: obtaining the acquisition time of the laser radar and the hoist speed during the acquisition time; assigning a corresponding hoist speed to each point cloud data in the panoramic point cloud data according to the hoist speed during the acquisition time; integrating the hoist speed corresponding to each point cloud data to obtain the posture corresponding to each point cloud data; back-propagating the posture corresponding to each point cloud data, converting each point cloud data to the same coordinate system, and obtaining the panoramic point cloud data after motion distortion correction.

[0086] It takes time for the laser radar to collect data, and while the laser radar is collecting data, loading and unloading work is also in progress. For example, a laser radar needs 100ms to complete a data collection. Assume that one edge of a container is line segment AB. The laser radar scans the endpoint A of line segment AB at 0ms and obtains the corresponding point cloud data, scans the midpoint of line segment AB at 50ms and obtains the corresponding point cloud data, and scans the endpoint B of line segment at 100ms and obtains the corresponding point cloud data, completing a collection. However, the position of the container did not change during this 100ms, but the laser radar moved a distance with the sling, that is, at 0ms and 100ms, the position of the laser radar relative to endpoints A and B changed, and the coordinate system with the laser radar as the origin moved, and there was motion distortion in the point cloud data.

[0087] This embodiment proposes a method for eliminating motion distortion. Specifically, the spreader speed within the acquisition time is obtained, and the speed can be obtained through the crane system data. Then, the corresponding spreader speed is assigned to each point cloud data in the panoramic point cloud data. For example, within the acquisition time, the spreader speed at 0ms is 0m / s, the spreader speed at 50ms is 1m / s, and the spreader speed at 100ms is 2m / s. Then, the endpoint A in the above example is assigned a speed of 0m / s; the midpoint of the AB line segment is assigned a speed of 1m / s; and the endpoint B is assigned a speed of 2m / s.

[0088] In addition, if the frequency of the spreader speed data within the acquisition time is low, for example, if the crane system outputs the spreader speed once every 100ms, then the speed can be interpolated. For example, if the spreader speed at 0ms is 0m / s and the spreader speed at 100ms is 2m / s within the acquisition time, then the spreader speed at 50ms can be interpolated and calculated to be 1m / s for supplementation.

[0089] Then, the speed of the spreader corresponding to each point cloud data is integrated, and the result of the speed integration is displacement, so the position and posture corresponding to each point cloud data can be obtained. The position and posture corresponding to each point cloud data are back-propagated so that each point cloud data is converted to the same coordinate system. For example, a basic coordinate system is established with the position of the radar at time 0ms as the origin, and the point cloud data at times 50ms and 100ms are back-propagated to achieve coordinate conversion, and the coordinate system of the point cloud data at times 50ms and 100ms is converted to the basic coordinate system to achieve motion distortion correction, and obtain the panoramic point cloud data after motion distortion correction.

[0090] Optionally, plane feature fitting and coordinate calculation are performed on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene, including: selecting point cloud data in the segmented point cloud data to form a plane point cloud variance matrix; calculating the first eigenvalue of the plane corresponding to the plane perpendicular to the first coordinate axis, the second eigenvalue of the plane corresponding to the plane perpendicular to the second coordinate axis, and the third eigenvalue of the plane corresponding to the plane perpendicular to the third coordinate axis of the plane point cloud variance matrix; when the first eigenvalue of the plane is smaller than both the second eigenvalue of the plane and the third eigenvalue of the plane, fitting the point cloud data corresponding to the plane point cloud variance matrix into a container plane; calculating the coordinates of the point cloud data on the container plane to obtain the position of the container in the loading and unloading operation scene.

[0091] The specific implementation method of plane feature fitting is similar to the principle of the method of line feature fitting for edge point cloud data mentioned above, the difference is that the spatial structure formed by the point cloud is changed from line segments to plane figures.

[0092] Specifically, each eigenvalue of the plane point cloud variance matrix represents the variance of the position distribution of all point cloud data in the matrix on a plane perpendicular to a certain coordinate axis. For example, the plane perpendicular to the X-axis is the Y-plane. When the eigenvalue corresponding to the Y-plane is large, it means that the spatial position distribution of the point cloud data constituting the matrix on the Y-plane is uneven, resulting in a large variance calculated for the X-coordinate, and the surface composed of the point cloud data constituting the matrix is ​​"uneven" in distribution on the Y-plane, forming a curved surface figure; when the eigenvalue corresponding to the Y-plane is small, it means that the position distribution of the point cloud data constituting the matrix on the Y-plane is relatively uniform, resulting in a small variance of the X-coordinate, and the distribution of the point cloud data constituting the matrix on the Y-plane is almost uneven, and should be a plane figure.

[0093] Therefore, the point cloud data corresponding to the plane point cloud variance matrix whose first eigenvalue is smaller than both the second and third eigenvalues ​​of the plane (it is generally believed that the first eigenvalue of the plane needs to be much smaller than the second and third eigenvalues ​​of the plane) is selected to fit the container plane, and the coordinates of the point cloud data on the container plane are calculated to obtain the position of the container in the loading and unloading operation scenario.

[0094] Similar to line feature fitting, when plane feature fitting fails, the more discrete data can be removed and the data with a higher degree of aggregation can be selected to form a new plane variance matrix for calculation until a plane variance matrix that meets the requirements is obtained.

[0095] Optionally, after acquiring the segmented point cloud data, the method further includes: counting the number of each group of point cloud data in the segmented point cloud data and comparing it with a preset value range to verify the accuracy of the segmented point cloud data.

[0096] This embodiment proposes a method for checking the accuracy of segmented point cloud data. Specifically, after completing the segmentation and obtaining the segmented point cloud data, because the collection distribution of the laser radar is relatively uniform, according to the law, each group of point cloud data in the segmented point cloud data should also fall within the preset numerical range. For example, collecting a long side of a container can roughly obtain about 1,000 point cloud data. The preset numerical range is set to 800-1200. When the number of a group of point cloud data in the segmented point cloud data is too large or too small. For example, when there are only 300 point cloud data or 2,000 point cloud data, the segmentation of this group of data may not be accurate. There may be a situation where the edge is not completely intercepted, resulting in fewer point clouds, or the two edges are divided into one group, resulting in more point clouds.

[0097] The third embodiment of the present application also proposes a container positioning system, including: a crane, used to load and unload the container; at least four laser radars, used to collect data when loading and unloading the container; a positioning device, used to obtain panoramic point cloud data, which is data collected by the laser radar when loading and unloading the container; obtaining the container slot gap, segmenting and clustering the panoramic point cloud data based on the container slot gap, and obtaining segmented point cloud data; performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene; the laser radar is installed under the trolley platform of the crane, and each two laser radars whose main collection directions are perpendicular to each other are a group, and at least two groups of laser radars are located on the diagonal of the trolley platform of the crane.

[0098] The container positioning system proposed in this embodiment installs the laser radar under the trolley platform of the crane and moves with the spreader. In order to ensure the coverage of the laser radar, at least four laser radars are required. Each group of two laser radars with mutually perpendicular main acquisition directions is a group, and at least two groups of laser radars are located on the diagonal line of the crane's trolley platform.

[0099] For example, Figure 2 The figure shows a top view of the spreader and the container. Two groups of laser radars are installed in the lower left corner and the upper right corner respectively. In each group of laser radars, one laser radar mainly collects data along the X-axis direction, and the other laser radar mainly collects data along the Y-axis direction to increase the data collection range.

[0100] The fourth embodiment of the present application also provides a container positioning device, such as Figure 3 As shown, including:

[0101] The first acquisition unit 301 is used to acquire panoramic point cloud data, where the panoramic point cloud data is data collected by a laser radar during loading and unloading operations of the container;

[0102] The second acquisition unit 302 is used to acquire the container slot gap, segment and cluster the panoramic point cloud data based on the container slot gap, and acquire segmented point cloud data;

[0103] The calculation unit 303 is used to perform plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene.

[0104] Optionally, after obtaining the panoramic point cloud data, obtain the container slot clearance, including:

[0105] Obtaining point cloud data within a preset range from the panoramic point cloud data as first point cloud data, performing clustering and edge feature fitting on the first point cloud data, and obtaining edge point cloud data;

[0106] Perform line feature fitting on the edge point cloud data to obtain the edge line of the container and the edge line of the adjacent container;

[0107] The container slot gap is calculated based on the edge line of the container and the edge line of the adjacent container.

[0108] Optionally, performing line feature fitting on the edge point cloud data to obtain the edge line of the container and the edge line of the adjacent container includes:

[0109] Select point cloud data from edge point cloud data to form a point cloud variance matrix;

[0110] Calculate a first eigenvalue along a first coordinate axis, a second eigenvalue along a second coordinate axis, and a third eigenvalue along a third coordinate axis of a point cloud variance matrix;

[0111] When the first eigenvalue is greater than both the second eigenvalue and the third eigenvalue, the point cloud data corresponding to the point cloud variance matrix is ​​fitted as the edge line of the local container and the edge line of the neighboring container.

[0112] Optionally, obtain panoramic point cloud data, including:

[0113] Obtain multiple sets of discrete point cloud data collected by LiDAR;

[0114] The discrete point cloud data are converted to the same coordinate system through the external parameter transformation matrix and spliced ​​and merged to obtain the panoramic point cloud data. The external parameter transformation matrix is ​​used to perform coordinate transformation on the data collected by the lidar.

[0115] Optionally, after obtaining the panoramic point cloud data, the following is also included:

[0116] Get the laser radar acquisition time and the spreader speed during the acquisition time;

[0117] According to the spreader speed during the acquisition time, a corresponding spreader speed is assigned to each point cloud data in the panoramic point cloud data;

[0118] Integrate the lifting device speed corresponding to each point cloud data to obtain the position and posture corresponding to each point cloud data;

[0119] Back propagation is performed on the pose corresponding to each point cloud data, and each point cloud data is converted to the same coordinate system to obtain the panoramic point cloud data after motion distortion correction.

[0120] Optionally, plane feature fitting and coordinate calculation are performed on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene, including:

[0121] Selecting point cloud data from the segmented point cloud data to form a plane point cloud variance matrix;

[0122] Calculate the first eigenvalue of the plane corresponding to the plane perpendicular to the first coordinate axis, the second eigenvalue of the plane corresponding to the plane perpendicular to the second coordinate axis, and the third eigenvalue of the plane corresponding to the plane perpendicular to the third coordinate axis of the planar point cloud variance matrix;

[0123] When the first eigenvalue of the plane is smaller than both the second eigenvalue of the plane and the third eigenvalue of the plane, the point cloud data corresponding to the plane point cloud variance matrix is ​​fitted as the container plane;

[0124] Calculate the coordinates of the point cloud data on the container plane to obtain the position of the container in the loading and unloading operation scene.

[0125] Optionally, after obtaining the segmented point cloud data, the following steps are further included:

[0126] The number of each group of point cloud data in the segmented point cloud data is counted and compared with a preset value range to verify the accuracy of the segmented point cloud data.

[0127] The container positioning device provided in this embodiment and the container positioning method provided in the above embodiments of this application belong to the same application concept, and can execute the container positioning method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in this embodiment, please refer to the specific processing content of the container positioning method provided in the above embodiments of this application, and will not be repeated here.

[0128] The positioning device in the container positioning system proposed in the third embodiment of the present application includes the container positioning apparatus provided in this embodiment.

[0129] The functions implemented by the above-mentioned first acquisition unit 301, the second acquisition unit 302 and the calculation unit 303 may be implemented by the same or different processors respectively, which is not limited in the embodiment of the present application.

[0130] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0131] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0132] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0133] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0134] The fifth embodiment of the present application also provides an electronic device, see Figure 4 As shown, the device includes:

[0135] Memory 400 and processor 410;

[0136] The memory 400 is connected to the processor 410 and is used to store programs;

[0137] The processor 410 is configured to implement the container positioning method disclosed in any of the above embodiments by running the program stored in the memory 400 .

[0138] Specifically, the electronic device may further include: a bus, a communication interface 420 , an input device 430 and an output device 440 .

[0139] The processor 410, the memory 400, the communication interface 420, the input device 430 and the output device 440 are connected to each other via a bus.

[0140] A bus may include a pathway that transfers information between components of a computer system.

[0141] The processor 410 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0142] The processor 410 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0143] The memory 400 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 400 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0144] The input device 430 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0145] Output device 440 may include a device that allows information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0146] The communication interface 420 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0147] The processor 410 executes the program stored in the memory 400 and calls other devices, which can be used to implement each step of any container positioning method provided in the above embodiments of the present application.

[0148] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0149] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0150] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0151] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0152] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0153] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0155] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0156] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0157] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0158] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A container positioning method, characterized in that: include: Obtaining panoramic point cloud data, wherein the panoramic point cloud data is data collected by a laser radar during loading and unloading operations of the container; Obtaining container slot gaps, segmenting and clustering the panoramic point cloud data based on the container slot gaps, and obtaining segmented point cloud data; Performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene; The performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene includes: Selecting point cloud data from the segmented point cloud data to form a plane point cloud variance matrix; Calculate a first eigenvalue of a plane corresponding to a plane perpendicular to the first coordinate axis, a second eigenvalue of a plane corresponding to a plane perpendicular to the second coordinate axis, and a third eigenvalue of a plane corresponding to a plane perpendicular to the third coordinate axis of the planar point cloud variance matrix; When the first eigenvalue of the plane is smaller than both the second eigenvalue of the plane and the third eigenvalue of the plane, fitting the point cloud data corresponding to the plane point cloud variance matrix into a container plane; The coordinates of the point cloud data on the container plane are calculated to obtain the position of the container in the loading and unloading operation scene.

2. The method according to claim 1, characterized in that After obtaining the panoramic point cloud data, obtaining the container slot clearance includes: Acquire point cloud data within a preset range in the panoramic point cloud data as first point cloud data, perform clustering and edge feature fitting on the first point cloud data, and acquire edge point cloud data; Performing line feature fitting on the edge point cloud data to obtain the edge line of the container and the edge line of the adjacent container; The container slot gap is calculated according to the own container edge line and the adjacent container edge line.

3. The method according to claim 2, characterized in that The performing line feature fitting on the edge point cloud data to obtain the edge line of the container and the edge line of the adjacent container includes: Selecting point cloud data from the edge point cloud data to form a point cloud variance matrix; Calculating a first eigenvalue along a first coordinate axis, a second eigenvalue along a second coordinate axis, and a third eigenvalue along a third coordinate axis of the point cloud variance matrix; When the first eigenvalue is greater than both the second eigenvalue and the third eigenvalue, the point cloud data corresponding to the point cloud variance matrix is ​​fitted as the Benbei container edge line and the adjacent container edge line.

4. The method according to claim 1, characterized in that: The step of obtaining the panoramic point cloud data includes: Acquire multiple groups of discrete point cloud data collected by the laser radar; The discrete point cloud data are converted to the same coordinate system and spliced ​​and merged through an external parameter conversion matrix to obtain the panoramic point cloud data. The external parameter conversion matrix is ​​used to perform coordinate conversion on the data collected by the laser radar.

5. The method according to claim 1, characterized in that After obtaining the panoramic cloud data, the method further includes: Obtaining the acquisition time of the laser radar and the speed of the spreader during the acquisition time; According to the spreader speed during the acquisition time, assigning a corresponding spreader speed to each point cloud data in the panoramic point cloud data; Integrate the lifting device speed corresponding to each point cloud data to obtain the position and posture corresponding to each point cloud data; Back-propagation is performed on the posture corresponding to each point cloud data, each point cloud data is converted to the same coordinate system, and the panoramic point cloud data after motion distortion correction is obtained.

6. The method according to claim 1, characterized in that After obtaining the segmented point cloud data, the method further includes: The number of each group of point cloud data in the segmented point cloud data is counted and compared with a preset value range to verify the accuracy of the segmented point cloud data.

7. A container positioning system, characterized in that: include: Cranes, used for loading and unloading containers in Benbe; at least four laser radars for collecting data during loading and unloading operations of the container; A positioning device, used for obtaining panoramic point cloud data, wherein the panoramic point cloud data is data collected by a laser radar when loading and unloading a container; obtaining container slot gaps, segmenting and clustering the panoramic point cloud data based on the container slot gaps, and obtaining segmented point cloud data; performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene; performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene, including: selecting point cloud data in the segmented point cloud data to form a plane point cloud variance matrix; calculating a first plane eigenvalue corresponding to a plane perpendicular to a first coordinate axis, a second plane eigenvalue corresponding to a plane perpendicular to a second coordinate axis, and a third plane eigenvalue corresponding to a plane perpendicular to a third coordinate axis of the plane point cloud variance matrix; when the first plane eigenvalue is less than both the second plane eigenvalue and the third plane eigenvalue, fitting the point cloud data corresponding to the plane point cloud variance matrix to a container plane; calculating the coordinates of the point cloud data on the container plane to obtain the position of the container in the loading and unloading operation scene; The laser radar is installed under the trolley platform of the crane. Every two laser radars whose main collection directions are perpendicular to each other form a group, and at least two groups of laser radars are located on the diagonal lines of the trolley platform of the crane.

8. A container positioning device, characterized in that: include: A first acquisition unit is used to acquire panoramic point cloud data, where the panoramic point cloud data is data collected by a laser radar during loading and unloading operations of the container; A second acquisition unit is used to acquire container slot gaps, and segment and cluster the panoramic point cloud data based on the container slot gaps to acquire segmented point cloud data; A calculation unit, used to perform plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene; The performing plane feature fitting and coordinate calculation on the segmented point cloud data to obtain the position of the container in the loading and unloading operation scene includes: selecting point cloud data in the segmented point cloud data to form a plane point cloud variance matrix; calculating a first plane eigenvalue corresponding to a plane perpendicular to a first coordinate axis, a second plane eigenvalue corresponding to a plane perpendicular to a second coordinate axis, and a third plane eigenvalue corresponding to a plane perpendicular to a third coordinate axis of the plane point cloud variance matrix; when the first plane eigenvalue is smaller than both the second plane eigenvalue and the third plane eigenvalue, fitting the point cloud data corresponding to the plane point cloud variance matrix into a container plane; calculating the coordinates of the point cloud data on the container plane to obtain the position of the container in the loading and unloading operation scene.

9. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the container positioning method according to any one of claims 1 to 6 by running the program in the memory.

Images