Methanol tank truck filling port position positioning method, terminal device and storage medium

Abstract

The application discloses a methanol tank truck filling port pose positioning method, a terminal device and a storage medium, image and point cloud data of a methanol tank truck filling port are collected through a depth camera, and image enhancement is performed on the collected image data based on CsNorm; a yolov8 detection model is used to obtain the approximate pose of the left and right filling port centers; the initial point cloud is segmented based on the normal vector constraint to obtain the local point cloud data of the left and right filling ports; edge feature extraction is performed on the local point cloud of the left and right filling ports based on the longitude and latitude line scanning method to obtain an edge feature set; a CPO descriptor of the edge feature set of the left and right filling ports is constructed; target point cloud and initial point cloud are registered according to a feature matching algorithm and a similarity score to obtain a measurement transformation pose matrix; and an ICP algorithm is used to finely register the coarse transformation pose to obtain the accurate pose information of the left and right filling ports of the tank truck. The application can improve the robustness of the filling port detection in insufficient or excessively strong light environments.

Description

Technical Field

[0001] The present invention relates to the field of automatic filling of tank trucks, and in particular to a method for positioning the filling port of a methanol tank truck, terminal equipment and storage medium. Background Art

[0002] With the acceleration of industrialization and the continued growth of energy demand, methanol, as an alternative energy source, is becoming increasingly prominent in the energy system. As a key tool for methanol transportation, methanol tanker trucks require rapid and accurate positioning of their filling ports to improve filling efficiency and safety.

[0003] However, traditional methanol tanker filling port positioning relies heavily on manual operation, a method that is not only inefficient but also poses potential safety risks. Manual positioning accuracy and safety are particularly difficult to guarantee at night or in low-visibility environments. Furthermore, methanol is toxic and corrosive, and manual operation can pose a health risk to operators.

[0004] Currently, research on methanol tanker filling port positioning is limited, and a mature technical solution is yet to be developed. While some automated technologies have been applied to filling port positioning, these methods often suffer from high computational complexity, slow processing speed, low robustness, high image quality requirements, and are easily affected by factors such as ambient light. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a methanol tanker filling port posture positioning method, terminal equipment and storage medium to address the problems of poor robustness, slow speed and great influence of light on the positioning of the methanol filling port.

[0006] To solve the above technical problems, the technical solution adopted by the present invention is: a method for positioning the filling port of a methanol tank truck, comprising the following steps:

[0007] Collecting image data of a methanol tanker filling port, and using the image data to construct a sample dataset of a methanol tanker filling port;

[0008] Using the sample data set to train a neural network model, and using the trained neural network model to roughly locate the methanol filling port to obtain the approximate positions of the centers of the two filling ports;

[0009] The approximate pose information of the center points is obtained based on the approximate pose of the centers of the two filling ports. The initial point cloud collected by the depth camera is segmented based on the normal vector constraint to obtain the local point cloud data of the left and right filling ports respectively.

[0010] Extract edge features from the local point cloud data of the left and right filling ports and the initial point cloud based on a longitude and latitude scanning method to obtain an edge feature point set;

[0011] Constructing a CPO descriptor set of edge feature points of the target point cloud and the initial point cloud respectively according to the edge feature point set and the approximate position of the filling port center;

[0012] The target point cloud and the initial point cloud are roughly registered according to the feature matching algorithm and similarity score to obtain the corresponding rough transformation pose matrix;

[0013] The ICP algorithm is used to perform fine registration on the coarse transformation pose matrix to obtain the final accurate pose information of the tank truck filling port.

[0014] The specific process of obtaining the approximate pose of the centers of the two filling ports includes:

[0015] Use a depth camera to acquire real-time image data around the filling port of a methanol tanker truck, and enhance the collected image data based on CsNorm.

[0016] The enhanced image data is fed into the trained neural network model for target detection, which outputs the two-dimensional pixel coordinates of the two filling ports.

[0017] According to the image data collected by the depth camera, the depth value of the center of the filling port of the methanol tank truck is obtained through the affine function;

[0018] The two-dimensional pixel coordinates are converted into two-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and depth value, so as to obtain the approximate position of the centers of the two filling ports.

[0019] The specific process of obtaining the local point cloud data of the left and right filling ports includes:

[0020] The farthest plane and its normal vector of the initial point cloud are extracted based on the RANSAC fitting algorithm, and the initial point cloud is filtered according to the extracted farthest plane to obtain local point cloud data;

[0021] Based on the obtained approximate position information of the center point and the physical model of the filling port, the approximate positions of the left and right filling ports are calculated in the local point cloud data and segmented to obtain the local point cloud data of the left and right filling ports, respectively.

[0022] The specific process of obtaining the edge feature point set includes:

[0023] Project the target point cloud data and the initial point cloud data onto the xoy plane to obtain a two-dimensional data point set with changed coordinates;

[0024] Sort the two-dimensional data point set according to the value of the horizontal coordinate and store the sorted points in a linked list;

[0025] Starting from the minimum value of the horizontal coordinate, traverse the range of the horizontal coordinate with a certain step size dx; for each meridian, calculate the maximum and minimum values ​​of the meridian on the vertical coordinate, and record the indexes of the points corresponding to these values, and store the indexes in a new linked list;

[0026] Similarly, calculate the maximum and minimum values ​​on the vertical axis and record the index of the corresponding point;

[0027] After obtaining the indexes of all longitudes and latitudes, duplicate index values ​​are deleted, and the boundary points extracted from the target point cloud data and the initial point cloud data are obtained according to the longitude and latitude indexes to obtain the edge feature point set.

[0028] The following processes are implemented on the edge feature point sets of the target point cloud and the initial point cloud respectively to construct the CPO descriptor sets of the edge feature points of the target point cloud and the initial point cloud respectively:

[0029] The center of the filling port is roughly positioned to form a circular area so that the circular area contains the two-dimensional projection points of all edge feature points;

[0030] Calculate the distance from the edge feature point to the center of the filling port to obtain the first part of the CPO descriptor;

[0031] Project the edge feature points onto the circular surface and calculate the distance between the feature points and the projection points, that is, the vertical distance from the feature points to the circular surface, to obtain the second part of the CPO descriptor;

[0032] According to the symmetry of the filling port, the approximate center of the filling port is moved a specified distance in the direction where the angle between the horizontal axis and the positive vertical axis is 45 degrees to obtain a new center point;

[0033] Calculate the distance from the edge feature point to the new center point again to obtain the third part of the CPO descriptor;

[0034] Calculate the angle between the projection point of the edge feature point, the original center point, and a point on the vertical axis to obtain the fourth part of the CPO descriptor.

[0035] The coarse transformation pose matrix acquisition process includes:

[0036] A hash function is constructed using the fourth part of the CPO descriptor set as the key value, and a hash table of the initial point cloud is constructed using the CPO descriptor set as the target value;

[0037] Traverse the target point cloud descriptor set, encode the value of the fourth part of the CPO descriptor set to form a query key value, query the hash table of the initial point cloud for the number of CPO descriptors whose key value difference is within a certain interval, score the target point cloud descriptors according to the number, and obtain a potential descriptor pair set;

[0038] According to the descriptor scores of the target point cloud, the top M% descriptors are screened out to obtain a new set of descriptor pairs;

[0039] Traverse the new set of descriptor pairs, and for each pair of descriptors, calculate the difference and sum of the first, second, and third parts of the descriptor, and retain the descriptor pairs whose differences of the first, second, and third parts are within the threshold and whose sum of differences is the smallest;

[0040] Traverse the last set of retained descriptor pairs, calculate the pose transformation matrix between each pair of descriptors, and perform pose transformation on the remaining descriptor pairs according to the pose transformation matrix. Score the pose transformation matrix of the corresponding descriptor according to the difference between the first and third parts of the transformed descriptor and the number within the threshold. The pose transformation matrix with the highest score is the coarse transformation pose matrix.

[0041] The specific process of obtaining the final precise pose information of the tank truck filling port includes:

[0042] For any point in the initial point cloud, use the KD tree to find the point closest to it in the target point cloud to form a pair of matching points;

[0043] Calculate the Euclidean distance between each pair of matching points and evaluate the score of the current coarse transformation pose matrix;

[0044] According to the score of the coarse transformation pose matrix, iterative optimization is continuously performed so that after the transformation, the Euclidean distance between the points in the initial point cloud and the corresponding points in the target point cloud is the minimum, thereby obtaining the fine registration transformation matrix.

[0045] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.

[0046] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon; the computer program / instruction implements the steps of the above method when executed by a processor.

[0047] As an inventive concept, the present invention also provides a computer program product, comprising a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the above method are implemented.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1) The present invention provides a fast and high-precision method for positioning the filling port of a methanol tank truck. It combines 3D point cloud, 2D image data and image processing technology to achieve accurate positioning of the filling port of a methanol tank truck, and can be applied to the field of automatic filling technology for methanol tank trucks.

[0050] 2) The present invention provides a fast and high-precision method for positioning the filling port of a methanol tanker truck. According to the visual coarse positioning results and the physical model of the filling port, a point cloud segmentation algorithm based on normal vector constraints is designed to obtain local point cloud data of the filling port, which can remove irrelevant factors while reducing the interference caused by noise. At the same time, an edge feature extraction algorithm based on the longitude and latitude scanning method is designed, which reduces the amount of calculation while ensuring feature information, and can speed up subsequent point cloud alignment tasks.

[0051] 3) The present invention provides a fast and high-precision method for positioning the filling port of a methanol tanker truck. Based on the physical model characteristics of the filling port and the visual positioning results, a CPO descriptor specifically for the filling port is designed. This descriptor can maintain high robustness in the case of missing local point cloud data, that is, sensor errors caused by environmental factors (light). At the same time, a feature matching algorithm and a similarity scoring mechanism are designed to act on the pairing between CPO descriptors, which can improve the success rate of alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flow chart of an algorithm for implementing a method for locating the center of a methanol tanker filling port according to the present invention;

[0053] Figure 2 This is a framework diagram of the methanol tanker filling port center positioning system implemented in the present invention;

[0054] Figure 3 This is a diagram showing the structure of the CPO descriptor of the present invention;

[0055] Figure 4 This is the result diagram of the visual part detection;

[0056] Figure 5 This is the point cloud segmentation result image based on normal vector constraint;

[0057] Figure 6 This is the edge feature extraction result image based on the longitude and latitude scanning method;

[0058] Figure 7 The result of registering the left and right filling ports. DETAILED DESCRIPTION

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0060] In this document, the terms "first", "second" and other similar words are not intended to imply any order, quantity and importance, but are merely used to distinguish different elements. In this document, the terms "one", "an" and other similar words are not intended to indicate that there is only one of the things described, but rather that the relevant description is only for one of the things described, and the things described may have one or more. In this document, the terms "comprise", "include" and other similar words are intended to indicate logical relationships, and cannot be regarded as indicating relationships in spatial structure. For example, "A includes B" is intended to indicate that B logically belongs to A, and does not mean that B is spatially located inside A. In addition, the meanings of the terms "comprise", "include" and other similar words should be regarded as open, not closed. For example, "A includes B" is intended to indicate that B belongs to A, but B does not necessarily constitute the whole of A, and A may also include other elements such as C, D, and E.

[0061] Example 1

[0062] This embodiment provides a fast and high-precision overall flow chart of the positioning method of the filling port of a methanol tanker, as shown in FIG. Figure 1 As shown, the following steps are included:

[0063] Step 1: Use a depth camera to acquire real-time image data around the filling port of a methanol tanker from different vehicles and at different times. Create a methanol filling port dataset from the acquired image data and use the dataset for training the yolov8 detection model. This includes the following steps:

[0064] Step 1: Use a depth camera equipped with a slide rail system to collect image data around the filling port of a methanol tanker from different vehicles and angles.

[0065] Step 2: Zoom in and out on the collected images of the methanol tanker tanks to obtain a multi-scale image dataset.

[0066] Step 3: Use the LabelImg annotation tool to annotate the multi-scale image dataset and save it in a yolo trainable data format to obtain the methanol filling port dataset;

[0067] Step 4: Input the methanol filling port dataset into the yolov8 target detection network for training, obtain the optimal weights for network training, and save it as the optimal.pt file;

[0068] Step 2: Use the trained weight file to roughly locate the filling port of the methanol tanker on site, including the following steps:

[0069] Step 1: Use a depth camera to collect image data around the filling port of the methanol tanker on site and perform CsNorm image enhancement processing on it to avoid recognition errors caused by lighting conditions:

[0070] Step 2: Input the enhanced image data into the trained yolov8 detection network to identify the left and right filling port positions, and save the two-dimensional pixel coordinate values ​​of their center positions as the center_2d.txt file. The detection results are as follows: Figure 4 As shown;

[0071] Step 3: Convert the 2D pixel coordinates of the filling port into camera coordinates that can be recognized in the subsequent point cloud registration. In this example: Center tar_l =(-70.1778, 93.3569, 1061.51) and Center tar_r =(291.103, 100.141, 1065.17);

[0072] Step 3: Process the collected initial point cloud based on the approximate position of the center point of the filling port detected by visual inspection to obtain local point cloud data of the left and right filling ports, including the following steps:

[0073] Step 1: After obtaining the approximate position of the center point of the filling port, use the depth camera to collect the point cloud data input.pcd around the filling port and perform filtering on it;

[0074] The process of obtaining the approximate pose of the center point of the filling port is as follows:

[0075] The camera internal parameters and the depth value of the filling port center are used to convert the two-dimensional pixel coordinate values ​​(u, v) of the two filling port centers into two-dimensional coordinates (x, y) in the camera coordinate system, and the approximate position (x) of the left filling port center is obtained. L ,y L ,z L ) and the approximate position of the center of the left filling port (x R ,y R ,z R ), the relevant formula is as follows:

[0076]

[0077] Among them, cx ,c y ,f x ,f y is the internal parameter of the camera.

[0078] Step 2: Segment the filtered point cloud data based on the normal vector constraint and the approximate pose of the filling port center point to obtain local point cloud data of the left and right filling ports;

[0079] Step 2.1: Repeatedly extract the farthest plane and its depth value z in the initial point cloud based on the RANSAC fitting algorithm p , compare the depth value extracted each time to ensure that the farthest plane is obtained, and calculate the average normal vector of the farthest plane. In this example, the average normal vector of the farthest plane is

[0080] Step 2.2, for each point p in the point cloud i , calculate the covariance matrix C of the point and its points in the domain, perform eigendecomposition on the covariance matrix C, and obtain the minimum eigenvector Then calculate and The angle θ between them, and set the angle threshold θ max , if θ>θ max , then remove the point from the point cloud to obtain the local point cloud data of the filling box. In this example, the angle threshold θ max =5°, the relevant calculation formula is as follows:

[0081]

[0082] Step 2.3, based on the extracted farthest plane and its depth value z p The local point cloud data of the filling box is filtered to remove irrelevant point clouds and obtain the local point cloud data of the filling port.

[0083] Step 2.4: Segment the local point cloud data of the left and right filling ports according to the approximate positions of the centers of the left and right filling ports and the physical models of the left and right filling ports {radius: r; filling port height: h} to obtain the local point cloud data of the left and right filling ports, and calculate the average normal vector of the local point cloud data of the left and right filling ports respectively. and The segmentation results are as follows Figure 5 As shown, in this example

[0084] Step 4: Extract edge features from the local point cloud data of the left and right filling ports and the initial point cloud based on the longitude and latitude scanning method to obtain an edge feature point set;

[0085] Step 1: Project the local point cloud data of the filling port onto the xoy plane, i.e. remove the coordinate value in the z direction, and obtain the converted two-dimensional data point set {(x a ,y a ),(x b ,y b )…,(x n ,y n )};

[0086] Step 2: Sort the two-dimensional data point set by the size of the x-coordinate and store the sorted x-coordinates in the list List sort {x1,x2…,x n};

[0087] Step 3: From the List sort Starting from the minimum value of the x coordinate, traverse the range of x coordinates and calculate the longitude of each x i The corresponding maximum value of y max_i and the minimum value y min_i And record the indexes corresponding to these points in a new list List record_x ;

[0088] Step 4: Use the same method to calculate the maximum and minimum values ​​of the y coordinate on the latitude line, and record the index of the corresponding point to get List record_y ;

[0089] Step 5: Traverse the List record_x and List record_y The index value in the target point cloud is used to delete the duplicate values, and the edge feature set Edge of the local point cloud data of the left and right filling ports of the target point cloud is extracted according to the index value. target {point1,point2…,point k}where k is the number of feature set points in the initial point cloud;

[0090] Step 6: Apply the above steps to the local point cloud data of the left and right filling ports of the initial point cloud to obtain the edge feature set Edge of the left and right filling ports of the initial point cloud source {point1,point2…,point L}, where L is the number of initial point cloud feature set points, Figure 6 This is the edge feature extraction result of the local point cloud data of the right filling port and the initial point cloud.

[0091] Step 5: Based on the approximate pose of the filling port center, construct the CPO descriptor of the edge feature set of the target point cloud and the initial point cloud;

[0092] Step 1: Center the filler port tar / sou_l / ris the center of the plane, Construct a plane for the normal vector so that the plane can contain the two-dimensional projection point of the edge feature point.

[0093] Step 2: Calculate the edge feature set Edge target / source Point to Center tar_l The distance d1 is obtained to obtain the first part of the CPO descriptor. The relevant calculation formula is:

[0094]

[0095] Step 3: Put the point P(x P ,y P ,z P ) is mapped onto the plane, and the projection point B(x B ,y B ,z B ), and calculate the distance d2 between the feature point and the projection point, that is, the distance from the feature point to the plane. The relevant calculation formula is as follows:

[0096]

[0097] Step 4: Determine the positive direction of the x-axis and y-axis according to the changes in the x and y coordinates, and move the center of the filling port to a specified distance in the direction of the angle 45 degrees between the positive directions of the x-axis and the y-axis to obtain the new center point C1 (x c1 ,y c1 ,z c1 ).

[0098] Step 5: Calculate the distance from edge feature points to the new center point C1(x c1 ,y c1 ,z c1 ) and obtain the third part of the CPO descriptor.

[0099] Step 6: Calculate the projection point B(x B ,y B ,z B ), the original center point C(x C ,y C ,z C ), a point F(x F ,y F ,z F ) to form an angle α between them, and the fourth part of the descriptor is obtained. The relevant calculation formula is as follows:

[0100]

[0101] Step 7: Implement steps 1 to 6 on the edge feature sets of the left and right filling ports of the target point cloud and the initial point cloud respectively, and construct the CPO descriptors CPO of the left and right filling ports of the target point cloud and the initial point cloud target_l 、CPO source_l 、CPO target_r and CPO source_r .

[0102] Step 6: Roughly align the target point cloud and the initial point cloud based on the feature matching algorithm and similarity score to obtain the corresponding rough transformation pose matrix;

[0103] Step 1: Using α in the CPO descriptor of the initial point cloud left filling port edge feature set as the key value, CPO source_l Build a hash table Hash of the initial point cloud for the target value source_l .

[0104] Step 2: Traverse the target point cloud left filling port descriptor set CPO target_l , extract CPO target_l All α in the set and encode this part to get query target_l {q1,q2,…q m}, in the hash table Hash source_l Query the difference in the interval (q i -5°,q i The sum of the number of descriptors that match within +5° L , and obtain a set of potential matching pairs

[0105] Step 3: Get the score of the target point cloud according to the number of matching descriptors L , filter out the top 20% of descriptors and enter the next stage for feature association, and get a new set of descriptor pairs Pairs{pairs1,pairs2,…pairs k};

[0106] Step 4: Traverse the descriptor pair set Pairs{pairs1,pairs2,…pairs k ,}, for pairs i The descriptor of each pair in the set Calculate the difference and sum of d1, d2, d3, and finally keep the difference between d1, d2, d3 in the interval (d i -2,d i +2) and the difference and the minimum corresponding descriptor pair, and get the final retained descriptor pair set Match{(CPO target_1 ,CPO source_1 ),(CPO target_2 ,CPOsource_2 ),…(CPO target_k ,CPO source_k )}

[0107] Step 5: Traverse the last retained descriptor pair set Match and calculate the pose transformation T between each pair of descriptors ij (R ij ,t ij ), use the pose transformation to transform the remaining descriptor pairs in Match, and calculate the number of d1, d2, d3, α differences in the remaining descriptors after transformation that are within the threshold range, and score the pose transformation matrix based on the number. The pose transformation matrix with the highest score is the coarse transformation matrix of the left filling port;

[0108] In this example, the rough transformation matrix of the left filling port is:

[0109]

[0110] Step 6: In the same way, perform the above steps on the right filling port to obtain the coarse transformation matrix of the right filling port. In this example, the coarse transformation matrix of the right filling port is:

[0111]

[0112] Step 7: Use the ICP algorithm to fine-align the coarse transformed pose to obtain the final accurate pose information of the tank truck filling port.

[0113] Step 1: Multiply the initial point cloud data by the corresponding coarse transformation matrix T ij_L , get the initial point cloud data source of the left filling port after the change L ;

[0114] Step 2: For source L For each point in the target point cloud, use the KD tree to find the point closest to it in the local point cloud of the left filling port to form a pair of matching points;

[0115] Step 3: Based on the matching point pairs, calculate the Euclidean distance between each matching point to evaluate the score of the current transformation matrix;

[0116] Step 4: Based on the transformation matrix score, iterative optimization is continuously performed so that after the transformation, the points in the initial point cloud can be closer to the corresponding points in the target point cloud, the left filling port local point cloud, thereby obtaining the left filling port fine registration transformation matrix. In this example, the left filling port fine registration transformation matrix is:

[0117]

[0118] Step 5: The coarse registration matrix T ij_Land the fine registration matrix T' ij_L Multiplying them together can get the change matrix T between the left filling port target point cloud and the initial point cloud L , in this example, T L for:

[0119]

[0120] Step 6: Apply the above steps to the local point cloud data of the right filling port to obtain the change matrix T between the target point cloud and the initial point cloud of the right filling port. R :

[0121]

[0122] Step 7: Transform the filling port center of the initial point cloud and the transformation matrix T R / L The final registration result and the center of the left and right filling ports can be obtained by multiplying them. The registration result is as follows Figure 6 As shown, in this example, the center of the left filler port is L and the center of the right filler port R for:

[0123]

[0124] Example 2

[0125] Embodiment 2 of the present invention provides a terminal device corresponding to the above-mentioned embodiment 1. The terminal device can be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0126] The terminal device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in the above-mentioned embodiment 1.

[0127] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage.

[0128] In other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors, which are not limited herein.

[0129] Example 3

[0130] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to the above-mentioned embodiment 1, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the above-mentioned embodiment 1 are implemented.

[0131] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0132] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0135] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0136] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for positioning the filling port of a methanol tanker, characterized in that: The following steps are involved: Collecting image data of a methanol tanker filling port, and using the image data to construct a sample dataset of a methanol tanker filling port; Using the sample data set to train a neural network model, and using the trained neural network model to roughly locate the methanol filling port to obtain the approximate positions of the centers of the two filling ports; The approximate pose information of the center points is obtained based on the approximate pose of the centers of the two filling ports. The initial point cloud collected by the depth camera is segmented based on the normal vector constraint to obtain the local point cloud data of the left and right filling ports respectively. Extract edge features from the local point cloud data of the left and right filling ports and the initial point cloud based on a longitude and latitude scanning method to obtain an edge feature point set; Constructing a CPO descriptor set of edge feature points of the target point cloud and the initial point cloud respectively according to the edge feature point set and the approximate position of the filling port center; The target point cloud and the initial point cloud are roughly registered according to the feature matching algorithm and similarity score to obtain the corresponding rough transformation pose matrix; The ICP algorithm is used to fine-register the rough transformation pose matrix to obtain the final accurate pose information of the tank truck filling port; The following processes are implemented on the edge feature point sets of the target point cloud and the initial point cloud respectively to construct the CPO descriptor sets of the edge feature points of the target point cloud and the initial point cloud respectively: The center of the filling port is roughly positioned to form a circular area so that the circular area contains the two-dimensional projection points of all edge feature points; Calculate the distance from the edge feature point to the center of the filling port to obtain the first part of the CPO descriptor; Project the edge feature points onto the circular surface and calculate the distance between the feature points and the projection points, that is, the vertical distance from the feature points to the circular surface, to obtain the second part of the CPO descriptor; According to the symmetry of the filling port, the approximate center of the filling port is moved a specified distance in the direction where the angle between the horizontal axis and the positive vertical axis is 45 degrees to obtain a new center point; Calculate the distance from the edge feature point to the new center point again to obtain the third part of the CPO descriptor; Calculate the angle between the projection point of the edge feature point, the original center point, and a point on the vertical axis to obtain the fourth part of the CPO descriptor.

2. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the approximate pose of the centers of the two filling ports includes: Use a depth camera to acquire real-time image data around the filling port of a methanol tanker truck, and enhance the collected image data based on CsNorm. The enhanced image data is fed into the trained neural network model for target detection, which outputs the two-dimensional pixel coordinates of the two filling ports. According to the image data collected by the depth camera, the depth value of the center of the filling port of the methanol tank truck is obtained through the affine function; The two-dimensional pixel coordinates are converted into two-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and depth value, so as to obtain the approximate position of the centers of the two filling ports.

3. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the local point cloud data of the left and right filling ports includes: The farthest plane and its normal vector of the initial point cloud are extracted based on the RANSAC fitting algorithm, and the initial point cloud is filtered according to the extracted farthest plane to obtain local point cloud data; Based on the obtained approximate position information of the center point and the physical model of the filling port, the approximate positions of the left and right filling ports are calculated in the local point cloud data and segmented to obtain the local point cloud data of the left and right filling ports, respectively.

4. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the edge feature point set includes: Project the target point cloud data and the initial point cloud data onto the xoy plane to obtain a two-dimensional data point set with changed coordinates; Sort the two-dimensional data point set according to the value of the horizontal coordinate and store the sorted points in a linked list; Starting from the minimum value of the horizontal coordinate, traverse the range of the horizontal coordinate with a certain step size dx; for each meridian, calculate the maximum and minimum values ​​of the meridian on the vertical coordinate, and record the indexes of the points corresponding to these values, and store the indexes in a new linked list; Similarly, calculate the maximum and minimum values ​​on the vertical axis and record the index of the corresponding point; After obtaining the indexes of all longitudes and latitudes, duplicate index values ​​are deleted, and the boundary points extracted from the target point cloud data and the initial point cloud data are obtained according to the longitude and latitude indexes to obtain the edge feature point set.

5. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The coarse transformation pose matrix acquisition process includes: A hash function is constructed using the fourth part of the CPO descriptor set as the key value, and a hash table of the initial point cloud is constructed using the CPO descriptor set as the target value; Traverse the target point cloud descriptor set, encode the value of the fourth part of the CPO descriptor set to form a query key value, query the hash table of the initial point cloud for the number of CPO descriptors whose key value difference is within a certain interval, score the target point cloud descriptors according to the number, and obtain a potential descriptor pair set; According to the descriptor scores of the target point cloud, the top M% descriptors are screened out to obtain a new set of descriptor pairs; Traverse the new set of descriptor pairs, and for each pair of descriptors, calculate the difference and sum of the first, second, and third parts of the descriptor, and retain the descriptor pairs whose differences of the first, second, and third parts are within the threshold and whose sum of differences is the smallest; Traverse the last set of retained descriptor pairs, calculate the pose transformation matrix between each pair of descriptors, and perform pose transformation on the remaining descriptor pairs according to the pose transformation matrix. Score the pose transformation matrix of the corresponding descriptor according to the difference between the first and third parts of the transformed descriptor and the number within the threshold. The pose transformation matrix with the highest score is the coarse transformation pose matrix.

6. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the final precise pose information of the tank truck filling port includes: For any point in the initial point cloud, use the KD tree to find the point closest to it in the target point cloud to form a pair of matching points; Calculate the Euclidean distance between each pair of matching points and evaluate the score of the current coarse transformation pose matrix; According to the score of the coarse transformation pose matrix, iterative optimization is continuously performed so that after the transformation, the Euclidean distance between the points in the initial point cloud and the corresponding points in the target point cloud is minimized, thereby obtaining the fine registration transformation matrix.

7. A terminal device comprising a memory, a processor, and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions; characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images