Methanol canning vehicle filling port pose positioning method, terminal equipment and storage medium

Abstract

The invention discloses a methanol canning vehicle filling port pose positioning method, terminal equipment and a storage medium, and the method comprises the steps: collecting an image and point cloud data of a methanol canning vehicle filling port through a depth camera, and carrying out the image enhancement of the collected image data based on CsNorm; acquiring rough poses of the centers of the left and right filling ports through a yov8 detection model; segmenting the initial point cloud based on normal vector constraint to obtain local point cloud data of the left and right filling ports; performing edge feature extraction on the local point clouds of the left and right filling ports based on a longitude and latitude scanning method to obtain an edge feature set; constructing a CPO descriptor of the edge feature set of the left and right filling ports; registering the target point cloud and the initial point cloud according to a feature matching algorithm and the similarity score to obtain a transformation pose matrix; and performing fine registration on the rough transformation poses by using an ICP (Inductively Coupled Plasma) algorithm to obtain the final accurate pose information of the left and right filling ports of the canning vehicle. According to the invention, the robustness of filling port detection in an environment with insufficient light or too strong light can be improved.

Description

Technical Field

[0001] The invention relates to the field of automatic filling of tank trucks, and in particular to a method for positioning a filling port of a methanol tank truck, a terminal device and a storage medium. Background Art

[0002] With the acceleration of industrialization, energy demand continues to grow. As an alternative energy source, methanol has gradually become more prominent in the energy system. As an important tool for methanol transportation, the rapid and accurate positioning of the filling port of the methanol tanker is of great significance to improving filling efficiency and safety.

[0003] However, the traditional positioning of the filling port of methanol tank trucks mostly relies on manual operation, which is not only inefficient but also has potential safety hazards. Especially at night or in environments with low visibility, the accuracy and safety of manual positioning are difficult to guarantee. What is more important is that methanol is toxic and corrosive to a certain extent, and manual operation may also pose a threat to the health of operators.

[0004] At present, there are few studies on the positioning of the filling port of methanol tank trucks, and there is no mature technical solution. Although some automation technologies have been applied to the positioning of the filling port, these methods often have large computational complexity, slow processing speed, low robustness, high requirements on image quality, and are easily affected by factors such as ambient light. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for positioning the filling port of a methanol tank truck, a terminal device and a storage medium in view of the shortcomings of the existing technology, so as to solve the problems of poor robustness, slow speed and great influence of light on the positioning of the methanol filling port.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for positioning the filling port of a methanol tank truck, comprising the following steps:

[0007] Collecting image data of a filling port of a methanol tank truck, and using the image data to construct a sample data set of a filling port of a methanol tank truck;

[0008] Using the sample data set to train a neural network model, using the trained neural network model to roughly locate the methanol filling port, and obtaining the approximate positions of the centers of the two filling ports;

[0009] The approximate pose information of the center point is obtained according to the approximate pose of the center of the two filling ports, and the initial point cloud collected by the depth camera is segmented based on the normal vector constraint to obtain the local point cloud data of the left and right filling ports respectively;

[0010] Extract edge features from the local point cloud data of the left and right filling ports and the initial point cloud based on the longitude and latitude scanning method to obtain an edge feature point set;

[0011] According to the edge feature point set and the approximate position and posture of the center of the filling port, respectively construct a CPO descriptor set of edge feature points of the target point cloud and the initial point cloud;

[0012] The target point cloud and the initial point cloud are roughly registered according to the feature matching algorithm and the similarity score to obtain the corresponding rough transformation pose matrix;

[0013] The ICP algorithm is used to finely align the coarse transformation pose matrix to obtain the final accurate pose information of the tank truck filling port.

[0014] The specific process of obtaining the approximate position of the centers of the two filling ports includes:

[0015] Use the depth camera to obtain image data around the filling port of the methanol tanker in real time, and enhance the collected image data based on CsNorm;

[0016] The enhanced image data is input into the trained neural network model for target detection, and the two-dimensional pixel coordinates of the two filling ports are output;

[0017] According to the image data collected by the depth camera, the depth value of the center of the filling port of the methanol tanker is obtained through the affine function;

[0018] The two-dimensional pixel coordinates are converted into two-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and the depth value, so as to obtain the approximate position and posture of the centers of the two filling ports.

[0019] The specific process of acquiring the local point cloud data of the left and right filling ports includes:

[0020] The farthest plane and its normal vector of the initial point cloud are extracted based on the RANSAC fitting algorithm, and the initial point cloud is filtered according to the extracted farthest plane to obtain local point cloud data;

[0021] According to the obtained approximate position information of the center point and the physical model of the filling port, the approximate positions of the left and right filling ports are calculated in the local point cloud data and segmented to obtain the local point cloud data of the left and right filling ports, respectively.

[0022] The specific process of obtaining the edge feature point set includes:

[0023] Project the target point cloud data and the initial point cloud data onto the xoy plane to obtain a two-dimensional data point set after the coordinates are changed;

[0024] Sort the two-dimensional data point set according to the value of the horizontal axis, and store the sorted points in a linked list;

[0025] Starting from the minimum value of the horizontal coordinate, traverse the range of the horizontal coordinate with a certain step length dx; for each meridian, calculate the maximum and minimum values ​​of the meridian on the vertical coordinate, and record the indexes of the points corresponding to these values, and store the indexes in a new linked list;

[0026] Similarly, calculate the maximum and minimum values ​​on the ordinate and record the indexes of the corresponding points;

[0027] After obtaining the indexes of all longitudes and latitudes, duplicate index values ​​are deleted, and the boundary points extracted from the target point cloud data and the initial point cloud data are obtained according to the longitude and latitude indexes to obtain a set of edge feature points.

[0028] The following processes are implemented for the edge feature point sets of the target point cloud and the initial point cloud, respectively, to construct the CPO descriptor sets of the edge feature points of the target point cloud and the initial point cloud, respectively:

[0029] The center of the filling port is roughly positioned to form a circular area, so that the circular area contains the two-dimensional projection points of all edge feature points;

[0030] Calculate the distance from the edge feature point to the center of the filling port to obtain the first part of the CPO descriptor;

[0031] Project the edge feature points onto the circular surface, and calculate the distance between the feature points and the projection points, that is, the vertical distance from the feature points to the circular surface, to obtain the second part of the CPO descriptor;

[0032] According to the symmetry of the filling port, the approximate center of the filling port is moved a specified distance in the direction where the angle between the horizontal axis and the positive direction of the vertical axis is 45 degrees to obtain a new center point;

[0033] Calculate the distance from the edge feature point to the new center point again to get the third part of the CPO descriptor;

[0034] The angle formed between the projection point of the edge feature point, the original center point, and a point on the vertical axis is calculated to obtain the fourth part of the CPO descriptor.

[0035] The coarse transformation pose matrix acquisition process includes:

[0036] A hash function is constructed using the fourth part of the CPO descriptor set as a key value, and a hash table of the initial point cloud is constructed using the CPO descriptor set as a target value;

[0037] Traverse the target point cloud descriptor set, encode the value of the fourth part of the CPO descriptor set to form a query key value, query the number of CPO descriptors with key value differences within a certain range in the hash table of the initial point cloud, score the target point cloud descriptors according to the number, and obtain a potential descriptor pair set;

[0038] According to the descriptor scores of the target point cloud, the top M% descriptors are screened out to obtain a new set of descriptor pairs;

[0039] Traverse the new set of descriptor pairs, and for each set of paired descriptors, calculate the difference and sum of the first, second and third parts of the descriptors, and retain the descriptor pairs whose differences of the first, second and third parts are within the threshold and whose sum of differences is the smallest;

[0040] Traverse the last set of descriptor pairs retained, calculate the pose transformation matrix between each pair of descriptors, and perform pose transformation on the remaining descriptor pairs according to the pose transformation matrix. Score the pose transformation matrix of the corresponding descriptor according to the difference between the first and third parts of the transformed descriptor and the number within the threshold. The pose transformation matrix with the highest score is the coarse transformation pose matrix.

[0041] The specific process of obtaining the precise position information of the tank truck filling port includes:

[0042] For any point in the initial point cloud, use the KD tree to find the point closest to it in the target point cloud to form a pair of matching points;

[0043] Calculate the Euclidean distance between each pair of matching points and evaluate the score of the current rough transformation pose matrix;

[0044] According to the score of the rough transformation pose matrix, iterative optimization is continuously performed so that after the transformation, the Euclidean distance between the points in the initial point cloud and the corresponding points in the target point cloud is the minimum initial point cloud, thereby obtaining the precise registration transformation matrix.

[0045] As an inventive concept, the present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the steps of the above method.

[0046] As an inventive concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon; the computer program / instruction implements the steps of the above method when executed by a processor.

[0047] As an inventive concept, the present invention also provides a computer program product, including a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the above method are implemented.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1) The present invention provides a fast and high-precision method for positioning the filling port of a methanol tank truck, which combines 3D point cloud, 2D image data and image processing technology to achieve accurate positioning of the filling port of a methanol tank truck, and can be applied to the field of automatic filling technology for methanol tank trucks.

[0050] 2) The present invention provides a fast and high-precision method for positioning the filling port of a methanol tank truck. According to the visual coarse positioning result and the physical model of the filling port, a point cloud segmentation algorithm based on normal vector constraints is designed to obtain local point cloud data of the filling port, which can reduce the interference caused by noise while removing irrelevant factors; at the same time, an edge feature extraction algorithm based on the longitude and latitude scanning method is designed, which reduces the amount of calculation while ensuring the feature information, and can speed up the subsequent point cloud alignment task.

[0051] 3) The present invention provides a fast and high-precision method for positioning the filling port of a methanol tank truck. According to the physical model characteristics of the filling port and the visual positioning results, a CPO descriptor specifically used for the filling port is designed. The descriptor can maintain high robustness in the case of missing local point cloud data, i.e., sensor errors caused by environmental factors (light). At the same time, a feature matching algorithm and a similarity scoring mechanism are designed for pairing between CPO descriptors, which can improve the success rate of alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 The present invention is an algorithm flow chart of the method for locating the center of the filling port of a methanol tanker;

[0053] Figure 2 This is a framework diagram of the central positioning system for the filling port of a methanol tanker truck implemented in the present invention;

[0054] Figure 3 It is a composition diagram of the CPO descriptor of the present invention;

[0055] Figure 4 This is the result diagram of the visual part detection;

[0056] Figure 5 This is the point cloud segmentation result image based on normal vector constraint;

[0057] Figure 6 This is the edge feature extraction result image based on the longitude and latitude scanning method;

[0058] Figure 7 This is the result of registering the left and right filling ports. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In this article, the terms "first", "second" and other similar words are not intended to imply any order, quantity and importance, but are only used to distinguish different elements. In this article, the terms "one", "an" and other similar words are not intended to indicate that there is only one of the things described, but rather indicate that the relevant description is only for one of the things described, and the things described may have one or more. In this article, the terms "comprise", "include" and other similar words are intended to indicate logical mutual relationships, and cannot be regarded as indicating spatial structural relationships. For example, "A includes B" is intended to indicate that B belongs to A logically, but does not indicate that B is located inside A spatially. In addition, the meanings of the terms "comprise", "include" and other similar words should be regarded as open, rather than closed. For example, "A includes B" is intended to indicate that B belongs to A, but B does not necessarily constitute the whole of A, and A may also include other elements such as C, D, and E.

[0061] Example 1

[0062] This embodiment provides an overall flow chart of a method for positioning the filling port of a methanol tanker quickly and accurately. Figure 1 As shown, the following steps are included:

[0063] Step 1: Use the depth camera to obtain real-time image data around the filling port of methanol tankers of different vehicles and at different times, make the collected image data into a methanol filling port dataset, and use the dataset for yolov8 detection model training, including the following steps:

[0064] Step 1: Use a depth camera equipped with a slide rail system to collect image data around the filling port of a methanol tanker of different vehicles and different angles on site;

[0065] Step 2: Enlarge and reduce the collected images of the filling box of the methanol tank truck to obtain a multi-scale image dataset;

[0066] Step 3: Use the LabelImg annotation tool to annotate the multi-scale image dataset and save it in a yolo trainable data format to obtain the methanol filling port dataset;

[0067] Step 4: Input the methanol filling port dataset into the yolov8 target detection network for training, obtain the optimal weights for network training, and save it as the optimal.pt file;

[0068] Step 2: Use the trained weight file to roughly locate the filling port posture of the on-site methanol tanker, including the following steps:

[0069] Step 1: Use the depth camera to collect image data around the filling port of the methanol tanker on site, and perform CsNorm image enhancement processing on it to avoid recognition errors caused by light:

[0070] Step 2: Input the enhanced image data into the trained yolov8 detection network, identify the left and right filling port positions, and save the two-dimensional pixel coordinate values ​​of their center positions as the center_2d.txt file. The detection results are as follows: Figure 4 As shown;

[0071] Step 3: Convert the 2D pixel coordinates of the filling port into coordinates in the camera coordinate system that can be recognized in the subsequent point cloud registration. In this example: Center tar_l =(-70.1778, 93.3569, 1061.51) and Center tar_r =(291.103, 100.141, 1065.17);

[0072] Step 3: Process the collected initial point cloud according to the approximate position of the center point of the filling port detected by visual inspection to obtain the local point cloud data of the left and right filling ports, including the following steps:

[0073] Step 1: After obtaining the approximate position of the center point of the filling port, use the depth camera to collect the point cloud data input.pcd around the filling port and perform filtering on it;

[0074] The process of obtaining the approximate position of the center point of the filling port is as follows:

[0075] The camera intrinsic parameters and the depth value of the filling port center are used to convert the two-dimensional pixel coordinate values ​​(u, v) of the two filling port centers into two-dimensional coordinates (x, y) in the camera coordinate system, and the approximate position (x) of the left filling port center is obtained. L ,y L ,z L ) and the approximate position of the center of the left filling port (x R ,y R ,z R ), the relevant formula is as follows:

[0076]

[0077] Among them, cx ,c y ,f x ,f y is the internal parameter of the camera.

[0078] Step 2: Segment the filtered point cloud data based on the normal vector constraint and the approximate position of the center point of the filling port to obtain the local point cloud data of the left and right filling ports;

[0079] Step 2.1: Repeatedly extract the farthest plane and its depth value z in the initial point cloud based on the RANSAC fitting algorithm p , compare the depth value extracted each time to ensure that the farthest plane is obtained, and calculate the average normal vector of the farthest plane. In this example, the average normal vector of the farthest plane is

[0080] Step 2.2, for each point p in the point cloud i , calculate the covariance matrix C of the point and its domain, perform eigendecomposition on the covariance matrix C, and obtain the minimum eigenvector Then calculate and The angle θ between them, and set the angle threshold θ max , if θ>θ max , then remove the point from the point cloud to obtain the local point cloud data of the filling box. In this example, the angle threshold θ max =5°, the relevant calculation formula is as follows:

[0081]

[0082] Step 2.3, based on the extracted farthest plane and its depth value z p The local point cloud data of the filling box is filtered to remove irrelevant point clouds and obtain the local point cloud data of the filling port.

[0083] Step 2.4, segment the local point cloud data of the filling port according to the approximate position of the center of the left and right filling ports and the physical model of the left and right filling ports {radius: r; filling port height: h}, and obtain the local point cloud data of the left and right filling ports, and calculate the average normal vector of the local point cloud data of the left and right filling ports respectively. and The segmentation results are as follows Figure 5 As shown, in this example

[0084] Step 4: extract edge features from the local point cloud data of the left and right filling ports and the initial point cloud based on the longitude and latitude scanning method to obtain an edge feature point set;

[0085] Step 1: Project the local point cloud data of the filling port onto the xoy plane, that is, remove the coordinate value in the z direction, and obtain the converted two-dimensional data point set {(x a ,y a ),(x b ,y b )…,(x n ,y n )};

[0086] Step 2: Sort the two-dimensional data point set according to the size of the x-coordinate and store the sorted x-coordinates in the list List sort {x1,x2…,x n};

[0087] Step 3: From the List sort Starting from the minimum value of the x coordinate, traverse the range of x coordinates and calculate the value of each x of the meridian i The corresponding maximum value of y max_i and the minimum value y min_i , and record the indexes corresponding to these points in a new list List record_x ;

[0088] Step 4: Use the same method to calculate the maximum and minimum values ​​of the y coordinates on the latitude line, and record the indexes of the corresponding points to get List record_y ;

[0089] Step 5: Traverse the List record_x and List record_y The index value in , deletes the duplicate values, and extracts the edge feature set Edge of the local point cloud data of the left and right filling ports of the target point cloud according to the index value target {point1,point2…,point k}where k is the number of initial point cloud feature set points;

[0090] Step 6: Implement the above steps on the local point cloud data of the left and right filling ports of the initial point cloud to obtain the edge feature set Edge of the left and right filling ports of the initial point cloud source {point1,point2…,point L}, where L is the number of initial point cloud feature set points, Figure 6 This is the edge feature extraction result diagram of the local point cloud data of the right filling port and the initial point cloud.

[0091] Step 5: According to the approximate position of the filling port center, construct the CPO descriptor of the edge feature set of the target point cloud and the initial point cloud;

[0092] Step 1: Center the filler port tar / sou_l / ris the center of the plane, Construct a plane for the normal vector so that the plane can contain the two-dimensional projection point of the edge feature point.

[0093] Step 2: Calculate the edge feature set Edge target / source Point to Center tar_l The distance d1 is obtained to obtain the first part of the CPO descriptor. The relevant calculation formula is:

[0094]

[0095] Step 3: Put the point P(x P ,y P ,z P ) is mapped onto the plane, and the projection point B(x B ,y B ,z B ), and calculate the distance d2 between the feature point and the projection point, that is, the distance from the feature point to the plane. The relevant calculation formula is as follows:

[0096]

[0097] Step 4: Determine the positive direction of the x-axis and y-axis according to the changes in the x and y coordinates, and move the center of the filling port to a specified distance in the direction of the positive direction of the x-axis and y-axis at an angle of 45 degrees to obtain a new center point C1 (x c1 ,y c1 ,z c1 ).

[0098] Step 5: Calculate the distance from the edge feature point to the new center point C1(x c1 ,y c1 ,z c1 ) and obtain the third part of the CPO descriptor.

[0099] Step 6: Calculate the projection point B(x B ,y B ,z B ), the original center point C(x C ,y C ,z C ), a point F(x F ,y F ,z F ) to form an angle α between them, and the fourth part of the descriptor is obtained. The relevant calculation formula is as follows:

[0100]

[0101] Step 7: Implement steps 1 to 6 on the edge feature sets of the left and right filling ports of the target point cloud and the initial point cloud respectively, and construct the CPO descriptors CPO of the left and right filling ports of the target point cloud and the initial point cloud target_l 、CPO source_l 、CPO target_r and CPO source_r .

[0102] Step 6: Roughly align the target point cloud and the initial point cloud according to the feature matching algorithm and similarity score to obtain the corresponding rough transformation pose matrix;

[0103] Step 1: Use α in the CPO descriptor of the left filling port edge feature set of the initial point cloud as the key value, CPO source_l Construct the hash table Hash of the initial point cloud for the target value source_l .

[0104] Step 2: Traverse the target point cloud left filling port descriptor set CPO target_l , extract CPO target_l All α in the collection and encode this part to get query target_l {q1,q2,…q m}, in the hash table Hash source_l Query the difference in the interval (q i -5°,q i +5°) The sum of the number of matching descriptors within L , and obtain a set of potential matching pairs

[0105] Step 3: Get the score of the target point cloud according to the number of matching descriptors L , select the top 20% of descriptors to enter the next stage for feature association, and obtain a new descriptor pair set Pairs{pairs1,pairs2,…pairs k};

[0106] Step 4: Traverse the descriptor pair set Pairs{pairs1,pairs2,…pairs k ,}, for pairs i The descriptor of each pair in the set Calculate the difference and sum of d1, d2, d3, and finally keep the difference of d1, d2, d3 in the interval (d i -2,d i +2) and the difference and the minimum corresponding descriptor pair, and obtain the final retained descriptor pair set Match{(CPO target_1 ,CPO source_1 ),(CPO target_2 ,CPOsource_2 ),…(CPO target_k ,CPO source_k )}

[0107] Step 5: Traverse the last retained descriptor pair set Match and calculate the pose transformation T between each pair of descriptors ij (R ij ,t ij ), use the pose transformation to transform the remaining descriptor pairs in Match, and calculate the number of differences of d1, d2, d3, and α in the remaining descriptors after the transformation that are within the threshold range, and score the pose transformation matrix according to the number. The pose transformation matrix with the highest score is the rough transformation matrix of the left filling port;

[0108] In this example, the rough transformation matrix of the left filling port is:

[0109]

[0110] Step 6: In the same way, the above steps are performed on the right filling port to obtain the rough transformation matrix of the right filling port. In this example, the rough transformation matrix of the right filling port is:

[0111]

[0112] Step 7: Use the ICP algorithm to fine-align the coarsely transformed pose to obtain the precise pose information of the tank truck filling port.

[0113] Step 1: Multiply the initial point cloud data by the corresponding coarse transformation matrix T ij_L , get the initial point cloud data source of the left filling port after the change L ;

[0114] Step 2: For source L For each point in the target point cloud, use the KD tree to find the point closest to it in the local point cloud of the left filling port to form a pair of matching points;

[0115] Step 3: Based on the matching point pairs, calculate the Euclidean distance between each matching point to evaluate the score of the current transformation matrix;

[0116] Step 4: Iterate and optimize continuously according to the score of the transformation matrix, so that after the transformation, the points in the initial point cloud can be closer to the corresponding points in the local point cloud of the left filling port of the target point cloud, thereby obtaining the left filling port precise registration transformation matrix. In this example, the precise registration transformation matrix of the left filling port is:

[0117]

[0118] Step 5: The rough registration matrix T ij_Land the fine registration matrix T' ij_L Multiplying them together will give the change matrix T between the left filling port target point cloud and the initial point cloud L In this example, T L for:

[0119]

[0120] Step 6: Apply the above steps to the local point cloud data of the right filling port to obtain the change matrix T between the target point cloud and the initial point cloud of the right filling port R :

[0121]

[0122] Step 7: Substitute the filling port center of the initial point cloud and the transformation matrix T R / L The final registration result and the center of the left and right filling ports can be obtained by multiplying them. The registration result is as follows Figure 6 As shown, in this example, the center of the left filling port is L and right filler port center R for:

[0123]

[0124] Example 2

[0125] Embodiment 2 of the present invention provides a terminal device corresponding to the above-mentioned embodiment 1. The terminal device may be a processing device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute the method of the above-mentioned embodiment.

[0126] The terminal device of this embodiment includes a memory, a processor, and a computer program stored in the memory; the processor executes the computer program in the memory to implement the steps of the method in the above-mentioned embodiment 1.

[0127] In some implementations, the memory may be a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0128] In some other implementations, the processor may be a central processing unit (CPU), a digital signal processor (DSP), or other general-purpose processors of various types, which are not limited herein.

[0129] Example 3

[0130] Embodiment 3 of the present invention provides a computer-readable storage medium corresponding to the above embodiment 1, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the above embodiment 1 are implemented.

[0131] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0132] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0135] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0136] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for positioning the filling port of a methanol tanker, characterized in that: The following steps are involved: Collecting image data of a filling port of a methanol tank truck, and using the image data to construct a sample data set of a filling port of a methanol tank truck; Using the sample data set to train a neural network model, using the trained neural network model to roughly locate the methanol filling port, and obtaining the approximate positions of the centers of the two filling ports; The approximate pose information of the center point is obtained according to the approximate pose of the center of the two filling ports, and the initial point cloud collected by the depth camera is segmented based on the normal vector constraint to obtain the local point cloud data of the left and right filling ports respectively; Extract edge features from the local point cloud data of the left and right filling ports and the initial point cloud based on the longitude and latitude scanning method to obtain an edge feature point set; According to the edge feature point set and the approximate position and posture of the center of the filling port, respectively construct a CPO descriptor set of edge feature points of the target point cloud and the initial point cloud; The target point cloud and the initial point cloud are roughly registered according to the feature matching algorithm and the similarity score to obtain the corresponding rough transformation pose matrix; The ICP algorithm is used to finely align the coarse transformation pose matrix to obtain the final accurate pose information of the tank truck filling port.

2. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the approximate position of the centers of the two filling ports includes: Use the depth camera to obtain image data around the filling port of the methanol tanker in real time, and enhance the collected image data based on CsNorm; The enhanced image data is input into the trained neural network model for target detection, and the two-dimensional pixel coordinates of the two filling ports are output; According to the image data collected by the depth camera, the depth value of the center of the filling port of the methanol tanker is obtained through the affine function; The two-dimensional pixel coordinates are converted into two-dimensional coordinates in the camera coordinate system through the camera intrinsic parameter matrix and the depth value, so as to obtain the approximate position and posture of the centers of the two filling ports.

3. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of acquiring the local point cloud data of the left and right filling ports includes: The farthest plane and its normal vector of the initial point cloud are extracted based on the RANSAC fitting algorithm, and the initial point cloud is filtered according to the extracted farthest plane to obtain local point cloud data; According to the obtained approximate position information of the center point and the physical model of the filling port, the approximate positions of the left and right filling ports are calculated in the local point cloud data and segmented to obtain the local point cloud data of the left and right filling ports, respectively.

4. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the edge feature point set includes: Project the target point cloud data and the initial point cloud data onto the xoy plane to obtain a two-dimensional data point set after the coordinates are changed; Sort the two-dimensional data point set according to the value of the horizontal axis, and store the sorted points in a linked list; Starting from the minimum value of the horizontal coordinate, traverse the range of the horizontal coordinate with a certain step length dx; for each meridian, calculate the maximum and minimum values ​​of the meridian on the vertical coordinate, and record the indexes of the points corresponding to these values, and store the indexes in a new linked list; Similarly, calculate the maximum and minimum values ​​on the ordinate and record the indexes of the corresponding points; After obtaining the indexes of all longitudes and latitudes, duplicate index values ​​are deleted, and the boundary points extracted from the target point cloud data and the initial point cloud data are obtained according to the longitude and latitude indexes to obtain a set of edge feature points.

5. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The following processes are implemented for the edge feature point sets of the target point cloud and the initial point cloud, respectively, to construct the CPO descriptor sets of the edge feature points of the target point cloud and the initial point cloud, respectively: The center of the filling port is roughly positioned to form a circular area, so that the circular area contains the two-dimensional projection points of all edge feature points; Calculate the distance from the edge feature point to the center of the filling port to obtain the first part of the CPO descriptor; Project the edge feature points onto the circular surface, and calculate the distance between the feature points and the projection points, that is, the vertical distance from the feature points to the circular surface, to obtain the second part of the CPO descriptor; According to the symmetry of the filling port, the approximate center of the filling port is moved a specified distance in the direction where the angle between the horizontal axis and the positive direction of the vertical axis is 45 degrees to obtain a new center point; Calculate the distance from the edge feature point to the new center point again to get the third part of the CPO descriptor; The angle formed between the projection point of the edge feature point, the original center point, and a point on the vertical axis is calculated to obtain the fourth part of the CPO descriptor.

6. The method for positioning the filling port of a methanol tanker according to claim 5, characterized in that: The coarse transformation pose matrix acquisition process includes: A hash function is constructed using the fourth part of the CPO descriptor set as a key value, and a hash table of the initial point cloud is constructed using the CPO descriptor set as a target value; Traverse the target point cloud descriptor set, encode the value of the fourth part of the CPO descriptor set to form a query key value, query the number of CPO descriptors with key value differences within a certain interval in the hash table of the initial point cloud, score the target point cloud descriptors according to the number, and obtain a potential descriptor pair set; according to the descriptor scores of the target point cloud, filter out the top M% of descriptors to obtain a new descriptor pair set; Traverse the new set of descriptor pairs, and for each set of paired descriptors, calculate the difference and sum of the first, second and third parts of the descriptors, and retain the descriptor pairs whose differences of the first, second and third parts are within the threshold and whose sum of differences is the smallest; Traverse the last set of descriptor pairs retained, calculate the pose transformation matrix between each pair of descriptors, and perform pose transformation on the remaining descriptor pairs according to the pose transformation matrix. Score the pose transformation matrix of the corresponding descriptor according to the difference between the first and third parts of the transformed descriptor and the number within the threshold. The pose transformation matrix with the highest score is the coarse transformation pose matrix.

7. The method for positioning the filling port of a methanol tanker according to claim 1, characterized in that: The specific process of obtaining the precise position information of the tank truck filling port includes: For any point in the initial point cloud, use the KD tree to find the point closest to it in the target point cloud to form a pair of matching points; Calculate the Euclidean distance between each pair of matching points and evaluate the score of the current rough transformation pose matrix; According to the score of the rough transformation pose matrix, iterative optimization is continuously performed so that after the transformation, the Euclidean distance between the points in the initial point cloud and the corresponding points in the target point cloud is minimized, thereby obtaining the precise registration transformation matrix.

8. A terminal device comprising a memory, a processor and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions; characterized in that When the computer program / instructions are executed by a processor, the steps of the method of any one of claims 1 to 7 are implemented.

Images