Three-dimensional landmark recognition method, system, storage medium and terminal

Abstract

The present invention discloses a three-dimensional marker point recognition method, system, storage medium and terminal, the method comprising: collecting a three-dimensional point cloud, a grayscale image and structural parameters of a marker point for three-dimensional marker point recognition; segmenting the grayscale image according to a first segmentation method and a second segmentation method respectively and obtaining first grayscale image data and second grayscale image data; performing an OR operation based on the first grayscale image data and the second grayscale image data to generate a binary image; utilizing a clustering algorithm to segment mutually unconnected regions in the binary image into different independent sets; performing feature discrimination on multiple target feature discrimination items in each independent set respectively to generate multiple discrimination results; generating a confidence level of a marker point recognition result by weighted calculation of the multiple discrimination results; and calculating the center coordinates of the marker point using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point based on the confidence level satisfying preset requirements.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional marker point recognition technology, and in particular to a three-dimensional marker point recognition method, system, storage medium and terminal. Background Art

[0002] 3D landmark recognition technology is widely used in areas such as road sign recognition, aircraft parts processing and inspection, automotive component process testing, and human posture recognition. Currently, 3D landmark recognition methods include those based on point cloud depth information, those based on grayscale images, and those based on machine learning. These methods have been widely applied in various fields.

[0003] During the inspection and processing of curved plates, three-dimensional landmark point recognition technology is required to locate the rib position. However, the special application environment and the harsh working environment will cause complex problems of background noise and grayscale features on the surface of the curved plate. The landmark point recognition method based on point cloud depth information is prone to losing the depth information of the landmark point, resulting in low recognition reliability; the three-dimensional landmark point recognition method based on grayscale images will result in low recognition accuracy when identifying curved plate landmark points; the method based on machine learning not only has high computational complexity, but also has generalization performance that is difficult to meet the application requirements for high recognition reliability. It can be seen that the existing three-dimensional landmark point recognition method is difficult to meet the application requirements for fast and reliable recognition of landmark points on the surface of curved plates in complex environments. Summary of the Invention

[0004] The purpose of the present invention is to provide a three-dimensional landmark point recognition method, system, storage medium and terminal to solve the problems of low reliability, low accuracy, complex calculation and resource consumption in the existing technology when identifying landmark points on the surface of a curved plate in a complex environment.

[0005] The above-mentioned purpose of the present invention can be achieved by adopting the following technical solutions:

[0006] The present invention provides a three-dimensional landmark point recognition method, comprising:

[0007] Collect 3D point clouds, grayscale images and structural parameters of landmark points for 3D landmark point recognition;

[0008] Segment the grayscale image according to a first segmentation method and a second segmentation method respectively and obtain first grayscale image data and second grayscale image data;

[0009] Performing an OR operation on the first grayscale image data and the second grayscale image data to generate a binary image;

[0010] Using a clustering algorithm to segment mutually disconnected regions in the binary image into different independent sets;

[0011] Performing feature discrimination on the multiple target feature discrimination items in each of the independent sets to generate multiple discrimination results;

[0012] Generating a confidence level of a landmark point recognition result by weighted calculation of the plurality of discrimination results;

[0013] Based on the confidence level meeting a preset requirement, the center coordinates of the marker point are calculated using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point.

[0014] Preferably, segmenting the grayscale image according to the first segmentation method and obtaining first grayscale image data includes:

[0015] Calculating a first grayscale mean value of the grayscale image;

[0016] Calculating, based on the first grayscale mean, a second grayscale mean of the grayscale image having a grayscale greater than the first grayscale mean;

[0017] Generate a global grayscale threshold by weighted calculation based on the first grayscale mean and the second grayscale mean;

[0018] The grayscale image is segmented using the global grayscale threshold and the first grayscale image data whose grayscale values ​​meet a first condition is retained.

[0019] Preferably, segmenting the grayscale image according to the second segmentation method and obtaining second grayscale image data includes:

[0020] Dividing the grayscale image into a plurality of sub-blocks according to a preset scheme;

[0021] Each of the sub-blocks is segmented according to a preset grayscale threshold and the second grayscale image data whose grayscale values ​​in the grayscale image meet a second condition is retained.

[0022] Preferably, the size of the long side and the short side of each sub-block is 32 pixels to 64 pixels.

[0023] Preferably, the step of calculating the center coordinates of the marker point using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point comprises:

[0024] Extracting point cloud data located on the upper surface of the marker point from the three-dimensional point cloud to fit the upper surface plane of the marker point;

[0025] Based on the parameter information of the upper surface plane, generating a normal vector corresponding thereto;

[0026] Processing the three-dimensional point cloud within the distance range of the marker point and the target point so that the upper surface plane is parallel to the preset xoy plane;

[0027] Calculate the height difference between each data point within the target distance range and the center of the marker point, and count the number of data points whose height difference meets the preset requirements;

[0028] Based on the number of data points being greater than a preset value, the mark point is confirmed as a target mark point;

[0029] The center coordinates of the marker point are generated based on the upper surface parameter information of the target marker point, the normal vector and the structural parameters of the marker point.

[0030] Preferably, the multiple target feature discrimination items include at least the number of effective points, size, symmetry, perimeter and shape.

[0031] Preferably, the symmetry feature identification includes determining whether the number of valid points in the four quadrants is greater than 1 / 6 of the total number of valid points.

[0032] The present invention also provides a system, which includes: a module for executing any one of the aforementioned three-dimensional landmark point recognition methods.

[0033] The present invention also provides a storage medium having a computer program stored thereon, wherein the storage medium is a computer-readable storage medium, and when the program is executed, any one of the aforementioned three-dimensional landmark point recognition methods is implemented.

[0034] The present invention also provides a terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the aforementioned three-dimensional landmark point recognition methods when executing the computer program.

[0035] The present invention has at least the following characteristics and advantages:

[0036] 1. The present invention achieves reliable recognition of landmarks on large-scale curved plates in complex environments by distinguishing multiple scale features of grayscale image data and combining it with depth map verification, thereby improving the recognition accuracy of three-dimensional landmarks in complex environments.

[0037] 2. The present invention utilizes conventional image processing operations and numerical operations to perform data screening and feature discrimination, which has low computational complexity and improves computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 3D landmark recognition method of the present invention;

[0040] Figure 2 3D landmark recognition method of the present invention;

[0041] Figure 3 3D landmark recognition method of the present invention;

[0042] Figure 4 3D landmark recognition method of the present invention;

[0043] Figure 5 Flowchart of the three-dimensional landmark recognition method of the present invention;

[0044] Figure 6 It is a structural schematic diagram of the marker point of the present invention;

[0045] Figure 7 It is a structural block diagram of the terminal of the present invention.

[0046] Reference numerals and descriptions:

[0047] 10000, terminal; 11000, memory; 11100, computer program; 12000, processor. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0049] Implementation Method 1

[0050] The present invention provides a three-dimensional landmark recognition method, see Figures 1 to 6 , including the following steps:

[0051] S1. Collecting 3D point clouds, grayscale images, and structural parameters of landmark points for 3D landmark point recognition;

[0052] S2. Segment the grayscale image according to the first segmentation method and the second segmentation method respectively and obtain first grayscale image data and second grayscale image data;

[0053] S3, performing an OR operation on the first grayscale image data and the second grayscale image data to generate a binary image;

[0054] S4. Use clustering algorithm to divide the mutually disconnected areas in the binary graph into different independent sets;

[0055] S5, performing feature discrimination on the multiple target feature discrimination items in each independent set to generate multiple discrimination results;

[0056] In some embodiments, the multiple target feature discrimination items include at least the number of valid points, size, symmetry, perimeter, and shape. Furthermore, when performing symmetry feature discrimination, the item includes determining whether the number of valid points in the four quadrants is greater than 1 / 6 of the total number of valid points. In this embodiment, a valid point refers to a valid pixel with a value of 1 in a binary image.

[0057] S6. Generate confidence of the landmark recognition result by weighted calculation of multiple discrimination results;

[0058] S7. Based on the confidence level meeting the preset requirements, the center coordinates of the marker point are calculated using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point.

[0059] In some embodiments, see Figure 2 Segmenting the grayscale image according to the first segmentation method and obtaining first grayscale image data includes:

[0060] S21, calculating a first grayscale mean value of the grayscale image;

[0061] S22. Based on the first grayscale mean, calculate a second grayscale mean in the grayscale image whose grayscale is greater than the first grayscale mean;

[0062] S23, generating a global grayscale threshold value based on the weighted calculation of the first grayscale mean and the second grayscale mean;

[0063] S24 , performing grayscale segmentation on the grayscale image using a global grayscale threshold and retaining first grayscale image data in the grayscale image whose grayscale value meets a first condition.

[0064] In some embodiments, see Figure 3 Segmenting the grayscale image according to the second segmentation method and obtaining second grayscale image data includes:

[0065] S25, dividing the grayscale image into multiple sub-blocks according to a preset scheme;

[0066] In some embodiments, the size of the long side and the short side of each sub-block is 32 pixels to 64 pixels.

[0067] S26 , segmenting each sub-block according to a preset grayscale threshold and retaining second grayscale image data in which grayscale values ​​meet a second condition.

[0068] In some embodiments, see Figure 4, using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point to calculate the center coordinates of the marker point includes:

[0069] S71, extracting point cloud data located on the upper surface of the marker point from the three-dimensional point cloud to fit the upper surface plane of the marker point;

[0070] S72. Generate a normal vector corresponding to the upper surface plane based on parameter information thereof;

[0071] S73, processing the three-dimensional point cloud within the distance range of the marker point and the target point so that the upper surface plane is parallel to the preset xoy plane;

[0072] S74, calculating the height difference between each data point within the target distance range and the center of the marker point, and counting the number of data points whose height differences meet the preset requirements;

[0073] S75, based on the number of data points being greater than a preset value, confirming the mark point as a target mark point;

[0074] S76. Generate the center coordinates of the marker point based on the upper surface parameter information, the normal vector and the structural parameters of the marker point.

[0075] The present invention has at least the following characteristics and advantages:

[0076] 1. The present invention achieves reliable recognition of landmarks on large-scale curved plates in complex environments by distinguishing multiple scale features of grayscale image data and combining it with depth map verification, thereby improving the recognition accuracy of three-dimensional landmarks in complex environments.

[0077] 2. The present invention utilizes conventional image processing operations and numerical operations to perform data screening and feature discrimination, which has low computational complexity and improves computational efficiency.

[0078] The present invention is further studied below through a specific embodiment. Figures 1 to 6 , including the following steps:

[0079] Step 1: Input data that can be used for landmark recognition, including 3D point cloud, grayscale image, and structural parameters of landmark points;

[0080] Among them, the structural features of the marking point include: 1) the upper surface of the marking point is circular with a diameter of 50mm; 2) the upper surface is 30mm above the steel plate surface; 3) the diameter of the central circular hole is 2mm, etc. Figure 6 shown.

[0081] Step 2: Calculate the grayscale mean H1 of the grayscale image, and then calculate the mean H2 of the grayscale data greater than H1. Weight H1 and H2 to obtain the global grayscale threshold for grayscale segmentation. Utilizing the fact that most grayscale values ​​in the entire grayscale image are lower than the grayscale values ​​of the landmark points, most non-marker point areas are removed, and the grayscale image data that meets the global grayscale threshold is retained.

[0082] Step 3: Divide the grayscale image into M*N blocks. Perform grayscale threshold segmentation on each grayscale image. Taking advantage of the high grayscale value of the surface of the local marker point, remove the non-marker point area around the marker point and retain the grayscale image data with higher local grayscale value.

[0083] Step 4: OR the grayscale images obtained in steps 2 and 3 to obtain a binary image. Use a clustering algorithm to segment the disconnected regions in the binary image into independent sets to facilitate feature identification in the next step.

[0084] Step 5: Perform landmark feature discrimination on each independent set, that is, calculate the following discriminant items for each set:

[0085] 1) Valid point quantity determination: Count the valid point data in the area and determine whether it meets the data volume range. (The purpose is to remove areas with a small number of discrete points and large data volumes);

[0086] 2) Size determination: calculate the row and column values ​​occupied and determine whether they meet the row and column value range;

[0087] 3) Symmetry judgment: calculate whether the valid points in the four quadrants are greater than (total number of points / 6) to determine whether it is symmetrical; in some preferred embodiments, calculate whether the valid points in the four quadrants are greater than (total number of points / 4) to determine whether it is symmetrical.

[0088] 4) Perimeter discrimination: Calculate the perimeter, determine whether it meets the threshold, and remove other shapes;

[0089] 5) Shape identification: circle identification based on the ratio of perimeter to area;

[0090] Step 6: Weight the results of each discriminant item in step 5 to obtain the confidence level of the landmark point recognition result. If the confidence level is greater than 90%, it is considered to be a landmark point.

[0091] Step 7: Extract the upper surface point cloud data of the marker point and fit it to the plane of the upper surface to obtain the upper surface normal vector. Rotate the marker point and the adjacent area until the upper surface of the marker point is parallel to the xoy plane.

[0092] Step 8: Calculate the Z value difference between the points around the marker and the center of the marker, and count the number of data with a height difference between 20 and 40. If the threshold is met, it is considered to be a marker.

[0093] Step 9: Calculate the center coordinates of the upper surface of the marker point, and according to the direction of the upper surface normal vector, translate 30mm toward the steel plate to obtain the corresponding coordinates on the steel plate.

[0094] The method of the present invention uses global and block grayscale threshold segmentation to roughly identify the marker point area. It then uses cluster segmentation and multi-scale feature recognition of the marker point to accurately extract the marker point area. The depth map information is then used to verify the marker point features and extract the marker point data. The coordinates of the center of the marker point circle are calculated and projected onto the surface of the steel plate to obtain the coordinates of the center of the circle mapped to the steel plate surface, thereby realizing the recognition of the three-dimensional marker point. The present invention can achieve rapid and reliable three-dimensional marker point recognition of measured point cloud data of large-scale curved plates in complex environments, thereby improving the recognition accuracy.

[0095] Implementation Method 2

[0096] The embodiment of the present invention further provides a system, which includes a module for executing the steps of the method in any one of the embodiments in Implementation Method 1. Those skilled in the art will appreciate that the system provided by the present invention has the same beneficial effects as the embodiments in Implementation Method 1, and will not be described in detail here.

[0097] Implementation Method 3

[0098] The embodiment of the present invention further provides a storage medium having a computer program 11100 stored thereon. The storage medium is a computer-readable storage medium, and when the program is executed by the processor 12000, the steps of the method of any embodiment in Implementation Method 1 are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data. The specific execution process can be found in the detailed description of the method embodiment in Implementation Method 1, and will not be repeated here.

[0099] Those skilled in the art should understand that the storage medium provided by the present invention has the same beneficial effects as the embodiment in Implementation Method 1, and will not be described in detail here.

[0100] Implementation Method 4

[0101] The embodiment of the present invention further provides a terminal 10000, see Figure 7, including a memory 11000, a processor 12000, and a computer program 11100 stored in the memory 11000 and executable on the processor 12000. When the processor 12000 executes the computer program 11100, the method of any one of the embodiments in Implementation Method 1 is implemented. The specific implementation process can be found in the detailed description of the above-mentioned method embodiments and will not be described in detail here.

[0102] In the embodiment of the present invention, the processor 12000 is the control center of the computer system and can be the processor of a physical machine or a virtual machine. In the embodiment of the present invention, the memory 11000 stores at least one instruction, which is loaded and executed by the processor 12000 to implement the methods in the above embodiments.

[0103] In one embodiment of the present invention, the processor 12000 may include one or more processing cores, such as a quad-core processor or an octal-core processor. The processor 12000 may be implemented using at least one of the following hardware forms: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 12000 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0104] Memory 11000 may include one or more computer-readable storage media, which may be non-transitory. Memory 11000 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments of the present invention, the non-transitory computer-readable storage medium in memory 11000 is used to store at least one instruction, which is used to be executed by processor 12000 to implement the method in the embodiment of the present invention.

[0105] Those skilled in the art should understand that the terminal 10000 provided by the present invention has the same beneficial effects as the embodiment in Implementation Method 1, and will not be described in detail here.

[0106] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above in terms of preferred embodiments, they are not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A three-dimensional landmark point recognition method, characterized in that: include: Collect 3D point clouds, grayscale images and structural parameters of landmark points for 3D landmark point recognition; Segment the grayscale image according to a first segmentation method and a second segmentation method respectively and obtain first grayscale image data and second grayscale image data; Performing an OR operation on the first grayscale image data and the second grayscale image data to generate a binary image; Using a clustering algorithm to segment mutually disconnected regions in the binary image into different independent sets; Performing feature discrimination on the multiple target feature discrimination items in each of the independent sets to generate multiple discrimination results; Generating a confidence level of a landmark point recognition result by weighted calculation of the plurality of discrimination results; Based on the confidence level meeting a preset requirement, calculating the center coordinates of the marker point using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point; Segmenting the grayscale image according to the first segmentation method and obtaining first grayscale image data includes: Calculating a first grayscale mean value of the grayscale image; Calculating, based on the first grayscale mean, a second grayscale mean of the grayscale image having a grayscale greater than the first grayscale mean; Generate a global grayscale threshold by weighted calculation based on the first grayscale mean and the second grayscale mean; Performing grayscale segmentation on the grayscale image using the global grayscale threshold and retaining the first grayscale image data whose grayscale values ​​in the grayscale image meet a first condition; and segmenting the grayscale image according to a second segmentation method and obtaining second grayscale image data includes: Dividing the grayscale image into a plurality of sub-blocks according to a preset scheme; Each of the sub-blocks is segmented according to a preset grayscale threshold and the second grayscale image data whose grayscale values ​​in the grayscale image meet a second condition is retained.

2. The three-dimensional landmark point recognition method according to claim 1, characterized in that: The sizes of the long side and the short side of each sub-block are both 32 pixels to 64 pixels.

3. The three-dimensional landmark recognition method according to claim 1, characterized in that: Calculating the center coordinates of the marker point using the three-dimensional point cloud corresponding to the marker point and the structural parameters of the marker point includes: Extracting point cloud data located on the upper surface of the marker point from the three-dimensional point cloud to fit the upper surface plane of the marker point; Based on the parameter information of the upper surface plane, generating a normal vector corresponding thereto; Processing the three-dimensional point cloud within the distance range of the marker point and the target point so that the upper surface plane is parallel to the preset xoy plane; Calculate the height difference between each data point within the target distance range and the center of the marker point, and count the number of data points whose height difference meets the preset requirements; Based on the number of data points being greater than a preset value, the marker point is confirmed as a target marker point; The center coordinates of the target marker point are generated based on the upper surface parameter information of the target marker point, the normal vector and the structural parameters of the marker point.

4. The three-dimensional landmark recognition method according to any one of claims 1 to 3, characterized in that: The multiple target feature discrimination items include at least the number of effective points, size, symmetry, perimeter and shape.

5. The three-dimensional landmark recognition method according to claim 4, characterized in that: When performing symmetry feature judgment, it includes judging whether the number of valid points in the four quadrants is greater than 1 / 6 of the total number of valid points.

6. A three-dimensional landmark point recognition system, characterized in that: include: A module for executing the three-dimensional landmark point recognition method according to any one of claims 1 to 5.

7. A storage medium having a computer program stored thereon, characterized in that: The storage medium is a computer-readable storage medium, and when the program is executed, the three-dimensional landmark point recognition method according to any one of claims 1 to 5 is implemented.

8. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the three-dimensional landmark point recognition method according to any one of claims 1 to 5 is implemented.

Images