A Countersunk Hole Measurement Method Based on Visual Point Cloud Fusion Analysis

Abstract

This invention discloses a countersunk hole measurement method based on visual point cloud fusion analysis, comprising: Step 1, using a binocular structured light measurement device to photograph the countersunk product, obtaining two-dimensional visual image data and three-dimensional point cloud data of the countersunk product in the camera coordinate system; Step 2, identifying the inner and outer circular contours of the countersunk hole based on the two-dimensional visual image data of the countersunk product, and recording the contour pixel coordinates; Step 3, based on the inner and outer circular contour pixel coordinates, automatically dividing the three-dimensional point cloud data into the surface area of ​​the countersunk product, the countersunk cone surface area, and the interface area between the countersunk cone surface and the inner diameter surface; Fourth, for the point cloud data within the surface area of ​​the drilled product, calculate a first-order plane function of the drilled product surface to characterize the theoretical plane of the drilled product surface; Fifth, for the point cloud data within the countersunk cone surface area, fit a surface function of the countersunk cone surface to characterize the theoretical surface of the countersunk cone surface; Sixth, calculate the intersection line of the two theoretical surfaces, where the diameter of the intersection line is the countersunk diameter; Seventh, based on the point cloud data of the area where the countersunk cone surface intersects with the inner diameter surface and the theoretical plane of the drilled product surface, after filtering and denoising the point cloud data of the area where the countersunk cone surface intersects with the inner diameter surface, calculate the countersunk depth. The countersunk hole measurement method provided by this invention realizes the automatic and rapid measurement of the countersunk hole diameter and depth, promoting the development of cost reduction and efficiency improvement in industrial production.

Description

Technical Field

[0001] This invention relates to, but is not limited to, the fields of measurement technology and data processing technology, and particularly to a method for measuring countersunk holes based on visual point cloud fusion analysis. Background Technology

[0002] Hole making is a critical manufacturing process in the aerospace industry, involving drilling, reaming, and countersinking on structural components for subsequent riveting or bolting connections. The numerous connection holes on the structural components directly affect the assembly accuracy, service life, and safety of aerospace vehicles. Therefore, hole inspection is a crucial step in the hole making process.

[0003] Currently, common hole inspection methods typically employ contact measurement techniques, which use mechanical equipment such as coordinate measuring machines, inside micrometers, and bevel gauges for contact inspection. This method suffers from high cost, low efficiency, and the risk of damaging the workpiece surface. Especially for the millions of connection holes inside aircraft, contact measurement methods are ill-suited to meet the demands of high-efficiency on-site inspection.

[0004] With the development of 3D measurement technology, binocular structured light technology, which has advantages such as wide range, high precision, and non-contact operation, is being used more and more widely. However, in the inspection of countersunk holes in aluminum alloy materials, there is a problem of missing 3D point cloud data caused by highly reflective surfaces, which makes it difficult to measure the countersunk surface of the countersunk hole. Summary of the Invention

[0005] The purpose of this invention is to solve the above-mentioned technical problems. This invention provides a countersunk hole measurement method based on visual point cloud fusion analysis to solve the problem that the lack of high reflective surface data makes it difficult to measure the countersunk surface in the countersunk hole measurement using binocular structured light technology.

[0006] The technical solution of the present invention: The embodiments of the present invention provide a method for measuring countersunk holes based on visual point cloud fusion analysis, the method comprising:

[0007] Step 1: Use a binocular structured light measurement device to photograph the perforated product to obtain two-dimensional visual image data and three-dimensional point cloud data of the perforated product in the camera coordinate system.

[0008] Step 2: For the two-dimensional visual image data of the hole-making product, use manual selection or automatic circle recognition to identify the inner and outer circle contours of the countersunk hole, and record the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data;

[0009] Step 3: For the 3D point cloud data of the countersunk hole, based on the pixel coordinates of the inner and outer circular contours recorded in Step 2, the 3D point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk cone surface area, and the interface area between the countersunk cone surface and the inner diameter surface.

[0010] Step 4: For the point cloud data within the surface area of ​​the hole-making product divided in Step 3, the least squares method is used to fit and calculate the first-order plane function of the hole-making product surface, which is used to characterize the theoretical plane of the hole-making product surface.

[0011] Step 5: For the point cloud data within the countersunk cone region divided in Step 3, the least squares fitting and maximum likelihood fitting algorithms are used to fit the surface function of the countersunk cone to characterize the theoretical surface of the countersunk cone.

[0012] Step 6: Based on the functions of the surface of the drilled product and the countersink cone surface, calculate the intersection line of the two theoretical surfaces, where the diameter of the intersection line is the countersink diameter;

[0013] Step 7: Based on the point cloud data of the junction area between the countersunk cone surface and the inner diameter surface and the function of the surface of the drilled product, after filtering and denoising the point cloud data of the junction area between the countersunk cone surface and the inner diameter surface, calculate the average distance between the junction area of ​​the inner diameter surface and the surface of the drilled product. The average distance is the countersunk depth.

[0014] Optionally, in the method described above, the binocular structured light measurement device in step one includes two cameras and a projection instrument, and the two cameras have the function of simultaneous exposure and shooting, and the projection instrument is used to perform planar light projection or phase-shift structured light projection according to the settings.

[0015] Optionally, in the method described above, step one includes:

[0016] Step 11: Control the projector to project planar light to illuminate the perforated product and control the camera to capture images, thereby obtaining two-dimensional visual image data in the camera coordinate system;

[0017] Step 12: Control the projector to project phase-shift structured light to illuminate the hole-making product and control the camera to take pictures. Obtain the three-dimensional point cloud data in the camera coordinate system according to the binocular measurement algorithm.

[0018] Optionally, in the method described above, before or during step one of taking pictures of the hole-making product using a binocular structured light measurement device, the method further includes:

[0019] By using a handheld or collaborative robot mounted device, the planar light and phase-shifting structured light projected by the projector in the structured light measurement equipment are adjusted to be perpendicular to the surface of the hole-making product.

[0020] Optionally, in the method described above, the two-dimensional visual image data and the three-dimensional point cloud data in step one are both in the same camera coordinate system. The two-dimensional visual image data has pixel coordinates in the x and y directions, and the three-dimensional point cloud has pixel coordinates in the x and y directions and a depth coordinate in the z direction.

[0021] Optionally, in the method described above, during the process of manually selecting the inner and outer circle contours of the countersunk hole in step two, the size of the inner and outer circles of the countersunk hole is manually determined, the inner and outer circle contours are moved and marked to confirm, and the pixel coordinate information of the contours is returned.

[0022] Optionally, in the method described above, during the automatic circle recognition process in step two, the inner and outer circular contours of the countersunk hole can be automatically identified based on the grayscale information of the two-dimensional visual image data according to the preset inner and outer diameter range, and the pixel coordinate information of the contour can be returned.

[0023] Alternatively, in the method described above,

[0024] The method for fitting the first-order plane function of the hole-making product surface using the least squares method in step four is as follows: the fitting of the first-order plane function of the hole-making product surface is implemented using computer languages ​​such as Java, C, Matlab, and Python.

[0025] In step five, the method of fitting the surface function of the countersunk cone surface using least squares fitting and maximum likelihood fitting algorithms is as follows: the surface function of the countersunk cone surface in the hole-making product is fitted using computer languages ​​such as Java, C, Matlab, and Python.

[0026] Optionally, in the method described above, step six includes:

[0027] Step 61: Calculate the intersection line between the theoretical curved surface of the countersink cone and the theoretical plane of the surface of the drilled product;

[0028] Step 62: Use the ellipse fitting algorithm to calculate the ellipse function corresponding to the intersection of the two theoretical surfaces;

[0029] Step 63: Obtain the principal diameter and secondary diameter of the ellipse based on the elliptic function, and calculate the average value of the two diameters. This average value is the diameter of the countersink.

[0030] Optionally, in the method described above, step seven includes:

[0031] Step 71: For all point cloud data in the area where the countersink cone surface and the inner diameter surface intersect, calculate the normal distance value from each point to the theoretical plane of the surface of the drilled product.

[0032] Step 72: For the dataset consisting of all normal distance values, calculate the standard deviation of the dataset, identify data that are more than n times the standard deviation as noisy data, and discard the noisy data to complete the denoising process, wherein the standard deviation is between 1.3 and 1.6.

[0033] Step 73: Perform a mean operation on the denoised dataset, and the mean obtained is the depth of the countersink.

[0034] The beneficial effects of this invention: This invention provides a countersunk hole measurement method based on visual point cloud fusion analysis. A binocular structured light measurement device is used to photograph the countersunk product, obtaining two-dimensional visual image data and three-dimensional point cloud data of the product in the camera coordinate system. On one hand, for the two-dimensional visual image data of the countersunk product, the inner and outer circular contours of the countersunk hole are identified using manual selection or automatic circular recognition, and the pixel coordinates of the inner and outer circular contours in the two-dimensional visual image data are recorded. On the other hand, for the three-dimensional point cloud data of the countersunk hole, based on the pixel coordinates of the inner and outer circular contours recorded in step two, the three-dimensional point cloud data is automatically divided into the surface area of ​​the countersunk product, the countersunk cone surface area, and the boundary area between the countersunk cone surface and the inner diameter surface. Regarding the above division… Point cloud data within the surface area of ​​the drilled product are used to fit a linear plane function of the drilled product surface using the least squares method, which is used to characterize the theoretical plane of the drilled product surface. Furthermore, for the point cloud data within the aforementioned divided countersunk cone surface area, the surface function of the countersunk cone surface is fitted using least squares fitting and maximum likelihood fitting algorithms, which is used to characterize the theoretical surface of the countersunk cone surface. Based on the functions of the drilled product surface and the countersunk cone surface, the intersection line of the two theoretical planes is calculated, and the diameter of this intersection line is the countersunk diameter. Finally, based on the point cloud data of the boundary region between the countersunk cone surface and the inner diameter surface and the function of the drilled product surface, the point cloud data of the boundary region between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the boundary region of the inner diameter surface and the drilled product surface is calculated, which is the countersunk depth. The technical solution provided by this invention addresses the problem of missing point cloud data caused by highly reflective surfaces during structured light 3D measurement. It utilizes 2D visual information to assist in processing 3D point cloud data, enabling automatic measurement of the countersunk hole diameter and depth. This provides an automatic detection method for the extensive countersunk hole inspection work in aerospace assembly, promoting cost reduction and efficiency improvement in industrial production. Attached Figure Description

[0035] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of the present invention and do not constitute a limitation on the technical solutions of the present invention.

[0036] Figure 1 A flowchart of a countersunk hole measurement method based on visual point cloud fusion analysis provided in an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of two-dimensional visual image data and three-dimensional point cloud data obtained by a binocular structured light measurement device in an embodiment of the present invention. Figure 2 Image a in the diagram represents two-dimensional visual image data, while image b represents three-dimensional point cloud data.

[0038] Figure 3 This is a schematic diagram of the inner and outer circular contours of the countersunk hole extracted from two-dimensional visual image data in an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the segmentation result of three-dimensional point cloud data based on the inner and outer circular contour data of the countersunk hole in the two-dimensional visual image data in an embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of the theoretical plane obtained by fitting point cloud data within the surface area of ​​the perforated product in an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the theoretical surface obtained by fitting point cloud data within the countersunk cone region in an embodiment of the present invention;

[0042] Figure 7 This is a schematic diagram showing the fitting results of the theoretical plane of the hole-making product surface and the theoretical curved surface of the countersink cone surface in an embodiment of the present invention;

[0043] Figure 8 This is a schematic diagram illustrating the depth measurement results obtained by using the countersunk hole measurement method based on visual point cloud fusion analysis according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

[0045] As explained in the background section, hole making is a critical manufacturing process in the aerospace field, and hole inspection is a crucial step in the hole making process. Although three-dimensional measurement technology has been applied in hole inspection, the inspection of countersunk holes in aluminum alloy materials suffers from the problem of missing three-dimensional point cloud data caused by highly reflective surfaces, making it difficult to measure the countersunk surface of the countersunk hole.

[0046] To address the aforementioned issues, this invention proposes a countersunk hole measurement method based on visual point cloud fusion analysis. By fusing two-dimensional visual image information and three-dimensional point cloud information (using a visual and point cloud data fusion method, utilizing visual image information to divide the three-dimensional point cloud, and assisting in three-dimensional point cloud data processing), the automatic high-precision measurement of the countersunk hole's countersunk hole diameter and depth is achieved.

[0047] The present invention provides the following specific embodiments, which can be combined with each other. For the same or similar concepts or processes, they may not be described again in some embodiments.

[0048] Figure 1 This is a flowchart illustrating a countersunk hole measurement method based on visual point cloud fusion analysis, provided as an embodiment of the present invention. Figure 1As shown in the embodiment of the present invention, the countersunk hole measurement method based on visual point cloud fusion analysis includes the following steps:

[0049] Step 1: Use a binocular structured light measurement device to photograph the perforated product to obtain two-dimensional visual image data and three-dimensional point cloud data of the perforated product in the camera coordinate system.

[0050] Step 2: For the two-dimensional visual image data of the hole-making product, use manual selection or automatic circle recognition to identify the inner and outer circle contours of the countersunk hole, and record the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data;

[0051] Step 3: For the 3D point cloud data of the countersunk hole, based on the pixel coordinates of the inner and outer circular contours recorded in Step 2, the 3D point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk cone surface area, and the interface area between the countersunk cone surface and the inner diameter surface.

[0052] Step 4: For the point cloud data within the surface area of ​​the hole-making product divided in Step 3, the least squares method is used to fit and calculate the first-order plane function of the hole-making product surface, which is used to characterize the theoretical plane of the hole-making product surface.

[0053] Step 5: For the point cloud data within the countersunk cone region divided in Step 3, the least squares fitting and maximum likelihood fitting algorithms are used to fit the surface function of the countersunk cone to characterize the theoretical surface of the countersunk cone.

[0054] Step 6: Based on the functions of the surface of the drilled product and the countersink cone surface, calculate the intersection line of the two theoretical surfaces, where the diameter of the intersection line is the countersink diameter;

[0055] Step 7: Based on the point cloud data of the junction area between the countersunk cone surface and the inner diameter surface and the function of the surface of the drilled product, after filtering and denoising the point cloud data of the junction area between the countersunk cone surface and the inner diameter surface, calculate the average distance between the junction area of ​​the inner diameter surface and the surface of the drilled product. The average distance is the countersunk depth.

[0056] In one implementation of this invention, the binocular structured light measurement device in step one includes two cameras and a projection instrument, and the two cameras have the function of simultaneous exposure and shooting. The projection instrument is used to perform planar light projection or phase-shift structured light projection according to the settings.

[0057] In this implementation method, the implementation process of step one may include:

[0058] Step 11: Control the projector to project planar light to illuminate the perforated product and control the camera to capture images, thereby obtaining two-dimensional visual image data in the camera coordinate system;

[0059] Step 12: Control the projector to project phase-shift structured light to illuminate the hole-making product and control the camera to take pictures. Obtain the three-dimensional point cloud data in the camera coordinate system according to the binocular measurement algorithm.

[0060] In its specific implementation, before or during the process of using a binocular structured light measurement device to photograph the hole-making product in step one, this method also includes:

[0061] By using a handheld or collaborative robot mounted device, the planar light and phase-shifting structured light projected by the projector in the structured light measurement equipment are adjusted to be perpendicular to the surface of the hole-making product.

[0062] It should be noted that the two-dimensional visual image data and the three-dimensional point cloud data in step one are both in the same camera coordinate system. The two-dimensional visual image data has pixel coordinates in the x and y directions, and the three-dimensional point cloud has pixel coordinates in the x and y directions and z depth direction coordinates.

[0063] In one implementation of this invention, during the process of manually selecting the inner and outer circle contours of the countersunk hole in step two, the size of the inner and outer circles of the countersunk hole is manually determined, the inner and outer circle contours are moved and marked to confirm, and the pixel coordinate information of the contours is returned.

[0064] In one implementation of this invention, during the automatic circular recognition process in step two, the inner and outer contours of the countersunk hole can be automatically identified based on the grayscale information of the two-dimensional visual image data according to the preset inner and outer diameter range, and the pixel coordinate information of the contour is returned.

[0065] In one implementation of this invention, the method of fitting the first-order plane function of the hole-making product surface using the least squares method in step four above is as follows: the fitting of the first-order plane function of the hole-making product surface is implemented using Java, C, Matlab, or Python computer languages.

[0066] Similarly, in step five, the least squares fitting and maximum likelihood fitting algorithms are used to fit the surface function of the countersunk cone surface. The surface function of the countersunk cone surface in the hole-making product is fitted using Java, C, Matlab, and Python computer languages.

[0067] In one implementation of this invention, step six may include:

[0068] Step 61: Calculate the intersection line between the theoretical curved surface of the countersink cone and the theoretical plane of the surface of the drilled product;

[0069] Step 62: Use the ellipse fitting algorithm to calculate the ellipse function corresponding to the intersection of the two theoretical surfaces;

[0070] Step 63: Obtain the principal diameter and secondary diameter of the ellipse based on the elliptic function, and calculate the average value of the two diameters. This average value is the diameter of the countersink.

[0071] In one implementation of this invention, step seven may include:

[0072] Step 71: For all point cloud data in the area where the countersink cone surface and the inner diameter surface intersect, calculate the normal distance value from each point to the theoretical plane of the surface of the drilled product.

[0073] Step 72: For the dataset consisting of all normal distance values, calculate the standard deviation of the dataset, identify data that are more than n times the standard deviation as noisy data, and discard the noisy data to complete the denoising process, wherein the standard deviation is between 1.3 and 1.6.

[0074] Step 73: Perform a mean operation on the denoised dataset, and the mean obtained is the depth of the countersink.

[0075] This invention provides a countersunk hole measurement method based on visual point cloud fusion analysis. The method involves using a binocular structured light measurement device to photograph the countersunk hole product, obtaining two-dimensional visual image data and three-dimensional point cloud data of the product in the camera coordinate system. Firstly, for the two-dimensional visual image data, the inner and outer circular contours of the countersunk hole are identified using manual selection or automatic circular recognition, and the pixel coordinates of the inner and outer circular contours in the two-dimensional visual image data are recorded. Secondly, for the three-dimensional point cloud data of the countersunk hole, based on the pixel coordinates of the inner and outer circular contours recorded in step two, the three-dimensional point cloud data is automatically divided into the surface area of ​​the countersunk product, the countersunk cone surface area, and the boundary area between the countersunk cone surface and the inner diameter surface. The method then proceeds to define the countersunk hole area based on these defined dimensions. Point cloud data within the product surface area are used to fit a first-order plane function of the hole-making product surface using the least squares method, which is used to characterize the theoretical plane of the hole-making product surface. Furthermore, for the point cloud data within the aforementioned divided countersunk cone surface area, the surface function of the countersunk cone surface is fitted using least squares fitting and maximum likelihood fitting algorithms, which is used to characterize the theoretical surface of the countersunk cone surface. Based on the functions of the hole-making product surface and the countersunk cone surface, the intersection line of the two theoretical planes is calculated, and the diameter of this intersection line is the countersunk diameter. Finally, based on the point cloud data of the boundary region between the countersunk cone surface and the inner diameter surface, and the function of the hole-making product surface, the point cloud data of the boundary region between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the boundary region of the inner diameter surface and the hole-making product surface is calculated, which is the countersunk depth. The technical solution provided by this invention addresses the problem of missing point cloud data caused by highly reflective surfaces during structured light 3D measurement. It utilizes 2D visual information to assist in processing 3D point cloud data, enabling automatic measurement of the countersunk hole diameter and depth. This provides an automatic detection method for the extensive countersunk hole inspection work in aerospace assembly, promoting cost reduction and efficiency improvement in industrial production.

[0076] The following example illustrates the implementation of the countersunk hole measurement method based on visual point cloud fusion analysis provided by the present invention.

[0077] Implementation Example

[0078] Reference Figure 1 The diagram shows the flow chart of the countersunk hole measurement method based on visual point cloud fusion analysis provided for this implementation example. This implementation example is carried out through the following steps:

[0079] Step 1: Use a binocular structured light measurement device to photograph the perforated product to obtain two-dimensional visual image data and three-dimensional point cloud data of the perforated product in the camera coordinate system.

[0080] In step one, the binocular structured light measurement device has two cameras and a projection instrument. The two cameras have the function of simultaneous exposure and shooting, and the projection instrument is used to perform planar light projection or phase-shift structured light projection according to the settings. Figure 2 This is a schematic diagram of two-dimensional visual image data and three-dimensional point cloud data obtained by a binocular structured light measurement device in an embodiment of the present invention. Figure 2 Image a in the diagram represents two-dimensional visual image data, and image b represents three-dimensional point cloud data. The shooting method in step one can include:

[0081] On one hand, the projector is controlled to project planar light to illuminate the perforated product, and the camera is controlled to capture the image, thus obtaining... Figure 2 Figure a shows the two-dimensional visual image data in the camera coordinate system; on the other hand, the projector is controlled to project phase-shift structured light to illuminate the perforated product and the camera is controlled to capture the image, and the result is obtained according to the binocular measurement algorithm. Figure 2 The three-dimensional point cloud data in the camera coordinate system shown in Figure b.

[0082] It should be noted that, before or during the photography of the hole-making product using a binocular structured light measurement device, step one may also include: adjusting the plane light and phase-shifting structured light projected by the projector in the binocular structured light measurement device to be perpendicular to the surface of the hole-making product by means of handheld or collaborative robot mounting.

[0083] Additionally, it should be noted that the two-dimensional visual image data and the three-dimensional point cloud data obtained in step one are both within the same camera coordinate system. The two-dimensional visual image data has pixel coordinates in the x and y directions, while the three-dimensional point cloud has pixel coordinates in the x and y directions and a depth coordinate in the z direction.

[0084] Step two: For the two-dimensional visual image data of the perforated product, identify the circular shapes using either manual selection or automatic circular recognition. Figure 3 The inner and outer circular contours of the countersunk hole are shown, and the pixel coordinates of the inner and outer circular contours in the two-dimensional visual image are recorded. Figure 3This is a schematic diagram of the inner and outer circular contours of the countersunk hole extracted from two-dimensional visual image data in an embodiment of the present invention.

[0085] Step 3: For the 3D point cloud data of the countersunk hole, based on the pixel coordinates of the inner and outer circular contours recorded in Step 2, the 3D point cloud data is automatically divided into... Figure 4 The diagram shows the surface area of ​​the drilled product, the countersunk cone area, and the area where the countersunk cone and the inner diameter surface meet. Figure 4 This is a schematic diagram of the segmentation result of three-dimensional point cloud data based on the inner and outer circular contour data of the countersunk hole in the two-dimensional visual image data in an embodiment of the present invention.

[0086] Step four: For the point cloud data within the surface area of ​​the perforated product defined in step three, the least squares method is used for fitting and calculation. Figure 5 The first-order plane function shown is used to characterize the theoretical plane of the hole-making product surface; Figure 5 This is a schematic diagram of the theoretical plane obtained by fitting point cloud data within the surface area of ​​the perforated product in an embodiment of the present invention.

[0087] Step 5: For the point cloud data within the countersunk cone region defined in Step 3, calculate using least squares fitting and maximum likelihood fitting algorithms. Figure 6 The surface function shown is the fitted surface function of the countersunk cone, used to characterize the theoretical surface of the countersunk cone. Figure 6 This is a schematic diagram of the theoretical surface obtained by fitting point cloud data within the countersunk cone region in an embodiment of the present invention.

[0088] Step Six, based on Figure 7 The surface of the hole-making product and the countersunk cone surface function are shown. The intersection of the two theoretical surfaces is calculated. By fitting an ellipse, the theoretical function of the intersection line is obtained, and then the main diameter (20.14 mm) and secondary diameter (19.96 mm) of the ellipse are obtained. The average value of the main diameter and secondary diameter (20.05 mm) is calculated, which is the countersunk diameter. Figure 7 This is a schematic diagram showing the fitting results of the theoretical plane of the hole-making product surface and the theoretical curved surface of the countersink cone surface in an embodiment of the present invention.

[0089] Step 7: Based on the point cloud data of the interface region between the countersunk cone surface and the inner diameter surface, and the plane function of the surface of the drilled product, calculate the normal distance from each point in the interface region to the theoretical surface of the drilled product, and obtain... Figure 8 The dataset shown consists of all distance data. Figure 8 This is a schematic diagram illustrating the depth measurement results obtained by using the countersunk hole measurement method based on visual point cloud fusion analysis according to an embodiment of the present invention.

[0090] against Figure 8The dataset shown, after outlier detection and discarding, retains the average value of the remaining data (3.066 mm), which is the depth of the countersink. Outlier detection can be performed using common functions in computer languages ​​such as MATLAB and Python.

[0091] While the embodiments disclosed in this invention are as described above, they are merely illustrative of the embodiments to facilitate understanding of the invention and are not intended to limit the invention. Any person skilled in the art to which this invention pertains may make any modifications and variations in the form and details of the implementation without departing from the spirit and scope disclosed herein; however, the scope of patent protection for this invention shall still be determined by the scope defined in the appended claims.

Claims

1. A method for measuring countersunk holes based on visual point cloud fusion analysis, characterized in that, The method includes: Step 1: Use a binocular structured light measurement device to photograph the perforated product to obtain two-dimensional visual image data and three-dimensional point cloud data of the perforated product in the camera coordinate system. Step 2: For the two-dimensional visual image data of the hole-making product, use manual selection or automatic circle recognition to identify the inner and outer circle contours of the countersunk hole, and record the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data; Step 3: For the 3D point cloud data of the countersunk hole, based on the pixel coordinates of the inner and outer circular contours recorded in Step 2, the 3D point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk cone surface area, and the interface area between the countersunk cone surface and the inner diameter surface. Step 4: For the point cloud data within the surface area of ​​the hole-making product divided in Step 3, the least squares method is used to fit and calculate the first-order plane function of the hole-making product surface, which is used to characterize the theoretical plane of the hole-making product surface. Step 5: For the point cloud data within the countersunk cone region divided in Step 3, the least squares fitting and maximum likelihood fitting algorithms are used to fit the surface function of the countersunk cone to characterize the theoretical surface of the countersunk cone. Step 6: Based on the functions of the surface of the drilled product and the countersink cone surface, calculate the intersection line of the two theoretical surfaces, where the diameter of the intersection line is the countersink diameter; Step 7: Based on the point cloud data of the junction area between the countersunk cone surface and the inner diameter surface and the function of the surface of the drilled product, after filtering and denoising the point cloud data of the junction area between the countersunk cone surface and the inner diameter surface, calculate the average distance between the junction area of ​​the inner diameter surface and the surface of the drilled product. The average distance is the countersunk depth.

2. The method according to claim 1, characterized in that, The binocular structured light measurement device in step one has two cameras and a projection instrument. The two cameras have the function of simultaneous exposure and shooting, and the projection instrument is used to perform planar light projection or phase-shift structured light projection according to the settings.

3. The method according to claim 2, characterized in that, Step one includes: Step 11: Control the projector to project planar light to illuminate the perforated product and control the camera to capture images, thereby obtaining two-dimensional visual image data in the camera coordinate system; Step 12: Control the projector to project phase-shift structured light to illuminate the hole-making product and control the camera to take pictures. Obtain the three-dimensional point cloud data in the camera coordinate system according to the binocular measurement algorithm.

4. The method according to claim 3, characterized in that, Before or during the process of using a binocular structured light measurement device to photograph the perforated product in step one, the following is also included: By using a handheld or collaborative robot mounted device, the planar light and phase-shifting structured light projected by the projector in the structured light measurement equipment are adjusted to be perpendicular to the surface of the hole-making product.

5. The method according to claim 1, characterized in that, In step one, both the two-dimensional visual image data and the three-dimensional point cloud data are in the same camera coordinate system. The two-dimensional visual image data has pixel coordinates in the x and y directions, and the three-dimensional point cloud has pixel coordinates in the x and y directions and a depth coordinate in the z direction.

6. The method according to claim 1, characterized in that, In step two, during the process of manually selecting the inner and outer circle contours of the countersunk hole, the size of the inner and outer circles of the countersunk hole is manually determined, the inner and outer circle contours are moved and marked to confirm, and the pixel coordinate information of the contours is returned.

7. The method according to claim 1, characterized in that, In the automatic circular recognition process in step two, the inner and outer contours of the countersunk hole can be automatically identified based on the grayscale information of the two-dimensional visual image data according to the preset inner and outer diameter range, and the pixel coordinate information of the contour is returned.

8. The method according to any one of claims 1 to 7, characterized in that, The method for fitting the first-order plane function of the hole-making product surface using the least squares method in step four is as follows: the fitting of the first-order plane function of the hole-making product surface is implemented using computer languages ​​such as Java, C, Matlab, and Python. In step five, the method of fitting the surface function of the countersunk cone surface using least squares fitting and maximum likelihood fitting algorithms is as follows: the surface function of the countersunk cone surface in the hole-making product is fitted using computer languages ​​such as Java, C, Matlab, and Python.

9. The method according to any one of claims 1 to 7, characterized in that, Step six includes: Step 61: Calculate the intersection line between the theoretical curved surface of the countersink cone and the theoretical plane of the surface of the drilled product; Step 62: Use the ellipse fitting algorithm to calculate the ellipse function corresponding to the intersection of the two theoretical surfaces; Step 63: Obtain the principal diameter and secondary diameter of the ellipse based on the elliptic function, and calculate the average value of the two diameters. This average value is the diameter of the countersink.

10. The method according to any one of claims 1 to 7, characterized in that, Step seven includes: Step 71: For all point cloud data in the area where the countersink cone surface and the inner diameter surface intersect, calculate the normal distance value from each point to the theoretical plane of the surface of the drilled product. Step 72: For the dataset consisting of all normal distance values, calculate the standard deviation of the dataset, identify data that are more than n times the standard deviation as noisy data, and discard the noisy data to complete the denoising process, where n is between 1.3 and 1.

6. Step 73: Perform a mean operation on the denoised dataset, and the mean obtained is the depth of the countersink.

Images