Counterbore measurement method based on visual point cloud fusion analysis

Abstract

The invention discloses a counterbore measurement method based on visual point cloud fusion analysis, and the method comprises the steps: 1, employing a binocular structured light measurement device to carry out the shooting of a hole-making product, and obtaining the two-dimensional visual image data and three-dimensional point cloud data of the hole-making product under a camera coordinate system; step 2, identifying the inner and outer circle contours of the counterbore according to the two-dimensional visual image data of the drilling product, and recording the pixel coordinates of the contours; 3, automatically dividing the three-dimensional point cloud data into a hole-making product surface area, a dimple conical surface area and a dimple conical surface and inner diameter surface boundary area based on the pixel coordinates of the inner and outer circle outlines; 4, according to the point cloud data in the surface area of the perforated product, a linear plane function of the surface of the perforated product is calculated and used for representing a theoretical plane of the surface of the perforated product; step 5, fitting a curved surface function of the dimple conical surface for the point cloud data in the dimple conical surface area to represent a theoretical curved surface of the dimple conical surface; step 6, calculating an intersecting line of the two theoretical surfaces, wherein the diameter of the intersecting line is the diameter of the dimpling pit; and step 7, based on the point cloud data of the interface area of the dimple conical surface and the inner diameter surface and the theoretical plane of the surface of the drilling product, filtering and denoising the point cloud data of the interface area of the dimple conical surface and the inner diameter surface, and then calculating the dimple depth. According to the counterbore measuring method provided by the invention, automatic and rapid measurement of the pit diameter and the pit depth of the counterbore is realized, and industrial cost-reducing and benefit-increasing production development is promoted.

Description

Technical Field

[0001] The present invention relates to the field of measurement technology data processing technology, but is not limited to it, and in particular to a countersunk hole measurement method based on visual point cloud fusion analysis. Background Art

[0002] Hole making is a key manufacturing process in the aerospace field, involving drilling, reaming, countersinking and other processes on the fuselage structure for subsequent riveting or bolting. There are a large number of connection holes on the fuselage structure, and the quality of hole making is directly related to the assembly accuracy, service life and safety of aerospace. Therefore, hole making detection is a crucial link in the hole making process.

[0003] At present, the commonly used hole detection usually adopts the contact measurement method, which uses mechanical equipment such as three-dimensional coordinates, internal diameter micrometers, and groove gauges for contact detection. This detection method has the risk of high cost, low efficiency, and possible damage to the surface of the workpiece. Especially for the millions of connection holes in aircraft, the contact measurement method is difficult to meet the high-efficiency detection needs on site.

[0004] With the development of 3D measurement technology, binocular structured light technology with advantages such as large range, high precision and non-contact has been widely used. However, in the detection of countersunk holes in aluminum alloy materials, there is a problem of missing 3D point cloud data caused by highly reflective surfaces, which makes it difficult to measure the countersunk surface of the countersunk hole. Summary of the invention

[0005] Purpose of the present invention: In order to solve the above technical problems, an embodiment of the present invention provides a countersunk hole measurement method based on visual point cloud fusion analysis to solve the problem that in the countersunk hole measurement using binocular structured light technology, due to the lack of high-reflective surface data, it is difficult to achieve countersunk surface measurement.

[0006] Technical solution of the present invention: An embodiment of the present invention provides a countersunk hole measurement method based on visual point cloud fusion analysis, the method comprising:

[0007] Step 1: Use binocular structured light measurement equipment to shoot the hole-making product to obtain two-dimensional visual image data and three-dimensional point cloud data of the hole-making product in the camera coordinate system;

[0008] Step 2: for the two-dimensional visual image data of the hole-making product, the inner and outer circle contours of the countersunk hole are identified by manual selection or automatic circle recognition, and the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data are recorded;

[0009] Step 3: For the 3D point cloud data of the countersunk hole, according to the pixel coordinates of the inner and outer circle contours recorded in step 2, the 3D point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk conical surface area, and the boundary area between the countersunk conical surface and the inner diameter surface;

[0010] Step 4: for the point cloud data in the surface area of ​​the hole-making product divided in step 3, a least squares method is used to fit and calculate a linear plane function of the surface of the hole-making product, so as to characterize a theoretical plane of the surface of the hole-making product;

[0011] Step 5: for the point cloud data in the countersunk cone area divided in step 3, the surface function of the countersunk cone surface is fitted by using the least square fitting and maximum likelihood fitting algorithms to characterize the theoretical surface of the countersunk cone surface;

[0012] Step 6: Based on the function of the surface of the hole-making product and the countersink cone surface, the intersection line of the two theoretical surfaces is calculated, and the diameter of the intersection line is the countersink diameter;

[0013] Step seven, based on the point cloud data of the interface area between the countersunk cone surface and the inner diameter surface and the function of the surface of the hole-making product, the point cloud data of the interface area between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the interface area of ​​the inner diameter surface and the surface of the hole-making product is calculated. The average distance is the countersunk depth.

[0014] Optionally, in the method described above, the binocular structured light measurement device in step one has two cameras and a projector component, and the two cameras have a simultaneous exposure and shooting function, and the projector component is used to perform planar light projection or phase-shifted structured light projection according to the settings.

[0015] Optionally, in the method as described above, step one comprises:

[0016] Step 11, controlling the projector to project plane light to illuminate the hole-making product and controlling the camera to shoot, to obtain two-dimensional visual image data in the camera coordinate system;

[0017] Step 12, controlling the projector to project phase-shifted structured light to illuminate the hole-making product and controlling the camera to shoot, and obtaining three-dimensional point cloud data in the camera coordinate system according to the binocular measurement algorithm.

[0018] Optionally, in the method described above, before or during the process of photographing the hole-making product using a binocular structured light measurement device in step 1, the method further includes:

[0019] By hand-held or collaborative robot mounting, adjust the plane light and phase-shifted structured light projected by the projection instrument in the target structured light measurement equipment to be perpendicular to the surface of the hole-making product.

[0020] Optionally, in the method described above, the two-dimensional visual image data and the three-dimensional point cloud data in step one are in the same camera coordinate system, the two-dimensional visual image data has x, y direction pixel coordinates, and the three-dimensional point cloud has x, y direction pixel coordinates and z depth direction coordinates.

[0021] Optionally, in the method described above, in the process of manually selecting the inner and outer circle contours of the countersunk hole in step 2, the sizes of the inner and outer circles of the countersunk hole are manually determined, the inner and outer circle contours are moved and marked to confirm, and the pixel coordinate information of the contours is returned.

[0022] Optionally, in the method described above, during the automatic circle recognition process in step 2, the inner and outer circle contours of the countersunk hole can be automatically identified according to the preset inner and outer diameter ranges and the grayscale information of the two-dimensional visual image data, and the pixel coordinate information of the contour can be returned.

[0023] Optionally, in the method described above,

[0024] The method of using the least square method to fit and calculate the linear plane function of the surface of the hole-making product in step 4 is: fitting the linear plane function of the surface of the hole-making product by using Java, C, Matlab, Python computer language;

[0025] The method of fitting the surface function of the countersink cone surface by using the least square fitting and maximum likelihood fitting algorithms in step 5 is: fitting the surface function of the countersink cone surface in the hole-making product is realized by using Java, C, Matlab, and Python computer languages.

[0026] Optionally, in the method described above, step six comprises:

[0027] Step 61, calculating the intersection line between the theoretical curved surface of the countersink cone surface and the theoretical plane of the hole-making product surface;

[0028] Step 62, using an ellipse fitting algorithm to calculate the ellipse function corresponding to the intersection of the two theoretical surfaces;

[0029] Step 63, obtaining the major diameter and minor diameter of the ellipse based on the ellipse function, and calculating the average value of the two diameters, which is the countersink diameter.

[0030] Optionally, in the method described above, step seven comprises:

[0031] Step 71, for all point cloud data of the interface area between the countersunk conical surface and the inner diameter surface, calculate the normal distance value from each point to the theoretical plane of the hole-making product surface;

[0032] Step 72, for the data set composed of all normal distance values, calculate the standard deviation of the data set, determine the data beyond n times the standard deviation as noise data, discard the noise data, and complete the denoising process, wherein the standard deviation is between 1.3 and 1.6;

[0033] Step 73, performing a mean operation on the denoised data set, and the obtained mean is the countersink depth.

[0034] Beneficial effects of the present invention: An embodiment of the present invention provides a countersunk hole measurement method based on visual point cloud fusion analysis, in which a binocular structured light measurement device is used to photograph a hole-making product to obtain two-dimensional visual image data and three-dimensional point cloud data of the hole-making product in a camera coordinate system; on the one hand, for the two-dimensional visual image data of the hole-making product, manual selection or automatic circle recognition is used to identify the inner and outer circle contours of the countersunk hole, and the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data are recorded; on the other hand, for the three-dimensional point cloud data of the countersunk hole, according to the pixel coordinates of the inner and outer circle contours recorded in step two, the three-dimensional point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk cone area, and the boundary area between the countersunk cone and the inner diameter surface; for the above division The point cloud data in the surface area of ​​the hole-making product obtained is used to fit the linear plane function of the hole-making product surface by the least squares method, which is used to characterize the theoretical plane of the hole-making product surface; and, for the point cloud data in the countersunk cone area divided as above, the surface function of the countersunk cone surface is fitted by the least squares fitting and maximum likelihood fitting algorithms, which is used to characterize the theoretical surface of the countersunk cone surface; based on the function of the hole-making product surface and the countersunk cone surface, the intersection line of the two theoretical surfaces is calculated, and the diameter of the intersection line is the countersunk diameter; finally, based on the point cloud data of the boundary area between the countersunk cone surface and the inner diameter surface and the function of the hole-making product surface, the point cloud data of the boundary area between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the boundary area of ​​the inner diameter surface and the hole-making product surface is calculated, which is the countersunk depth. The technical solution provided by the present invention addresses the problem of missing point cloud data caused by highly reflective surfaces during structured light three-dimensional measurement. It utilizes two-dimensional visual information to assist in processing three-dimensional point cloud data, thereby realizing automatic measurement of the diameter and depth of countersunk holes. It provides an automatic detection method for a large number of countersunk hole detection work in aviation assembly, and promotes the development of industrialized production with cost reduction and efficiency improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0036] Figure 1 A flow chart of a countersunk hole measurement method based on visual point cloud fusion analysis provided by an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of two-dimensional visual image data and three-dimensional point cloud data captured by a binocular structured light measurement device in an embodiment of the present invention; Figure 2 Figure a in the figure is two-dimensional visual image data, and figure b is three-dimensional point cloud data;

[0038] Figure 3 It is a schematic diagram of the inner and outer circle contours of the countersunk hole extracted by two-dimensional visual image data in an embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of the segmentation result of the three-dimensional point cloud data according to the inner and outer circle contour data of the countersunk hole in the two-dimensional visual image data in an embodiment of the present invention;

[0040] Figure 5 It is a schematic diagram of a theoretical plane obtained by fitting point cloud data in the surface area of ​​a hole-making product according to an embodiment of the present invention;

[0041] Figure 6 Schematic diagram of a theoretical surface obtained by fitting point cloud data in a countersunk conical surface area according to an embodiment of the present invention;

[0042] Figure 7 It is a schematic diagram of the fitting results of the theoretical plane of the surface of the hole-making product and the theoretical curved surface of the countersinking cone surface in the embodiment of the present invention;

[0043] Figure 8 A schematic diagram of a dimple depth measurement result obtained by using a countersunk hole measurement method based on visual point cloud fusion analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present invention more clear, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other arbitrarily without conflict.

[0045] As explained in the above background technology, hole making is a key manufacturing process in the field of aerospace, and hole making detection is an important part of the hole making process. Although 3D measurement technology has been used in hole making detection, there is a problem of missing 3D point cloud data caused by highly reflective surfaces in the detection of countersunk holes of aluminum alloy materials, which makes it difficult to measure the countersunk surface of the countersunk hole.

[0046] In response to the above problems, an embodiment of the present invention proposes a countersunk hole measurement method based on visual point cloud fusion analysis. Through the fusion analysis of two-dimensional visual image information and three-dimensional point cloud information (adopting a visual and point cloud data fusion method, using visual image information to divide the three-dimensional point cloud, and assisting in three-dimensional point cloud data processing), automatic and high-precision measurement of the countersunk hole diameter and depth is achieved.

[0047] The present invention provides the following specific embodiments which can be combined with each other. The same or similar concepts or processes may not be described in detail in some embodiments.

[0048] Figure 1 The present invention provides a flowchart of a method for measuring a countersunk hole based on visual point cloud fusion analysis. Figure 1As shown, the countersunk hole measurement method based on visual point cloud fusion analysis provided by the embodiment of the present invention includes the following steps:

[0049] Step 1: Use binocular structured light measurement equipment to shoot the hole-making product to obtain two-dimensional visual image data and three-dimensional point cloud data of the hole-making product in the camera coordinate system;

[0050] Step 2: for the two-dimensional visual image data of the hole-making product, the inner and outer circle contours of the countersunk hole are identified by manual selection or automatic circle recognition, and the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data are recorded;

[0051] Step 3: For the 3D point cloud data of the countersunk hole, according to the pixel coordinates of the inner and outer circle contours recorded in step 2, the 3D point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk conical surface area, and the boundary area between the countersunk conical surface and the inner diameter surface;

[0052] Step 4: for the point cloud data in the surface area of ​​the hole-making product divided in step 3, a least squares method is used to fit and calculate a linear plane function of the surface of the hole-making product, so as to characterize a theoretical plane of the surface of the hole-making product;

[0053] Step 5: for the point cloud data in the countersunk cone area divided in step 3, the surface function of the countersunk cone surface is fitted by using the least square fitting and maximum likelihood fitting algorithms to characterize the theoretical surface of the countersunk cone surface;

[0054] Step 6: Based on the function of the surface of the hole-making product and the countersink cone surface, the intersection line of the two theoretical surfaces is calculated, and the diameter of the intersection line is the countersink diameter;

[0055] Step seven, based on the point cloud data of the interface area between the countersunk cone surface and the inner diameter surface and the function of the surface of the hole-making product, the point cloud data of the interface area between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the interface area of ​​the inner diameter surface and the surface of the hole-making product is calculated. The average distance is the countersunk depth.

[0056] In one implementation of the embodiment of the present invention, the binocular structured light measurement device in step one has two cameras and a projector device, and the two cameras have a simultaneous exposure and shooting function, and the projector device is used to perform planar light projection or phase-shifted structured light projection according to the settings.

[0057] In this implementation, the implementation process of step 1 may include:

[0058] Step 11, controlling the projector to project plane light to illuminate the hole-making product and controlling the camera to shoot, to obtain two-dimensional visual image data in the camera coordinate system;

[0059] Step 12, controlling the projector to project phase-shifted structured light to illuminate the hole-making product and controlling the camera to shoot, and obtaining three-dimensional point cloud data in the camera coordinate system according to the binocular measurement algorithm.

[0060] In a specific implementation, the implementation method further includes:

[0061] By hand-held or collaborative robot mounting, adjust the plane light and phase-shifted structured light projected by the projection instrument in the target structured light measurement equipment to be perpendicular to the surface of the hole-making product.

[0062] It should be noted that the two-dimensional visual image data and the three-dimensional point cloud data in step one are in the same camera coordinate system. The two-dimensional visual image data has x, y direction pixel coordinates, and the three-dimensional point cloud has x, y direction pixel coordinates and z depth direction coordinates.

[0063] In one implementation of the embodiment of the present invention, in the process of manually selecting the inner and outer circle contours of the countersunk hole in step 2, the sizes of the inner and outer circles of the countersunk hole are manually determined, the inner and outer circle contours are moved and marked to confirm, and the pixel coordinate information of the contours is returned.

[0064] In one implementation of an embodiment of the present invention, during the automatic circle recognition process in step two, the inner and outer circle contours of the countersunk hole can be automatically identified based on the preset inner and outer diameter ranges and the grayscale information of the two-dimensional visual image data, and the pixel coordinate information of the contour can be returned.

[0065] In one implementation of an embodiment of the present invention, the method of using the least squares method to fit and calculate the linear plane function of the surface of the hole-making product in the above step four is: fitting the linear plane function of the surface of the hole-making product is implemented by Java, C, Matlab, and Python computer languages.

[0066] Similarly, in step five, the method of fitting the surface function of the countersink cone surface using the least squares fitting and maximum likelihood fitting algorithms is as follows: the surface function of the countersink cone surface in the hole-making product is fitted using Java, C, Matlab, and Python computer languages.

[0067] In an implementation of the embodiment of the present invention, the implementation process of the above step 6 may include:

[0068] Step 61, calculating the intersection line between the theoretical curved surface of the countersink cone surface and the theoretical plane of the hole-making product surface;

[0069] Step 62, using an ellipse fitting algorithm to calculate the ellipse function corresponding to the intersection of the two theoretical surfaces;

[0070] Step 63, obtaining the major diameter and minor diameter of the ellipse based on the ellipse function, and calculating the average value of the two diameters, which is the countersink diameter.

[0071] In an implementation of the embodiment of the present invention, the implementation process of the above step 7 may include:

[0072] Step 71, for all point cloud data of the interface area between the countersunk conical surface and the inner diameter surface, calculate the normal distance value from each point to the theoretical plane of the hole-making product surface;

[0073] Step 72, for the data set composed of all normal distance values, calculate the standard deviation of the data set, determine the data beyond n times the standard deviation as noise data, discard the noise data, and complete the denoising process, wherein the standard deviation is between 1.3 and 1.6;

[0074] Step 73, performing a mean operation on the denoised data set, and the obtained mean is the countersink depth.

[0075] The embodiment of the present invention provides a countersunk hole measurement method based on visual point cloud fusion analysis, which uses a binocular structured light measurement device to shoot a hole-making product to obtain two-dimensional visual image data and three-dimensional point cloud data of the hole-making product in the camera coordinate system; on the one hand, for the two-dimensional visual image data of the hole-making product, manual selection or automatic circle recognition is used to identify the inner and outer circle contours of the countersunk hole, and record the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data; on the other hand, for the three-dimensional point cloud data of the countersunk hole, according to the pixel coordinates of the inner and outer circle contours recorded in step two, the three-dimensional point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk cone surface area, and the boundary area between the countersunk cone surface and the inner diameter surface; for the hole-making product divided as above The point cloud data in the product surface area is fitted with the least squares method to calculate the linear plane function of the hole-making product surface, which is used to characterize the theoretical plane of the hole-making product surface; and, for the point cloud data in the countersunk cone area divided as above, the surface function of the countersunk cone surface is fitted by the least squares fitting and maximum likelihood fitting algorithms, which is used to characterize the theoretical surface of the countersunk cone surface; based on the function of the hole-making product surface and the countersunk cone surface, the intersection line of the two theoretical surfaces is calculated, and the diameter of the intersection line is the countersunk diameter; finally, based on the point cloud data of the boundary area between the countersunk cone surface and the inner diameter surface and the function of the hole-making product surface, the point cloud data of the boundary area between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the boundary area of ​​the inner diameter surface and the hole-making product surface is calculated, which is the countersunk depth. The technical solution provided by the present invention addresses the problem of missing point cloud data caused by highly reflective surfaces during structured light three-dimensional measurement. It utilizes two-dimensional visual information to assist in processing three-dimensional point cloud data, thereby realizing automatic measurement of the diameter and depth of countersunk holes. It provides an automatic detection method for a large number of countersunk hole detection work in aviation assembly, and promotes the development of industrialized production with cost reduction and efficiency improvement.

[0076] The following is a schematic illustration of an implementation example of the countersunk hole measurement method based on visual point cloud fusion analysis provided by an embodiment of the present invention.

[0077] Implementation Example

[0078] Reference Figure 1 As shown in FIG. 1 , the countersunk hole measurement method process based on visual point cloud fusion analysis provided by this implementation example is implemented through the following steps:

[0079] Step 1: Use binocular structured light measurement equipment to shoot the hole-making product to obtain two-dimensional visual image data and three-dimensional point cloud data of the hole-making product in the camera coordinate system.

[0080] In step 1, the binocular structured light measurement device used has two cameras and a projector, and the two cameras have a simultaneous exposure and shooting function, and the projector is used to perform planar light projection or phase-shifted structured light projection according to the settings; Figure 2 It is a schematic diagram of two-dimensional visual image data and three-dimensional point cloud data captured by a binocular structured light measurement device in an embodiment of the present invention; Figure 2 The image a in the figure is two-dimensional visual image data, and the image b is three-dimensional point cloud data. The shooting method of step 1 may include:

[0081] On the one hand, the projector is controlled to project plane light to illuminate the hole-making product and the camera is controlled to shoot, and the Figure 2 The two-dimensional visual image data in the camera coordinate system shown in Figure a; on the other hand, the projector is controlled to project phase-shifted structured light to illuminate the hole-making product and the camera is controlled to shoot, and the binocular measurement algorithm is used to obtain Figure 2 The three-dimensional point cloud data in the camera coordinate system shown in Figure b.

[0082] It should be noted that, before or during the photographing of the hole-making product using the binocular structured light measurement device, step one may also include: adjusting the plane light and phase-shifted structured light projected by the projection instrument in the binocular structured light measurement device to be perpendicular to the surface of the hole-making product by hand-held or collaborative robot-mounted mode.

[0083] In addition, it should be noted that the two-dimensional visual image data and the three-dimensional point cloud data obtained in step one are in the same camera coordinate system. The two-dimensional visual image data has x, y direction pixel coordinates, and the three-dimensional point cloud has x, y direction pixel coordinates and z depth direction coordinates.

[0084] Step 2: For the two-dimensional visual image data of the hole-making product, manually select or automatically identify the circle. Figure 3 The inner and outer circular contours of the countersunk hole are shown, and the pixel coordinates of the inner and outer circular contours in the two-dimensional visual image are recorded. Figure 3It is a schematic diagram of the inner and outer circle contours of the countersunk hole extracted by two-dimensional visual image data in an embodiment of the present invention.

[0085] Step 3: For the 3D point cloud data of the countersunk hole, the 3D point cloud data is automatically divided into Figure 4 The surface area of ​​the hole-making product, the countersunk conical surface area, and the interface area between the countersunk conical surface and the inner diameter surface are shown; Figure 4 It is a schematic diagram of the segmentation result of three-dimensional point cloud data based on the inner and outer circle contour data of the countersunk hole in the two-dimensional visual image data in an embodiment of the present invention.

[0086] Step 4: For the point cloud data in the surface area of ​​the hole-making product divided in step 3, the least squares method is used for fitting calculation Figure 5 The first-order plane function of the surface of the hole-making product shown is used to characterize the theoretical plane of the surface of the hole-making product; Figure 5 It is a schematic diagram of a theoretical plane obtained by fitting point cloud data in the surface area of ​​a hole-making product in an embodiment of the present invention.

[0087] Step 5: For the point cloud data in the countersunk cone area divided in step 3, the least squares fitting and maximum likelihood fitting algorithms are used to calculate Figure 6 The surface function of the fitting countersink cone surface shown is used to characterize the theoretical surface of the countersink cone surface; Figure 6 Schematic diagram of a theoretical surface obtained by fitting point cloud data in the countersunk conical surface area according to an embodiment of the present invention.

[0088] Step 6: Based on Figure 7 The surface of the hole-making product and the countersink cone function shown are used to calculate the intersection of the two theoretical surfaces. The theoretical function of the intersection is obtained by ellipse fitting, and then the major diameter (20.14mm) and minor diameter (19.96mm) of the ellipse are obtained. The average value of the major diameter and minor diameter (20.05mm) is calculated, and this value is the countersink diameter. Figure 7 It is a schematic diagram of the fitting results of the theoretical plane of the hole-making product surface and the theoretical curved surface of the countersunk conical surface in the embodiment of the present invention.

[0089] Step 7: Based on the point cloud data of the interface area between the countersunk conical surface and the inner diameter surface and the plane function of the hole-making product surface, the normal distance from each point in the interface area to the theoretical surface of the hole-making product surface is calculated to obtain Figure 8 The dataset consists of all the distance data shown. Figure 8 A schematic diagram of a dimple depth measurement result obtained by using a countersunk hole measurement method based on visual point cloud fusion analysis provided by an embodiment of the present invention.

[0090] against Figure 8After outlier detection and discarding of the data set shown, the average value of the retained data (3.066 mm) is the countersink depth. Among them, outlier detection can call common functions in computer languages ​​such as MATLAB and Python to complete the calculation.

[0091] Although the embodiments disclosed in the present invention are as above, the contents are only embodiments adopted to facilitate understanding of the present invention and are not intended to limit the present invention. Any technician in the field to which the present invention belongs can make any modifications and changes in the form and details of implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A countersunk hole measurement method based on visual point cloud fusion analysis, characterized in that: The method comprises: Step 1: Use binocular structured light measurement equipment to shoot the hole-making product to obtain two-dimensional visual image data and three-dimensional point cloud data of the hole-making product in the camera coordinate system; Step 2: for the two-dimensional visual image data of the hole-making product, the inner and outer circle contours of the countersunk hole are identified by manual selection or automatic circle recognition, and the pixel coordinates of the inner and outer circle contours in the two-dimensional visual image data are recorded; Step 3: For the 3D point cloud data of the countersunk hole, according to the pixel coordinates of the inner and outer circle contours recorded in step 2, the 3D point cloud data is automatically divided into the surface area of ​​the hole-making product, the countersunk conical surface area, and the boundary area between the countersunk conical surface and the inner diameter surface; Step 4: for the point cloud data in the surface area of ​​the hole-making product divided in step 3, a least squares method is used to fit and calculate a linear plane function of the surface of the hole-making product, so as to characterize a theoretical plane of the surface of the hole-making product; Step 5: for the point cloud data in the countersunk cone area divided in step 3, the surface function of the countersunk cone surface is fitted by using the least square fitting and maximum likelihood fitting algorithms to characterize the theoretical surface of the countersunk cone surface; Step 6: Based on the function of the surface of the hole-making product and the countersink cone surface, the intersection line of the two theoretical surfaces is calculated, and the diameter of the intersection line is the countersink diameter; Step seven, based on the point cloud data of the interface area between the countersunk cone surface and the inner diameter surface and the function of the surface of the hole-making product, the point cloud data of the interface area between the countersunk cone surface and the inner diameter surface is filtered and denoised, and the average distance between the interface area of ​​the inner diameter surface and the surface of the hole-making product is calculated. The average distance is the countersunk depth.

2. The method according to claim 1, characterized in that: The binocular structured light measurement device in step 1 has two cameras and a projector, and the two cameras have a simultaneous exposure and shooting function. The projector is used to perform planar light projection or phase-shifted structured light projection according to the settings.

3. The method according to claim 2, characterized in that The step one comprises: Step 11, controlling the projector to project plane light to illuminate the hole-making product and controlling the camera to shoot, to obtain two-dimensional visual image data in the camera coordinate system; Step 12, controlling the projector to project phase-shifted structured light to illuminate the hole-making product and controlling the camera to shoot, and obtaining three-dimensional point cloud data in the camera coordinate system according to the binocular measurement algorithm.

4. The method according to claim 3, characterized in that: Before or during the process of photographing the hole-making product using a binocular structured light measurement device in step 1, the method further includes: By hand-held or collaborative robot mounting, adjust the plane light and phase-shifted structured light projected by the projection instrument in the target structured light measurement equipment to be perpendicular to the surface of the hole-making product.

5. The method according to claim 1, characterized in that The two-dimensional visual image data and the three-dimensional point cloud data in step one are both in the same camera coordinate system. The two-dimensional visual image data has pixel coordinates in the x and y directions, and the three-dimensional point cloud has pixel coordinates in the x and y directions and coordinates in the z depth direction.

6. The method according to claim 1, characterized in that In the process of manually selecting the inner and outer circle contours of the countersunk hole in step 2, the sizes of the inner and outer circles of the countersunk hole are manually determined, the inner and outer circle contours are moved and marked to confirm, and the pixel coordinate information of the contours is returned.

7. The method according to claim 1, characterized in that During the automatic circle recognition process in step 2, the inner and outer circle contours of the countersunk hole can be automatically recognized according to the preset inner and outer diameter ranges and the grayscale information of the two-dimensional visual image data, and the pixel coordinate information of the contour can be returned.

8. The method according to any one of claims 1 to 7, characterized in that: The method of using the least square method to fit and calculate the linear plane function of the surface of the hole-making product in step 4 is: fitting the linear plane function of the surface of the hole-making product by using Java, C, Matlab, Python computer language; The method of fitting the surface function of the countersink cone surface by using the least square fitting and maximum likelihood fitting algorithms in step 5 is: fitting the surface function of the countersink cone surface in the hole-making product is realized by using Java, C, Matlab, and Python computer languages.

9. The method according to any one of claims 1 to 7, characterized in that: The step six comprises: Step 61, calculating the intersection line between the theoretical curved surface of the countersink cone surface and the theoretical plane of the hole-making product surface; Step 62, using an ellipse fitting algorithm to calculate the ellipse function corresponding to the intersection of the two theoretical surfaces; Step 63, obtaining the major diameter and minor diameter of the ellipse based on the ellipse function, and calculating the average value of the two diameters, which is the countersink diameter.

10. The method according to any one of claims 1 to 7, characterized in that: The step seven comprises: Step 71, for all point cloud data of the interface area between the countersunk conical surface and the inner diameter surface, calculate the normal distance value from each point to the theoretical plane of the hole-making product surface; Step 72, for the data set composed of all normal distance values, calculate the standard deviation of the data set, determine the data beyond n times the standard deviation as noise data, discard the noise data, and complete the denoising process, wherein the standard deviation is between 1.3 and 1.6; Step 73, performing a mean operation on the denoised data set, and the obtained mean is the countersink depth.

Images