Hole site measurement method and device, computer equipment and storage medium

Abstract

The invention relates to a hole site measurement method and device, computer equipment and a storage medium. The method comprises the steps that multi-frame hole site images, synchronously shot by at least two cameras, of a target hole of a to-be-measured workpiece are obtained; correcting the prior information of the target hole based on the hole position image to obtain target prior information; edge detection is carried out based on target prior information, and after three-dimensional reconstruction is carried out on edge point coordinates extracted through edge detection, three-dimensional data of multiple frames of hole position images are fitted to obtain hole position information of a target hole. According to the method and the device, the hole site image can be shot during scanning, the prior information of the hole is corrected by using the hole site image, and then three-dimensional reconstruction is performed based on the corrected target prior information to obtain the hole site information of the target hole, so that normal hole scanning can be realized, and the problem of relatively large deviation between the prior information of the hole site and the actual position is solved; and hole sweeping failure is caused.

Description

Technical Field

[0001] This application relates to the field of 3D scanning technology, and in particular to a hole position measurement method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the continuous development of sensor technology, 3D scanning equipment has been widely used in the automotive and aerospace industries due to its advantages of high accuracy and resolution. Currently, 3D scanning equipment is based on vision principles to scan and measure holes. It requires operating the 3D scanning equipment to capture images of the hole positions from different locations, and combining these images with prior information about the hole positions to obtain the actual 3D hole position information.

[0003] In practice, due to factors such as workpiece deformation, movement of testing equipment, and errors in workpiece loading and unloading, there is a large deviation between the prior information of the hole position and the actual position, which leads to the problem of subsequent hole scanning failure.

[0004] There is currently no effective solution to the problem that the prior information about the hole position in related technologies deviates significantly from the actual position, leading to hole scanning failure. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for measuring borehole position that can correct prior information about borehole position, in order to address the aforementioned technical problems.

[0006] Firstly, this embodiment provides a method for measuring hole position, including:

[0007] Acquire multiple frames of hole position images of the target hole in the workpiece being tested, simultaneously captured by at least two cameras;

[0008] Based on the hole location image, the prior information of the target hole is corrected to obtain the target prior information;

[0009] Edge detection is performed based on the prior information of the target. After three-dimensional reconstruction of the edge point coordinates extracted by edge detection, the three-dimensional data of multiple frames of the hole position image are fitted to obtain the hole position information of the target hole.

[0010] In some embodiments, the step of correcting the prior information of the target hole based on the hole location image to obtain the target prior information includes:

[0011] The pre-constructed reference object and the aperture image are fitted and aligned to calculate the first transformation matrix;

[0012] Based on the first transformation matrix, the prior information in the reference object coordinate system is corrected to obtain the target prior information.

[0013] In some embodiments, the method further includes:

[0014] Establish a reference object for the workpiece to be tested;

[0015] The prior information and the reference object are fitted and aligned to calculate the second transformation matrix.

[0016] Based on the second transformation matrix, the prior information is transformed in coordinate system to obtain the prior information in the coordinate system of the reference object.

[0017] In some embodiments, the step of correcting the prior information of the target hole based on the hole location image to obtain the target prior information includes:

[0018] The region of interest in the hole position image is determined based on the parameter information of the target hole in the prior information; the center of the target hole is projected in multiple frames of the hole position image;

[0019] Three-dimensional reconstruction is performed based on the edge points in the region of interest, and the prior information of the target is obtained by fitting.

[0020] In some embodiments, the step of performing 3D reconstruction based on edge points in the region of interest and fitting the prior information of the target includes:

[0021] Contour extraction is performed based on the region of interest in the hole location image to obtain edge points;

[0022] Based on the edge points, perform edge point matching to determine matching point pairs;

[0023] Based on the matching point pairs, three-dimensional reconstruction is performed, and the three-dimensional data corresponding to multiple frames of the hole location images are fitted to obtain the target prior information.

[0024] In some embodiments, the method further includes:

[0025] The search area of ​​the target hole is determined based on the target prior information and the hole position image, and the target point cloud data in the search area is determined based on preset conditions;

[0026] Based on the target point cloud data, a constraint plane is fitted to obtain the constraint plane;

[0027] The target hole of the target prior information is projected onto the constraint plane to obtain the constrained target prior information.

[0028] In some embodiments, the step of performing edge detection based on the target prior information, reconstructing the three-dimensional coordinates of the edge points extracted by the edge detection, and fitting the three-dimensional data of multiple frames of the hole location images to obtain the hole location information of the target hole includes:

[0029] Based on the center and normal of the target hole in the prior information of the target, the target sub-pixel edge points are searched in the hole position image;

[0030] Based on the constraint plane and camera parameters, the target sub-pixel edge points are back-projected onto the constraint plane to obtain three-dimensional data points;

[0031] By fusing the three-dimensional data points corresponding to the hole positions in multiple frames of the hole position images, the hole position information of the target hole is obtained.

[0032] In some embodiments, the method further includes:

[0033] When it is determined that the deviation between the prior information of the target hole and the actual position is greater than a preset threshold, the prior information of the target hole is corrected.

[0034] Secondly, this embodiment provides a hole position measuring device, including:

[0035] The prior information correction module is used to acquire multiple frames of hole position images of the target hole simultaneously captured by at least two cameras; and to correct the prior information of the target hole based on the hole position images to obtain the target prior information.

[0036] The hole position measurement module is used to perform edge detection based on the target prior information, and after performing three-dimensional reconstruction on the edge point coordinates extracted by edge detection, fit the three-dimensional data of multiple frames of the hole position image to obtain the hole position information of the target hole.

[0037] Thirdly, this embodiment provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the hole position measurement method described in the first aspect above.

[0038] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the hole position measurement method described in the first aspect above.

[0039] Compared with related technologies, the hole position measurement method, apparatus, computer equipment, and storage medium provided in this embodiment acquire multiple frames of hole position images of the target hole in the workpiece to be measured by simultaneously capturing images from at least two cameras; correct the prior information of the target hole based on the hole position images to obtain target prior information; perform edge detection based on the target prior information; reconstruct the three-dimensional coordinates of the edge points extracted by edge detection; and fit the three-dimensional data of the multiple frames of hole position images to obtain the hole position information of the target hole. Through this embodiment, hole position images can be captured during scanning, the prior information of the hole can be corrected using the hole position images, and then three-dimensional reconstruction can be performed based on the corrected target prior information to obtain the hole position information of the target hole. This enables normal hole scanning and solves the problem of large deviations between the prior information and the actual position of the hole, leading to scanning failure.

[0040] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 This is a hardware structure block diagram of the terminal of the hole position measurement method in one embodiment;

[0043] Figure 2 This is a flowchart of a hole position measurement method in one embodiment;

[0044] Figure 3 This is a flowchart of a hole position measurement method in another embodiment;

[0045] Figure 4 This is a flowchart of the hole position measurement method in another embodiment;

[0046] Figure 5 This is a structural block diagram of the hole position measuring device in one embodiment.

[0047] In the diagram: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 10, prior information correction module; 20, hole position measurement module. Detailed Implementation

[0048] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0050] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the hole position measurement method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0051] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the hole position measurement method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0053] This embodiment provides a method for measuring hole position. Figure 2 This is a flowchart of the hole position measurement method in this embodiment, as follows: Figure 2 As shown, the method includes the following steps:

[0054] Step S201: Acquire multi-frame hole position images of the target hole of the workpiece to be tested by at least two cameras simultaneously.

[0055] Specifically, there are at least two cameras, each capable of independently capturing multiple images of the target hole, but synchronized shooting is required. For example, a single processor can control each camera to capture images simultaneously. Alternatively, each camera can have a built-in timer, with the timer synchronized to control simultaneous shooting. In other embodiments, each hole image can be timestamped, and the timestamps can be used to determine whether the images were captured synchronously. Other implementation methods are not detailed here.

[0056] Step S202: Correct the prior information of the target hole based on the hole position image to obtain the target prior information.

[0057] Specifically, for the workpiece to be tested, prior information of all target holes can be obtained by coarse scanning of the target holes or by pre-defining the information. Prior information includes, but is not limited to, the center, normal, and radius of circular holes; the center, normal, major axis normal, height, and width of slots and rectangular holes; and the center, normal, vertex direction, and side length of hexagonal holes.

[0058] Based on the hole position images obtained during the scanning process, prior information is corrected to obtain target prior information. There are two main approaches: one is to pre-construct a reference object and coordinate system for the workpiece to be measured, and then fit and align the hole position images with the reference object to transform the prior information onto the data of this scan, thus obtaining the target prior information. This approach requires the establishment of a reference object, so it is suitable for hole position measurement and correction of tracking scanning equipment. The other approach does not require the establishment of a reference object. Instead, a coordinate system is established using the marked points on the workpiece to be measured during the scanning process. By determining the region of interest in the hole position image, three-dimensional reconstruction is performed based on the region of interest, and the target prior information is obtained by fitting. This method does not require the establishment of a reference object, so it is suitable for hole position measurement and correction of handheld tracking scanning equipment.

[0059] It should be noted that even if a reference object for the workpiece under test is established, a coordinate system can still be established using the marked points on the workpiece during the scanning process.

[0060] Step S203: Based on the prior information of the target, edge detection is performed. After three-dimensional reconstruction of the edge point coordinates extracted by edge detection, the three-dimensional data of multiple frames of hole position images are fitted to obtain the hole position information of the target hole.

[0061] Specifically, edge points of the target hole in the hole location image are extracted by edge detection. After back-projection of the edge points for 3D reconstruction, the 3D data of the 3D reconstruction in multiple frames of hole location images are fused to obtain the hole location information of the target hole. This can fuse the 3D reconstructed point cloud data and the hole location image to obtain high-precision hole location information.

[0062] Furthermore, in some embodiments, to avoid the case of a missed hole where the target prior information of the target hole does not match the actual hole position, the target prior information can be constrained by a fitted constraint plane.

[0063] Through the above steps, hole position images can be captured during scanning. The prior information of the hole can be corrected using the hole position images. Then, three-dimensional reconstruction can be performed based on the corrected target prior information. This can accurately and stably reconstruct the features of the hole and obtain the hole position information of the target hole, thus enabling normal hole scanning. This solves the problem that the prior information of the hole position has a large deviation from the actual position, which leads to hole scanning failure.

[0064] In some embodiments, step S202 above, which corrects the prior information of the target hole based on the hole location image to obtain the target prior information, includes the following steps:

[0065] The pre-constructed reference object and hole position image are fitted and aligned to calculate the first transformation matrix; the prior information in the reference object coordinate system is corrected based on the first transformation matrix to obtain the target prior information.

[0066] The prior information in the reference object coordinate system is obtained through the following steps:

[0067] Establish a reference object for the workpiece to be tested; fit and align the prior information and the reference object to calculate the second transformation matrix; perform coordinate system transformation on the prior information based on the second transformation matrix to obtain the prior information in the coordinate system of the reference object.

[0068] Specifically, the results obtained from fitting point clouds or extracting image contours during coarse scanning are used as prior information for the target hole. The reference object can be a CAD model of the workpiece under test or a model encapsulated after a single point cloud scan. Using the prior information and features on the reference object, the prior information and the reference object are fitted and aligned. Specifically, the transformation matrix (second transformation matrix) between the prior information and the reference object point clouds is solved through optimal fitting alignment. Then, based on the second transformation matrix, the prior information is transformed into the reference object coordinate system to bind the prior information to the reference object. Since the original prior information (especially that extracted from images or coarse scans) is usually of low precision and may contain distortion, noise, or systematic errors, while the reference object (CAD or encapsulated model) represents a more accurate and complete target geometry, the optimal alignment fitting seeks a transformation (translation and / or rotation) that minimizes the difference (e.g., point-to-point, point-to-surface distance) between the transformed prior information point cloud and the reference object point cloud in the corresponding region (e.g., around the hole feature). The high-precision reference object is used to correct the spatial position and orientation of the low-precision prior information.

[0069] During the scanning process, the hole position image obtained from the scan is best-fitted and aligned with the reference object. Then, the transformation matrix obtained from the best-fit alignment (the first transformation matrix) is applied to the prior information in the reference object coordinate system. Based on the first transformation matrix, the prior information in the reference object coordinate system is corrected to obtain the target prior information. In the best-fit alignment, the point cloud data of the hole position image and the reference object are first downsampled to reduce the amount of data and improve the efficiency of subsequent calculations. The downsampling can be done by voxel grid filtering, random sampling, and uniform sampling. Further, through iterative optimization, the distance from the source point (reference object point cloud) to the plane containing the target point (hole position image point cloud) is minimized, and the alignment accuracy is improved by using normal vector information to obtain the first transformation matrix.

[0070] This further aligns the prior information with the reference object after scanning, based on the previous static binding that binds the prior information with the hole position image that reflects the real-time state of the workpiece under test. This ensures that the prior information in the reference object coordinate system and the actual position of the target hole are close to each other in a dynamically changing real environment.

[0071] The prior information correction scheme based on the reference coordinate system provided in this embodiment can effectively resist the influence of local point cloud noise or missing information by using the reference object for registration. It ensures that the prior information of the hole is always kept in the approximate correct area of ​​the workpiece. It is suitable for scenarios where high-quality CAD models or initial point cloud scans can be obtained without relying on marker points or where marker points may fail.

[0072] In some embodiments, step S201 above, which corrects the prior information of the target hole based on the hole location image to obtain the target prior information, includes the following steps:

[0073] The region of interest in the borehole image is determined based on the parameter information of the target borehole in the prior information; the center projection of the target borehole is in the multi-frame borehole image.

[0074] Specifically, in this embodiment, a coordinate system is established using marker points on the workpiece to be measured. Since the same marker point data is used during the scanning process, optimal fitting and alignment cannot be performed on this basis. Therefore, the center of the target hole is projected onto the hole position images of at least two cameras. If the projected center of the target hole falls onto the hole position image, a small region of interest is determined on the hole position image based on parameters such as the center and radius of the target hole. If the projected center of the target hole does not fall onto the hole position image, the hole position image is no longer processed and is considered invalid. Determining the region of interest improves computational efficiency, eliminates external interference, focuses on local high signal-to-noise ratio areas, and automatically filters invalid viewpoints (i.e., target holes that do not fall onto the hole position image after projection), ensuring the reliability of the output results.

[0075] The parameter information of the target hole (center, radius, etc.) can be obtained by methods such as prior information of the target hole, CAD model, and point cloud data fitting.

[0076] Three-dimensional reconstruction is performed based on edge points in the region of interest, and prior information of the target is obtained by fitting.

[0077] Specifically, contour extraction is performed based on the region of interest in the borehole image to obtain edge points; edge point matching is performed based on the edge points to determine matching point pairs; and three-dimensional reconstruction is performed based on the matching point pairs to fit the three-dimensional data corresponding to multiple frames of borehole images and obtain the target prior information.

[0078] Specifically, for the region of interest in the borehole location image, contour extraction is performed based on the prior information of the target borehole to obtain the edge points of the contour. The features of these edge points are then used to match points in multiple simultaneously captured borehole location images to determine matching point pairs. These matching point pairs are then used for 3D reconstruction to obtain 3D data, which is then fitted to the 3D data from the multiple borehole location images to obtain the target prior information.

[0079] One method for 3D reconstruction is to estimate the spatial position of a 3D point by calculating the common perpendicular between the camera rays (lines of sight) connecting the matching point pairs and taking the midpoint (or weighted midpoint).

[0080] The modified prior information scheme based on region of interest reconstruction provided in this embodiment establishes a coordinate system using marker points, eliminating the need for a reference object and thus offering higher tolerance for workpiece deformation. Furthermore, determining the region of interest for local 3D reconstruction places lower demands on the quality of the global point cloud. Even if parts of the workpiece are occluded or the point cloud quality is poor, the system can still function as long as the hole is visible within the region of interest, reducing computational load to some extent. This is particularly suitable for scenarios relying on fixed marker points for positioning.

[0081] In some embodiments, to avoid the case of a missed hole where the target prior information of the target hole does not match the actual hole position, the target prior information can be constrained using a fitted constraint plane. The above method also includes the following steps:

[0082] The search area of ​​the target hole is determined based on the target prior information and hole position image, and the target point cloud data in the search area is determined based on preset conditions; a constraint plane is obtained by fitting the target point cloud data; the target hole of the target prior information is projected onto the constraint plane to obtain the constrained target prior information.

[0083] Specifically, the search area can be determined based on prior information about the target (such as center, radius, and normal), and this search area can be spherical. Within the search area, points are filtered based on preset conditions to retain valid point cloud data as the target point cloud data. These preset conditions can include, for example, removing points significantly higher or lower than the surface where the hole is located (e.g., threshold ±2mm) based on height constraints, retaining points where the angle between the normal vector and the theoretical normal is less than a threshold based on normal consistency, and removing points with excessive curvature based on curvature.

[0084] Based on the target point cloud data, a constraint plane is obtained by fitting using the least squares method. The constraint plane is used to constrain the normal and position of the target hole in the target prior information. The target hole is discretized into points and projected onto the constraint plane to correct the position of the target hole into the constraint plane, thus obtaining the constrained target prior information.

[0085] By using a constraint plane to constrain the target prior information in this embodiment, the "flying hole" situation can be effectively avoided, making the target hole in the target prior information more consistent with the actual point cloud.

[0086] In some embodiments, step S203 above involves edge detection based on prior target information, followed by 3D reconstruction of the edge point coordinates extracted by edge detection, and fitting the 3D data of multiple frames of hole location images to obtain the hole location information of the target hole. This includes the following steps:

[0087] Based on the center and normal of the target hole in the prior information of the target, the sub-pixel edge points of the target are searched in the hole position image; based on the constraint plane and camera parameters, the sub-pixel edge points of the target are back-projected onto the constraint plane to obtain three-dimensional data points; the three-dimensional data points corresponding to multiple frames of hole position images are fused to obtain the hole position information of the target hole.

[0088] Specifically, by projecting the center of the target hole onto the plane of the hole image, the corresponding search starting point can be obtained. An offset point is taken in 3D space along the normal of the target hole, and the offset point is projected onto the plane of the hole image, thereby generating the search direction.

[0089] Based on the search starting point and search direction, a search path is established by extending along the search direction and its perpendicular direction with the search starting point as the center. The image gradient is calculated along the search path, and the sub-pixel point where the gradient is maximum is obtained in the hole position image, which can further improve the edge accuracy to the pixel level.

[0090] An observation ray is constructed based on the camera parameters, and the intersection points of the ray and the constraint plane are calculated to obtain 3D data points. The camera parameters include an intrinsic parameter matrix and an extrinsic parameter matrix that transforms the world coordinate system to the camera coordinate system.

[0091] Finally, the three-dimensional data points corresponding to the multi-frame borehole images are registered and fused to obtain the borehole location information of the target borehole.

[0092] The constraint plane can be obtained in the following way:

[0093] The search area for the target hole is determined based on the target prior information and hole position image, and the target point cloud data in the search area is determined based on preset conditions; the constraint plane is obtained by fitting based on the target point cloud data.

[0094] In this embodiment, the corrected target prior information is used to search for sub-pixel edge points, and finally the three-dimensional data points corresponding to multiple frames of hole position images are fused to obtain hole position information with high accuracy and consistent with the actual position of the target hole, thereby realizing hole position measurement.

[0095] In some embodiments, the method further includes:

[0096] When the deviation between the prior information of the target hole and its actual position is greater than a preset threshold, the prior information of the target hole is corrected.

[0097] Specifically, different preset thresholds are set for different parameters of the target hole, such as setting thresholds for deviations in center, normal, and hole diameter.

[0098] The actual position here refers to the actual hole position of the target hole on the workpiece to be measured. This can be obtained before scanning based on the actual situation. If the deviation between the prior information and the actual position of the target hole is greater than a preset threshold, it indicates a significant discrepancy. Therefore, the prior information is corrected using the method described in the above embodiments before hole position detection. If the deviation between the prior information and the actual position of the target hole is less than a preset threshold, prior information correction can be omitted, and hole position measurement can be performed directly, thus reducing computational load.

[0099] The present embodiment will now be described and illustrated through preferred embodiments.

[0100] Figure 3 This is a flowchart of the hole position measurement method in this embodiment, as follows: Figure 3 As shown, the method includes the following steps:

[0101] Step S301: Acquire multi-frame hole position images of the target hole of the workpiece to be tested by at least two cameras simultaneously.

[0102] Step S302: Establish a reference object for the workpiece to be tested.

[0103] Step S303: Fit and align the prior information of the target hole with the reference object to calculate the second transformation matrix; perform coordinate system transformation on the prior information based on the second transformation matrix to obtain the prior information in the coordinate system of the reference object.

[0104] Step S304: Fit and align the pre-constructed reference object and hole position image to calculate the first transformation matrix; correct the prior information in the reference object coordinate system based on the first transformation matrix to obtain the target prior information.

[0105] Step S305: Determine the search area of ​​the target hole based on the target prior information and the hole position image, and determine the target point cloud data in the search area based on preset conditions; obtain the constraint plane based on the target point cloud data; project the target hole of the target prior information onto the constraint plane to obtain the constrained target prior information.

[0106] Step S306: Based on the center and normal of the target hole in the constrained prior information of the target, the target sub-pixel edge points are searched in the hole position image; based on the constraint plane and camera parameters, the target sub-pixel edge points are back-projected onto the constraint plane to obtain three-dimensional data points; the three-dimensional data points corresponding to multiple frames of hole position images are fused to obtain the hole position information of the target hole.

[0107] The prior information correction scheme based on a reference object in this embodiment, which utilizes the reference object for registration, can effectively resist the influence of local point cloud noise or missing data, exhibiting high stability. Simultaneously, dynamic projection and constrained plane projection can essentially ensure that the hole center point falls near the actual hole and on its corresponding plane. Finally, sub-pixel edge extraction and multi-frame fitting are used to obtain the hole position information, guaranteeing the final reconstruction accuracy. This approach is suitable for scenarios where relying on marker points is unnecessary or where marker points may be ineffective.

[0108] Figure 4 This is a flowchart of another hole position measurement method in this embodiment, such as... Figure 4 As shown, the method includes the following steps:

[0109] Step S401: Acquire multi-frame hole position images of the target hole of the workpiece to be tested by at least two cameras simultaneously.

[0110] Step S402: Determine the region of interest in the hole position image based on the parameter information of the target hole in the prior information of the target hole; the center projection of the target hole is in the multi-frame hole position image.

[0111] Step S403: Extract contours based on the region of interest in the borehole image to obtain edge points; perform edge point matching based on the edge points to determine matching point pairs; perform 3D reconstruction based on the matching point pairs, fit the 3D data corresponding to multiple frames of borehole images, and obtain target prior information.

[0112] Step S404: Determine the search area of ​​the target hole based on the target prior information and the hole position image, and determine the target point cloud data in the search area based on preset conditions; obtain the constraint plane based on the target point cloud data; project the target hole of the target prior information onto the constraint plane to obtain the constrained target prior information.

[0113] Step S405: Based on the center and normal of the target hole in the constrained prior information of the target, the target sub-pixel edge points are searched in the hole position image; based on the constraint plane and camera parameters, the target sub-pixel edge points are back-projected onto the constraint plane to obtain three-dimensional data points; the three-dimensional data points corresponding to multiple frames of hole position images are fused to obtain the hole position information of the target hole.

[0114] The scheme in this embodiment, which uses a modified prior information based on region of interest (ROI) reconstruction, establishes a coordinate system using marker points, eliminating the need for a reference object and thus offering higher tolerance for workpiece deformation. Simultaneously, the determination of the ROI and the constraint of planar projection effectively prevent fly-through holes. This local reconstruction has lower requirements for global point cloud quality; even if parts of the workpiece are occluded or the point cloud quality is poor, it can still function as long as the hole is visible within the ROI, reducing computational load to some extent. This is particularly suitable for scenarios relying on fixed marker points for positioning. Finally, sub-pixel edge extraction and multi-frame fitting are used to obtain hole location information, ensuring final reconstruction accuracy. This approach is suitable for scenarios that do not require marker points or where marker points may be ineffective.

[0115] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0116] This embodiment also provides a hole position measuring device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that achieve a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0117] Figure 5 This is a structural block diagram of the hole position measuring device in this embodiment, as shown below. Figure 5 As shown, the device includes:

[0118] The prior information correction module 10 is used to acquire multiple frames of hole position images of the target hole simultaneously captured by at least two cameras; and to correct the prior information of the target hole based on the hole position images to obtain the target prior information.

[0119] The hole position measurement module 20 is used to perform edge detection based on the target prior information. After performing three-dimensional reconstruction on the edge point coordinates extracted by edge detection, the three-dimensional data of multiple frames of hole position images are fitted to obtain the hole position information of the target hole.

[0120] The device provided in this embodiment can capture hole position images during scanning, use the hole position images to correct the prior information of the hole, and then perform three-dimensional reconstruction based on the corrected target prior information. This can accurately and stably reconstruct the features of the hole, obtain the hole position information of the target hole, and thus enable normal hole scanning. This solves the problem that the prior information of the hole position has a large deviation from the actual position, which leads to hole scanning failure.

[0121] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0122] This embodiment also provides a computer device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0123] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0124] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0125] Furthermore, in conjunction with the hole position measurement method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the hole position measurement methods described in the above embodiments.

[0126] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0127] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0128] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or alternative to other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for measuring hole position, characterized in that, include: Acquire multiple frames of hole position images of the target hole in the workpiece being tested, simultaneously captured by at least two cameras; Based on the hole location image, the prior information of the target hole is corrected to obtain the target prior information; Edge detection is performed based on the prior information of the target. After three-dimensional reconstruction of the edge point coordinates extracted by edge detection, the three-dimensional data of multiple frames of the hole position image are fitted to obtain the hole position information of the target hole.

2. The hole position measurement method according to claim 1, characterized in that, The step of correcting the prior information of the target hole based on the hole position image to obtain the target prior information includes: The pre-constructed reference object and the aperture image are fitted and aligned to calculate the first transformation matrix; Based on the first transformation matrix, the prior information in the reference object coordinate system is corrected to obtain the target prior information.

3. The hole position measurement method according to claim 2, characterized in that, The method further includes: Establish a reference object for the workpiece to be tested; The prior information and the reference object are fitted and aligned to calculate the second transformation matrix. Based on the second transformation matrix, the prior information is transformed in coordinate system to obtain the prior information in the coordinate system of the reference object.

4. The hole position measurement method according to claim 1, characterized in that, The step of correcting the prior information of the target hole based on the hole position image to obtain the target prior information includes: The region of interest in the hole position image is determined based on the parameter information of the target hole in the prior information; the center of the target hole is projected in multiple frames of the hole position image; Three-dimensional reconstruction is performed based on the edge points in the region of interest, and the prior information of the target is obtained by fitting.

5. The hole position measurement method according to claim 4, characterized in that, The step of performing 3D reconstruction based on edge points in the region of interest and fitting the prior information of the target includes: Contour extraction is performed based on the region of interest in the hole location image to obtain edge points; Based on the edge points, perform edge point matching to determine matching point pairs; Based on the matching point pairs, three-dimensional reconstruction is performed, and the three-dimensional data corresponding to multiple frames of the hole location images are fitted to obtain the target prior information.

6. The hole position measurement method according to any one of claims 1 to 5, characterized in that, The method further includes: The search area of ​​the target hole is determined based on the target prior information and the hole position image, and the target point cloud data in the search area is determined based on preset conditions; Based on the target point cloud data, a constraint plane is fitted to obtain the constraint plane; The target hole of the target prior information is projected onto the constraint plane to obtain the constrained target prior information.

7. The hole position measurement method according to claim 6, characterized in that, The step of performing edge detection based on the target prior information, reconstructing the three-dimensional coordinates of the edge points extracted by edge detection, and fitting the three-dimensional data of multiple frames of the hole location images to obtain the hole location information of the target hole includes: Based on the center and normal of the target hole in the prior information of the target, the target sub-pixel edge points are searched in the hole position image; Based on the constraint plane and camera parameters, the target sub-pixel edge points are back-projected onto the constraint plane to obtain three-dimensional data points; By fusing the three-dimensional data points corresponding to the hole positions in multiple frames of the hole position images, the hole position information of the target hole is obtained.

8. The hole position measurement method according to claim 1, characterized in that, The method further includes: When it is determined that the deviation between the prior information of the target hole and the actual position is greater than a preset threshold, the prior information of the target hole is corrected.

9. A hole position measuring device, characterized in that, include: The prior information correction module is used to acquire multi-frame images of the target hole taken simultaneously by at least two cameras; Based on the hole location image, the prior information of the target hole is corrected to obtain the target prior information; The hole position measurement module is used to perform edge detection based on the target prior information, and after performing three-dimensional reconstruction on the edge point coordinates extracted by edge detection, fit the three-dimensional data of multiple frames of the hole position image to obtain the hole position information of the target hole.

10. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the borehole measurement method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the borehole measurement method according to any one of claims 1 to 8.

Images